question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Change "$self" to "$parser" in "target agnostic grammars" instructions

See original GitHub issue

This is pretty critical.

The instructions for “Target agnostic grammars” are not correct. Following the third option, after adding field “self” to a base class, and using $self in an action, the Antlr tool will output "unknown attribute reference self in $self". In fact, there is no “$self” attribute. If one changes this to “self” (without the dollar sign), then the instructions would “work”. But, as fate sometimes works, I tried something else.

If one changes $self to $parser, skip adding a field “self” to the base class altogether, “target agnostic” for a parser grammar will work for all but two targets! Note, this is only good for parser grammars. I don’t think there is an equivalent “$lexer” for lexer actions, which really would be needed for actions in the lexer grammar. Or, really introduce “$self” for either lexer or parser actions.

The documentation for attributes really needs to include a discussion of “$parser”. I don’t know why it’s missing because this attribute works.

Note, it’s currently impossible to get a grammar with actions with a common syntax to work across all targets–at least for 4.9.3–but it is close using “$parser”–at least for a parser grammar. I would actually call the technique “not-quite target-agnostic grammars” because the common syntactic approach does not work for Cpp and PHP where there are include file issues and the non-conversion of “.” to “->” for a pointer reference (see attached zip file for a proof of the technique working for CSharp, Dart, Go, Java, JavaScript, Python3). I have not tested “target agnostic grammars” for the old Antlr4cs port, which people shouldn’t even be using at this point. target-agnostic2.zip

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Reactions:1
  • Comments:6 (6 by maintainers)

github_iconTop GitHub Comments

1reaction
RossPattersoncommented, Aug 4, 2022

While looking for something else in the ANTLR source, I found commit 38c4e2b0028d2a305d0cf05ce5669d23d67ea88e, from late 2014, which changed the name of the $self attribute to $parser. The doc had never caught up.

0reactions
kaby76commented, Aug 5, 2022

The new instructions in antlr4/doc/python-target.md look fine.

I do wish there was a “$lexer” for lexer grammar actions, but “$parser” works great at least for parser grammars. And, actually, because a few reasons, I still had to resort to a workaround:

  • In grammars-v4, for grammars requiring the superClass option, keep “this.” object references e.g., here.
  • Run a Python script to modify the grammar per target to what it should be, e.g., this
  • Provide a readme to instruct folks to run the script e.g., here.
  • Add a check to call the script in the build, e.g., here.

I had to resort to the workaround for several reasons–“target-agnostic grammars” isn’t really possible yet:

  • There’s no “$lexer.” object reference method call rewrite.
  • “$parser.” does not rewrite to “this->” for a generated C++ parser.
  • The Cpp target does not generate #include's of the superClass base classes. The only way for a compilation to work is to manually add @header {#include "baseparser.h"}, e.g. here.

Admittedly, it is a hack, but it fits into a build.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Syntax: language agnostic parser generator - Dmitry Soshnikov
In this post we'll briefly talk about the new language agnostic parser generator, called Syntax. Note: we don't describe here parsers algorithms in...
Read more >
Designing Language-Agnostic Code Transformation Engines
is dedicated to grammars, languages and parsing to understand how IRs ... by the engine, the user must convert the subtree by himself....
Read more >
Multi-Source Transfer of Delexicalized Dependency Parsers
We present a simple method for transferring dependency parsers from source languages with labeled training data to target languages.
Read more >
A Guide To Parsing: Algorithms And Terminology
An in-depth coverage of parsing terminology an issues, together with an explanation for each one of the major algorithms and when to use...
Read more >
Gramatron: Effective Grammar-Aware Fuzzing - HexHive
parse trees performing localized small-scale changes. ... It takes as input a grammar accepted by the fuzz target. Gramatron.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found