How to parse with the API: docs outdated?

I already have managed to port one of my parsing tasks to nearley (see https://github.com/loveencounterflow/mojikura-idl/blob/nearley/src/idlx.nearley etc), but this time around i seem to be doing something wrong.

For easier reference, I’ve uploaded a simple demo project to https://github.com/loveencounterflow/nearley-examples. To wit:

The grammar I use here is the JSON grammar from the nearley/examples folder; that I compiled with nearleyc grammars/json.ne -o grammars/json.js. Then, I copy-pasted (and adapted) this code from the README:

// from-the-docs.js
var grammar = require("./grammars/json.js");
var nearley = require("nearley");
var p = new nearley.Parser(grammar.ParserRules, grammar.ParserStart);
p.feed("[1,2,3]");
console.log( require( 'util' ).inspect( p.results ) );

But this fails:

$ ./run-example-from-the-docs.sh                 
xxx/nearley-examples/node_modules/nearley/lib/nearley.js:323
            throw err;
            ^

Error: invalid syntax at line 1 col 2:

  [1,2,3]
   ^
Unexpected "1"

    at Parser.feed (xxx/nearley-examples/node_modules/nearley/lib/nearley.js:320:23)
    at Object.<anonymous> (xxx/nearley-examples/from-the-docs.js:8:3)
...

OTOH, I can run nearleyc grammars/json.ne -o grammars/json.js && nearley-test grammars/json.js -i "[1,2,3]":

$ ./run-example-with-nearley-test.sh
...
1: {json$subexpression$1 → array ● }, from: 0
2: {json → _ json$subexpression$1 ● _}, from: 0
3: {_ →  ● }, from: 7
4: {_ →  ● %space}, from: 7
5: {json → _ json$subexpression$1 _ ● }, from: 0


Parse results: 
[ [ 1, 2, 3 ] ]

Did I miss something here?

BTW when studying the JSON example, I realized it uses the moo parser; I guess the docs would definitely profit from a link to that software, as it fits so nicely into the nearley setup.

[deltaidea]

Yeah, I’ve seen this problem before. Solution:

- var p = new nearley.Parser(grammar.ParserRules, grammar.ParserStart);
+ var p = new nearley.Parser(grammar.ParserRules, grammar.ParserStart, { lexer: grammar.Lexer });

Without this, nearley uses its default parser instead of moo because it doesn’t know about your custom one. Look at the compiled json.js, it exports { Lexer, ParserRules, ParserStart }.

You might ask, “Then what’s the point of @lexer and inlining lexer into .ne”? And the answer is that you’re right, there’s no point. You can have your lexer = moo.compile() in a separate file and pass it into new nearley.Parser() just like now.

This is an issue with docs. 😢 I’ll see if I can fix that when I have some time.

@Hardmath123 Alternatively, it might be a good idea to allow passing the whole grammar object as a single argument to new Parser(). Just duck-type that the first argument has ParserRules and ParserStart. This would make @lexer actually useful.

[loveencounterflow]

OK var p = new nearley.Parser(nearley.Grammar.fromCompiled(grammar)) looks better than what I’m doing now… should definitely get mention in the README, didn’t know about it.