Version 2 discussion

The issue is for brainstorming what we can do in a 2.x version of Parsimmon to make it easy to port code from 1.x but clean up any rough edges that have evolved since the library’s inception.

Ideas

Change our official supported environments

I’ll be honest. I haven’t tested any releases in IE7, and I don’t plan on doing that ever either. Setting up browser testing is expensive and probably not something I’ll end up doing for the project.

I think it would be best to state that we only routinely supported Node.js LTS releases, but that it should work in ES5 browsers.

Switch to TypeScript

TypeScript is really good at this point and I’m confident we could statically type probably all of Parsimmon, or at least enough of it that it’s really helpful and maybe redesign some things that don’t statically type well.

The current static types contributed by the DefinitelyTyped project are a bit out of date, and are going to be even more inaccurate once version 2 comes out.

Also this means we can use some modern features like classes and arrow functions but still compile down to ES5 for maximum support.

Rethink the documentation

The main problem with the current documentation is that it’s referenced straight from the master branch in the repo, so people can see documentation for features that have not been released yet.

The other problem is that it’s not as well organized as it could be since it’s a simple markdown file. If we turned it into a web page we could make it much easier to read.

I think the best way to do it would be to use TypeDoc to generate a JSON file from code comments and write a small website to use that. This is the strategy I used with Ramda (except that I used JSDoc there not TypeDoc).

Breaking API changes

Generally just think about all the breaking change issues here and if they still make sense and if there’s any other little tweaks like this.

Debugging

Parsimmon parsers are hard to debug. The callstack is filled with just Parsimmon._ functions the whole way usually, with no good way to tell where parser construction failed.

Tracking the flow of parsing is also difficult because no tools are provided.

See https://github.com/jneen/parsimmon/issues/193 and https://github.com/jneen/parsimmon/issues/143 for debugging ideas.

[anko]

More related experimental thoughts on debugging, which I’m not sure if I wrote down last time it came up:

I wrote a p.debug combinator for partser which prints ANSI-coloured terminal output, with what I guess is a similar structure to @hillin’s RDT thing, but more compact?

const parser = p.times(
  p.alt([p.string('ba'), p.string('na')]),
  0, 3)
const parserWithDebug = p.debug(parser)
const result = parserWithDebug('banana')
console.log(result)

Can wrap it around any parser, and it prints everything that happens in that “sub-tree”. The ? lines are printed when a parser is entered, and others when a parser is exited. The short window on the very left shows the current index in grey when entering a parser, and the consumed or rejected characters in green or red when a parser exits, with a configurable number of input stream context on either side.

Of course this only works in a console, and it’s extremely noisy if used at top-level on a deeply nested real-world parser, but this is the best I could come up with for use in non-interactive console context. I think it’s clearer than JSON logs, but I’m not sure by how much. 😅 It’s effectively a continuous call stack visualisation, but filtered to just parser calls, and with corresponding context on parse progress.

The main commit for the implementation was https://github.com/anko/partser/commit/f2cdec49853bbc88a186ab0888fc938cbec77bba. In short, I added an optional debug handler function to parser calls, which is called before and after parse logic; https://github.com/anko/partser/blob/870216a31c7f0943db8806b79c99b13d89e15c48/index.js#L76-L81. Parsers pass that handler function around when they call each other. All the fancy coloured indented terminal stuff is just the default debug handler function, which users can override with their own function (e.g. https://github.com/anko/partser/blob/870216a31c7f0943db8806b79c99b13d89e15c48/test.js#L856-L873) to handle the debug information in a different format.

It’s nifty, but I’m not sure if it’s actually a good idea.

• Do big pretty-printing routines really belong in a parser library, or should it be restricted to just the un-opinionated “pass a debug handler function” API?
• Could we instead be pre-empting further “pass data around between parsers” needs by just adding an optional user-data object to .parse calls (which e.g. P.map and P.chain parsers could access), and implement debugging through that?

[wavebeem]

I’ve spent some time over the last month working on a new library called bread-n-butter (bnb), a spiritual successor to parsimmon, that addresses some of these concerns:

👉 https://wavebeem-bread-n-butter.netlify.app

I’ve written a point-by-point summary of how Parsimmon v2 thoughts influenced the design of bnb below.

My question to anyone in this thread with the time to look at stuff is, would something like bnb be of interest to you? Parsimmon has a lot of little functions that don’t seem that useful in it. Sort of like how Lodash has tripled in size since it was forked from Underscore… but most people are still using the same few functions from it.

A quick note on the life of Parsimmon

This is not me announcing that Parsimmon is dead or on life support, but I have felt a little trapped by the current state of Parsimmon. bnb has allowed me to work on parser combinators with a fresh perspective.

I want to test the waters with what people want to see out of v2. Parsimmon seems to be the only major player in the parser combinators space in JS, so I don’t want to just release a version 2 that makes everyone mad.

Change our official supported environments

bnb requires ES2015 suppport.

Switch to TypeScript

No more out of date types, no overhead on library users to contribute back to DefinitelyTyped. This was also really helpful while developing the library as well.

Rethink the documentation

I tried using TypeDoc, but the resulting website was just not as well organized as I would like. So while I’m still providing brief TSDoc comments so IDE users can get descriptions populated in their intellisense, most documentation lives in markdown files and is rendered by 11ty.

There’s still the problem that the master branch is what’s shown on the website, so it’s possible to see docs for a feature before it exists in npm.

Breaking API changes

I shrunk the API drastically. parsimmon supports a lot of niche features (fantasy land, binary parsing, built-in parsers that people normally replace, character based parsers like oneOf or noneOf or test instead of just using regexp).

I could probably stand to add a few more convenience methods in, but overall it feels fairly comfortable to work in already, and is a much smaller library.

Custom parsers are even easier to write now, in the rare occasion you should need to.

bnb has 8 value exports, 9 type exports, and 16 methods or properties on the Parser class.

parsimmon has over 50 value exports, 36 parser methods (and several odd aliases for fantasy land compat)

Debugging

I still haven’t dug too deeply into this. One thing holding parsimmon back was IE7 support meant we couldn’t even rely on console.log existing (though I guess we could’ve done feature tests for this).

For now, I think .map((x) => { console.log("...", x); return x; }) is acceptable for debugging, but I might add in .tap to simplify that.

I’ll be honest that I still haven’t spent the time to look into the Scala parser combinators debugging support to figure out what kind of stuff we could add.