performance issue

As is noted in the project description, this library is way slower than the standard json library.

Is there plan to improve performance? What’s the design and timeline?

[dpranke]

Hi!

Great question (and sorry for the delay in replying).

This project is mostly a test case for parser generator I’m working on, and so raw speed is less important to me than being able to actually generate a correct parser cleanly from a grammar. That said, a generated grammar this slow is almost useless, so I do need to make sure the generator generates much better code.

Unsurprisingly, the builtin json module has both been tuned to be as fast as possible in pure Python, and has a C extension to be even faster. While I might eventually be able to generate a C extension using the parser generator, that’s a ways off on the roadmap.

The pure python builtin JSON parser is 6-10x slower than the C parser; currently, I think my generated code is about 160x - 200x slower than the builtin pure python version.

I think I can get the generated code to be a lot better, and the current dev version I have on the parser can produce a JSON parser that is only 10x slower than the pure python version; this appears to be comparable to what you’d get with PLY and significantly faster than an ANTLR-generated JSON parser. I expect I should be able to match this for JSON5 without too much more work.

That’s a long way of saying I expect I can get this 10-20x faster fairly soon now, maybe another week or two depending on how much free time I can find.

I don’t know how hard it will be to get that last 10x, to match the pure python implementation, or even if it’s doable at all, and I have no ETA or even a real expectation that I’ll be able to match the speed of the native JSON implementation, but I do plan to keep working on it and I will post updates with my progress.

Hopefully that answers your question, probably in much more detail than you were hoping for. I hope the answer is not too disappointing.

[Greedquest]

FWIW https://github.com/Kijewski/pyjson5 looks like a good alternative which is written with Cython and so is pretty fast (3 orders of magnitude on loads for a small document). Although I’ve only come across either of them today so I have no idea if their APIs are equivalent or anything