Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Compile grammer regex too slow.

See original GitHub issue

DEBUG [grammar compile time] 0.6472580432891846
DEBUG [timer] Grammer created: 9.5367431640625e-07 from start.
 INFO iredis start, host=127.0.0.1, port=6379, db=0.
DEBUG Using selector: KqueueSelector
DEBUG [timer] First REPL: 0.04548501968383789 from start.
DEBUG REPL waiting for command...

Takes 0.64secnds to compile when startup, painful slow.

Issue Analytics

State:
Created 4 years ago
Comments:7 (6 by maintainers)

Top GitHub Comments

1reaction

laixintaocommented, Jan 12, 2020

@guyskk About 3), Jonath is working on this: https://github.com/prompt-toolkit/python-prompt-toolkit/pull/1047

1reaction

guyskkcommented, Jan 3, 2020

Pickle just dump the regex text and recompile when load back, seems disappointing. https://stackoverflow.com/questions/4037339/is-there-a-way-to-really-pickle-compiled-regular-expressions-in-python

(1) It’s possible(but has risk) to hack sre_compile.compile, I found line p = sre_parse.parse(p, flags) and code = _code(p, flags) are most time-consuming and the return value can be really-pickled.

(2) The prompt_toolkit prompt_toolkit.contrib.regular_languages.compiler.compile compile 7000+ regex(_CompiledGrammar._re_prefix_patterns), and it seems no way to reduce the quantity of regex.

(3) But if we check the implementation of _CompiledGrammar.match_prefix, it seems reasonable to merge all regex into one, werkzeug.routing.Rule.compile is an example.

Life is short, I choose wait 10 seconds… 😂

Top Results From Across the Web

c# - Why is my regex so much slower compiled than interpreted?

The initial construction of the Regex will be slower, because code is compiled on-the-fly, but each match should be faster. If your regular...

How slow are regular expressions? - Quora

Regex is slow because it's more complex than it seems. Every expression needs to be parsed and compiled (yes, compiled). Then, it's created...

JEP draft: Predictable regex performance - OpenJDK

Problem. There is an increasing focus on the performance of regular expression matching, and especially the predictability.

The Greatest Regex Trick Ever (2014) - Hacker News

Perl hacked support for this in via backreferences and other extensions, but these are slow and illegible. Use a proper context-free grammar ......

Reading 12 & 13: Regular Expressions & Grammars

This compiled form of a regular grammar is called a regular expression. ... Lexical analysis takes a stream of fine-grained, low-level symbols (e.g., ......