Improve the the blib2to3 grammar caching mechanism

I get an error upon launching Jupyter Lab with the jupyterlab_code_formatter extension enabled:

[I 10:37:25.686 LabApp] Node v10.8.0

[I 10:37:26.075 LabApp] Build is up to date
Generating grammar tables from C:\Users\%USERNAME%\Anaconda3\lib\site-packages\blib2to3\Grammar.txt
Writing grammar tables to C:\Users\%USERNAME%\AppData\Local\black\black\Cache\18.9b0\Grammar3.7.1.final.0.pickle
Writing failed: [Errno 2] No such file or directory: 'C:\\Users\\%USERNAME%\\AppData\\Local\\black\\black\\Cache\\18.9b0\\Grammar3.7.1.final.0.pickle'
Generating grammar tables from C:\Users\%USERNAME%\Anaconda3\lib\site-packages\blib2to3\PatternGrammar.txt
Writing grammar tables to C:\Users\%USERNAME%\AppData\Local\black\black\Cache\18.9b0\PatternGrammar3.7.1.final.0.pickle
Writing failed: [Errno 2] No such file or directory: 'C:\\Users\\%USERNAME%\\AppData\\Local\\black\\black\\Cache\\18.9b0\\PatternGrammar3.7.1.final.0.pickle'

After manually creating the folder C:\\Users\\%USERNAME%\\AppData\\Local\\black\\black\\Cache\\18.9b0 and relaunching Jupyter lab the error disappears (even though the folder seems to stay empty).

The error appears to originate from this line. It might be related to #192. I think this line fails because the cache directory does not exist, but I don’t understand why it doesn’t exist.

Other users confirmed this happens on linux and with version 19.10b as well in this issue.

I tried to find the cause of this issue in the extension, but it doesn’t seem to do anything out of the ordinary with the cache folder, so I was hoping someone here can tell me why the cache folder would not exist?

[VasuBhog]

I am still seeing this error with papermill 2.2.2 black 20.8b1 Mac OS: 10.15.7

Is there any update or workaround for this error? Upon papermill execution: Generating grammar tables from /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/blib2to3/Grammar.txt Writing grammar tables to /Users/xxx/Library/Caches/black/20.8b1/Grammar3.6.8.final.0.pickle Writing failed: [Errno 2] No such file or directory: '/Users/xxx/Library/Caches/black/20.8b1/tmpp31361aj' Generating grammar tables from /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/blib2to3/PatternGrammar.txt Writing grammar tables to /Users/xxx/Library/Caches/black/20.8b1/PatternGrammar3.6.8.final.0.pickle Writing failed: [Errno 2] No such file or directory: '/Users/xxx/Library/Caches/black/20.8b1/tmpcq8qn401'

[darachm]

I get this error trying to run papermill (which apparently imports black, hello) inside of a Singularity container.

• https://github.com/psf/black/issues/1143#issuecomment-730814683 does not work, likely because the container is built as root and then run as a user
• similarly, mkdir won’t work because the container is supposed to be able to be run by different users - hence different user-specific caches
• similarly, this is being used inside of Nextflow for some pipeline management, so no I do not want it to be interacting or dependent on states in home cache directories across different machines, so I’m not binding those into the container

Before I go and remove black/papermill from my workflow, is there a way to move the CACHE_DIR to a predictable location? Can I set this with an environmental variable? Can the cache go into /tmp or something? Or can I disable it?