Feature Request: Brython Production App Development Pipeline

I’ve reported some of these issues before, some of them have been closed; but looking at the history of those discussions I don’t think it was clear what I was asking for, so I want to try to write up a central tracking issue so that the core team can understand what I wish Brython could do. The discussion mostly meandered around through performance issues, which, while extremely important, are not really what I care about now.

So, here’s the problem: I find it very hard to get started developing an application in Brython. The official ‘deploy’ documentation does not actually tell me how to deploy my application at all, but rather, just how to load Brython itself into a web page.

First off, as with any Python development tool, I expect to be able to pip install brython and start doing something with it. Even if all this did was put the .js file somewhere on my filesystem, similar to the way certifi does nothing but print the location of a .pem file, it would be handy having this on PyPI as a way to quickly get started, rather than hunting around for a .zip file.

Second, it’s not clear how to organize my front-end project. If I were writing a Python project, I’d write a setup.py and perhaps a requirements.txt to pin versions. If I were writing JavaScript, I’d use browserify along with npm install. If I’m using Brython, where should I put my files so that import works cleanly? If I want to write a Brython-compatible library, how do I install it? How do I share my code with other developers? Right now the only obvious thing is to either write your own web server, and invent some idiom for putting things onto sys.path from scratch.

Third, once you have more than a few modules, start-up performance is really, really bad. Unusably bad, even. This is a recurring complaint, most recently in #461. Brython’s performance itself is actually perfectly acceptable, if only it could skip the catastrophic load-time hit of transpiling everything from JavaScript every time. This is related to the second problem, because if we had a way to organize and declare a hierarchy of brython-friendly code that could be imported, and a command to install it and deploy it to the browser, then optimizing startup performance is just a question of translating .py files to .pyc.js files, and then adding them to a .vfs file. This is also related to the first problem, because if we had pip install brython, then in addition to printing the location of brython.js, this file could easily also have a brython compile subcommand that would produce said .vfs file.

Fourth, what does automated testing of a Brython project look like? Firing up an actual browser is really bad for an interactive red/green/refactor testing workflow, not to mention that it’s not clear how to deploy that to Travis-CI.

All together, I want to do something like this.

$ mkdir mybry
$ mkvirtualenv -p `which python 3.5` mybrythonapp
$ pip install brython
$ $EDTIOR setup.py
$ $EDITOR mybry/__init__.py blub.txt
$ $EDITOR web/index.html
$ python setup.py bdist_wheel
$ brython bundle --used-by=web/index.html --used-by=web/about.html --used-by=web/login.html --output web/__brython_cache__.vfs.js --include=dist/*.whl
$ twist web --path web/
$ open http://localhost:8080/
$ brython test 'mybry.test.*'

By convention, brython.js could then try pre-loading /__brython_cache__.vfs.js to prepopulate sys.modules with already-compiled-to-javascript modules. You could also have other brython shell commands which would set up symlinks for easy re-loading during development.

The exact details of this aren’t important though; I just wanted to give one possible example. There are a lot of ways that this could work.

The current state of the Brython introduction is the same as a Python tutorial that just says “put all your code in one file, you probably won’t need two files”.

In my opinion this is the most important thing that Brython needs to do, in order for mainstream web developers (who, right now, in JavaScript, have advanced build-and-deploy pipelines with grunt and gulp and npm and browserify and a dozen other tools, and in Python, have precompiled wheels and Docker images and so on) to be able to take it seriously for frontend applications that include more than one module.

I hope that this has made it clear what I want from Brython and serves for a good basis for a discussion.

[PierreQuentel]

Cent fois sur le métier remettez votre ouvrage…

The current solution to generate the brython_modules.js file is not what I am the most proud of… So I have resumed work on this and now have a different, and hopefully better, solution.

1. python -m brython --install installs the basic Brython files in a new application directory.

2. The developer writes HTML pages and Python scripts to develop the application

3. The new command-line option python -m brython --make_dist executes the following tasks :

• generate brython_modules.js by parsing the code in all Brython scripts (forget .bundle_include !). This is done in a new module in the Brython repo, /setup/list_modules.py ; it is based on an AST visitor to recursively detect imports
• copy the application files in a subdirectory called __dist__
• in this directory, for HTML pages, <script src="/path/to/brython/brython_stdlib.js"> is replaced by <script src="/path/to/brython/brython_modules.js">
• the user is asked to enter basic setup information : application name, version, author, author email, url. This information is stored in a file brython_setup.json to avoid having to enter the information each time a --make_dist is run
• this information is used to generate a setup.py script, and another script called like the application name (eg myapp.py)

1. from the __dist__ subdirectory, running python setup.py sdist and python setup.py install installs the package myapp. This is done manually in case the developer wishes to edit or add information in setup.py. The package is an ordinary Python package and can be distributed through PyPI and pip.

2. running python -m myapp --install installs the application files (HTML pages, brython.js, brython_modules.js, Python scripts, etc.) in an empty directory

A first version exists in the current development version. It’s certainly not complete but I wanted to have your and other people’s feedback before going on.

[PierreQuentel]

I have published version 0.0.4 of the brython package, with 2 changes :

• with python -m brython --update, if there is a file .bundle-include in the folder, only the modules in this list are included in brython_modules.js
• the object __BRYTHON__ supports a new method imports() that opens a popup window with the list of all the modules currently imported (the same as sys.modules)

With these features, the workflow to minimise the size of brython_modules.js would be :

• run the application in the browser
• open the browser console and execute __BRYTHON__.imports() on the console command line
• copy/paste the modules list in file .bundle-include
• run python -m brython --update

This works in the few tests I made (some of the examples in the gallery), with brython_modules.js limited to a few 10 kbs instead of 3.4 Mb for brython_stdlib.js.