Optimizing publish from the core-tools

A consistent theme across private preview has been the extended time taken to publish an app from the core-tools.

Current Workflow

Behind the scenes the publish command will: InternalPreparePythonDeployment Method

1. Pull and run microsoft/azure-functions-python3.6:v2.0.11651-alpha
2. Copy your python functions and requirements.txt to the running container
3. In the container, create a venv for your project named worker_venv under /home/site/wwwroot
4. Run pip install -r requirements.txt in the container
5. Package up your functions and venv
6. Copy package outside of container

Proposed Workflow

1. Create a venv for the project named worker_env. Activate it.

2. For packages in requirements.txt, determine if they are pure .py packages or c extensions with universal wheels available on PyPI. Install these packages to worker_env using pip.

3. If requirements.txt contains c extension packages that do not have universal wheels on PyPI,

   • Pull and run microsoft/azure-functions-python3.6:v2.0.11651-alpha
   • Copy worker_env and requirements.txt to the running container under /home/site/wwwroot
   • Pip install the remaining packages
   • Copy worker_env outside the container

4. Package the python functions and worker_env together as a zip file for deployment to Azure

Notes

Currently, there are no wheels available for azure.functions and azure.worker. This is related to https://github.com/Azure/azure-functions-python-worker/issues/118

A few related issues:

• https://github.com/Azure/azure-functions-core-tools/issues/484
• https://github.com/Azure/azure-functions-core-tools/issues/483
• https://github.com/Azure/azure-functions-core-tools/issues/476
• https://github.com/Azure/azure-functions-core-tools/issues/480
• https://github.com/Azure/azure-functions-core-tools/issues/466

[priyaananthasankar]

Adding another suggestion/question to the proposed workflow: When you pip install pure .py packages subsequently after a first installation - docker has a way to figure if it is a re-install or a fresh install, through a build context. Or maybe provide a flag if docker needs to forget and consider a re-install. Noticed that irrespective of first time or subsequent times, all the python packages are again installed. Would be great if this can be fixed.

I think when publishing anything for the first time, we are prepared for a decent latency. But subsequent publish - like changing a line of code and redeploying etc should be much faster than a first publish.

[elprans]

@ahmelsayed @asavaritayal I decided to build a standalone Python module to create Python app packages for Azure. It would take the requirements.txt and generate the correct app.zip:

azure-functions-python-packer -r requirements.txt --platform linux --python-version 36 --output app.zip

core-tools would then simply need to add the above call.