Adding binary wheels to a Function App

I’m having issues trying to add a custom, binary wheel (GDAL) to an Azure Functions app on Consumption. Unfortunately, manylinux wheels for GDAL are not provided in PyPI and building it is a bit of a chore (developing on Windows) so we have used a pre-built wheel in the function app directory to be able to use it.

Previously, deployment used to work with requirements.txt containing a path to the wheel:

./GDAL-2.4.0-cp36-cp36m-manylinux1_x86_64.whl

Which caused building with --build-native-deps to actually install the wheel as expected from the function app directory.

This does not seem to work any more and the change seems to have happened in Azure/azure-functions-core-tools@a8f620398818cb03ff59a2b96bbae922784ac666 - but the rather drastic change in behaviour is not documented anywhere as far as I can see.

Expected behavior

func azure functionapp publish succeeds.

Actual behavior

func azure functionapp publish fails, with

Could not install packages due to an EnvironmentError: [Errno 2] No such file or directory: '/GDAL-2.4.0-cp36-cp36m-manylinux1_x86_64.whl'

Tried workarounds

I did try building a .zip myself, as discussed in #182, but deploying the .zip to Consumption Function App using az functionapp deployment source config-zip fails with Failed to retrieve Scm Uri (due to Kudu not being available on Consumption plans?)

There is discussion in Azure/azure-functions-core-tools#1052 concerning func pack and deploying the .zip from func pack, but --no-build isn’t really applicable here - I would like to bundle the app normally, but just inject this wheel at some point.

I did try with --additional-packages, but unfortunately the GDAL package in apt is way too old to build the wheel.

Known workarounds

Currently, after digging through what feels like dozens of bug reports across different repositories (Azure/azure-functions-core-tools#1230 being the crucial one, and realizing to follow the commit referencing it…), I did find the (rather undocumented) environment option FUNCTIONS_PYTHON_DOCKER_RUN_COMMAND allowing me to (essentially manually) mount the Function App directory ($ENV:FUNCTIONS_PYTHON_DOCKER_RUN_COMMAND="run --rm -d -v ${PWD}:/mounted mcr.microsoft.com/azure-functions/python:2.0.12410-python3.6-buildenv sleep infinity") and allowing pip install to find the required wheel. However, this is not really a solution, as I guess I’ll need to update the docker image manually when it gets updated?

My question then would be; what is the recommended way of bringing binary wheels into an Azure Function on Linux Consumption? Not everything, after all, can be built on deploy and sometimes binary wheels are the only way to go. Consider, for example, having to interface with a third party provided binary library.

At the very least, consider this a bug report on the documentation - it could be heavily improved. There are little pieces and very ugly hacks easy to find, but actual documentation is, well, non-existent. For example, issue #6 is very easy to find, but seems to be mostly brainstorming about how to do it which isn’t immediately clear.

[Hazhzeng]

Please consider include the direct https:// (can include basic-auth authentication here) link in your requirements.txt and use func azure functionapp publish <appname> to publish your function app. Our remote build server will resolve the dependencies for you.

Here is a small example project using custom GDAL wheel from manthey.github.io PythonWheelExample.zip

What happen if I don’t have an external website link to my wheel Consider uploading your custom wheel into your Azure Storage Blob Container, you can generate an SAS link from the blob which allow you to access the blob from remote build. As long as you’re not sharing your SAS link to others, your blob is safe.

Why I cannot run my project locally now A single wheel is OS specific and Python version specific. If you’re using a manylinux wheel, it will NOT work on a Windows/MacOS environment.

This is a similar issue when you’re using PyTorch in Azure Functions, which the CPU version is not presented in PyPi. In order to make requirements.txt point to different wheels based on OS platform, consider building your own index.html and make your requirements.txt using the new index (pytorch index example),

-f https://download.pytorch.org/whl/torch_stable.html
torch==1.5.0+cpu

or modifying your requirements.txt pointing to different wheel based on sys_platform and python_version

https://download.pytorch.org/whl/cpu/torch-1.4.0%2Bcpu-cp36-cp36m-win_amd64.whl; sys_platform == 'win32' and python_version == '3.6'
https://download.pytorch.org/whl/cpu/torch-1.4.0%2Bcpu-cp36-cp36m-linux_x86_64.whl; sys_platform == 'linux' and python_version == '3.6'
https://download.pytorch.org/whl/cpu/torch-1.4.0%2Bcpu-cp37-cp37m-win_amd64.whl; sys_platform == 'win32' and python_version == '3.7'	
https://download.pytorch.org/whl/cpu/torch-1.4.0%2Bcpu-cp37-cp37m-linux_x86_64.whl; sys_platform == 'linux' and python_version == '3.7'	
https://download.pytorch.org/whl/cpu/torch-1.4.0%2Bcpu-cp38-cp38-win_amd64.whl; sys_platform == 'win32' and python_version == '3.8'	
https://download.pytorch.org/whl/cpu/torch-1.4.0%2Bcpu-cp38-cp38-linux_x86_64.whl; sys_platform == 'linux' and python_version == '3.8'

Why my project does not work on Azure Functions runtime Possibly, there’s a misalignment between your wheels and our runtime environment. We are using Debian Stretch for v2 functions, and Debian Buster for v3 functions. We recommend using the manylinux wheels.

Also, please check your Python runtime version in https://resources.azure.com/subscriptions/<subscription uuid>/resourceGroups/<resource group name>/providers/Microsoft.Web/sites/<function app name>/config. Search for LinuxFxVersion field. Make sure it matches your custom wheel.

[mebibou]

Same issue for me with GDAL as well, as reported here: https://github.com/Azure/azure-functions-core-tools/issues/1358#issuecomment-503013415