Cannot run Hello World against Azure

I have been trying to run a simple Hello World Lithops program against Azure:

import lithops


def hello_world(name):
    return 'Hello {}!'.format(name)

if __name__ == '__main__':


    config = {"lithops": {"backend": "azure_functions", "storage": "azure_storage", "storage_bucket": "apt-falk-meta", "mode": "serverless"}, "azure_storage": {"storage_account_name": "aptregstorageacc", "storage_bucket": "apt-falk-meta", "storage_account_key": "..."}, "azure_functions": {"location": "eastus", "runtime": "regoptsvc", "resource_group": "falk", "functions_version": 3}}

    fexec = lithops.FunctionExecutor(config=config)

    fexec.call_async(hello_world, 'World')
    print(fexec.get_result())

When I run it, I hit the following error(s) both on Lithops 2.5.9 and 2.6.0:

% python validate_lithops.py
2022-06-07 16:58:00,494 [INFO] lithops.config -- Lithops v2.6.0
2022-06-07 16:58:00,762 [INFO] lithops.storage.backends.azure_storage.azure_storage -- Azure Blob client created
2022-06-07 16:58:00,763 [INFO] lithops.serverless.backends.azure_functions.azure_functions -- Azure Functions client created - Location: eastus
2022-06-07 16:58:00,763 [INFO] lithops.invokers -- ExecutorID a20423-0 | JobID A000 - Selected Runtime: regoptsvc - 256MB
2022-06-07 16:58:00,905 [INFO] lithops.invokers -- Runtime regoptsvc with 256MB is not yet deployed
2022-06-07 16:58:00,906 [INFO] lithops.serverless.backends.azure_functions.azure_functions -- Creating new Lithops runtime for Azure Function: regoptsvc
2022-06-07 16:59:28,817 [INFO] lithops.serverless.backends.azure_functions.azure_functions -- Extracting Python modules from: regoptsvc
Traceback (most recent call last):
  File "/Users/fplk/PycharmProjects/predopt/lithops-venv/lib/python3.7/site-packages/lithops/serverless/backends/azure_functions/azure_functions.py", line 328, in _generate_runtime_meta
    payload=payload, return_result=True)
  File "/Users/fplk/PycharmProjects/predopt/lithops-venv/lib/python3.7/site-packages/lithops/serverless/backends/azure_functions/azure_functions.py", line 280, in invoke
    data = json.loads(resp.read().decode("utf-8"))
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/__init__.py", line 348, in loads
    return _default_decoder.decode(s)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "validate_lithops.py", line 26, in <module>
    fexec.call_async(hello_world, 'World')
  File "/Users/fplk/PycharmProjects/predopt/lithops-venv/lib/python3.7/site-packages/lithops/executors.py", line 205, in call_async
    runtime_meta = self.invoker.select_runtime(job_id, runtime_memory)
  File "/Users/fplk/PycharmProjects/predopt/lithops-venv/lib/python3.7/site-packages/lithops/invokers.py", line 120, in select_runtime
    runtime_meta = self.compute_handler.deploy_runtime(self.runtime_name, runtime_memory, runtime_timeout)
  File "/Users/fplk/PycharmProjects/predopt/lithops-venv/lib/python3.7/site-packages/lithops/serverless/serverless.py", line 73, in deploy_runtime
    return self.backend.deploy_runtime(runtime_name, memory, timeout=timeout)
  File "/Users/fplk/PycharmProjects/predopt/lithops-venv/lib/python3.7/site-packages/lithops/serverless/backends/azure_functions/azure_functions.py", line 92, in deploy_runtime
    metadata = self._generate_runtime_meta(runtime_name, memory)
  File "/Users/fplk/PycharmProjects/predopt/lithops-venv/lib/python3.7/site-packages/lithops/serverless/backends/azure_functions/azure_functions.py", line 330, in _generate_runtime_meta
    raise Exception("Unable to invoke 'extract-preinstalls' action")
Exception: Unable to invoke 'extract-preinstalls' action

On the Azure side I see quite a few resources being created:

• regoptsvc Application Insights and Function App
• EastUSLinuxDynamicPlan App Service plan
• aptregstorageacc Storage account
• aptregopt-lithops-runtime-v37-259-http Application Insights All under the same resource group.

Interestingly, under Application Insights I see Microsoft.Azure.WebJobs.Script.Workers.Rpc.RpcExceptions:

Exception while executing function: Functions.lithops_handler Result: Failure
Exception: ImportError: 

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

  * The Python version is: Python3.7 from "/usr/local/bin/python"
  * The NumPy version is: "1.21.6"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: No module named 'numpy.core._multiarray_umath'
. Troubleshooting Guide: https://aka.ms/functions-modulenotfound
Stack:   File "/azure-functions-host/workers/python/3.7/LINUX/X64/azure_functions_worker/dispatcher.py", line 309, in _handle__function_load_request
    func_request.metadata.entry_point)
  File "/azure-functions-host/workers/python/3.7/LINUX/X64/azure_functions_worker/utils/wrappers.py", line 42, in call
    raise extend_exception_message(e, message)
  File "/azure-functions-host/workers/python/3.7/LINUX/X64/azure_functions_worker/utils/wrappers.py", line 40, in call
    return func(*args, **kwargs)
  File "/azure-functions-host/workers/python/3.7/LINUX/X64/azure_functions_worker/loader.py", line 85, in load_function
    mod = importlib.import_module(fullmodname)
  File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/site/wwwroot/lithops_handler/__init__.py", line 23, in <module>
    from lithops.worker import function_handler
  File "/home/site/wwwroot/.python_packages/lib/site-packages/lithops/worker/__init__.py", line 1, in <module>
    from .handler import function_handler
  File "/home/site/wwwroot/.python_packages/lib/site-packages/lithops/worker/handler.py", line 38, in <module>
    from lithops.worker.jobrunner import JobRunner
  File "/home/site/wwwroot/.python_packages/lib/site-packages/lithops/worker/jobrunner.py", line 32, in <module>
    import numpy as np
  File "/home/site/wwwroot/.python_packages/lib/site-packages/numpy/__init__.py", line 150, in <module>
    from . import core
  File "/home/site/wwwroot/.python_packages/lib/site-packages/numpy/core/__init__.py", line 48, in <module>
    raise ImportError(msg)

Finally, I have also conducted an experiment to predeploy Lithops which failed the same way: I extracted the majority of the Dockerfile from here and tried to manually build and deploy it with the same results:

lithops clean -c falk_lithops_azure.config
lithops runtime build -b azure_functions regoptsvc -c falk_lithops_azure.config -f Dockerfile.azure_lithops
lithops runtime deploy -b azure_functions regoptsvc -c falk_lithops_azure.config -d --memory 8192 --timeout 360

Finally, this has been exacerbated by the fact that while the other targets have been well documented in the runtime folder, Azure is still missing from there.

Could you please advice how to proceed and get the hello world to run? Gil Vernik pointed me here.

Thank you very much in advance.

Best wishes, Falk

[JosepSampe]

This could happen if you have diferent versions of numpy in the client and in the runtime

[theathibm]

ok, in regards to custom runtime, we not only need our cythonized code on the runtime, but also we need some software packages installed. We currently build our own docker image and then modify the Dockerfile (e.g. gcp) as follows:

FROM us.icr.io/predopt/predopt_scripts:inc_2_plus_with_update

RUN apt-get update && apt-get install -y \
        zip \
        && rm -rf /var/lib/apt/lists/*

RUN pip install --upgrade setuptools six pip \
    && pip install --no-cache-dir \
        wheel \
        gunicorn \
        pika \
        flask \
        gevent \
        redis \
        requests \
        PyYAML \
        kubernetes \
        numpy \
        cloudpickle \
        ps-mem \
        tblib \
        cryptography \
        httplib2 \
        google-cloud-storage \
        google-api-python-client \
        gcsfs \
        google-auth

# Copy Lithops proxy and lib to the container image.
ENV APP_HOME /lithops
WORKDIR $APP_HOME

RUN touch __init__.py
COPY ./coscompat.py . 
COPY lithops_knative.zip .
RUN unzip lithops_knative.zip && rm lithops_knative.zip

CMD exec gunicorn --bind :$PORT lithopsproxy:proxy

Notice the FROM in the dockerfile.

If we can’t use DOCKER anymore, then how would we create a base image from which lithops could build and extend.

If azure lithops cannot use our built containers, then I’m concerned that the lithops support for building custom runtimes for Azure using just a requirements.txt won’t be feasible.

Our custom built docker images work on GCP Cloudrun and IBM Code Engine. Would be good to have Azure be able to handle it as well.

We use IBM CPLEX Opimization Studio and that software has to be on the Docker Image unfortunately