Feature planning: first class Python support

From @christopheranderson on May 4, 2016 23:4

This is a tracking item for First Class Python Support. We’ll be tracking the requirements and issues for first class Python support via this item. Feel free to leave comments about features/design patterns.

The Functions team and the Microsoft Python team have engaged in a partnership to investigate first class Python support for the Azure Functions runtime.

Script File

Python support for Functions will follow the existing scripting model and support a run file of .py format. We’ll use the workflow below to determine which file to run:

If the “scriptFile” attribute is set in function.json Else if the project contains a single .py file Else if main.py is present Else throw an error

Entry Point

Instead of wrapping everything in a class, we’ll define a main() method for the runtime to invoke during an execution. We’ll use the workflow below to determine which function to run:

If the “entryPoint” attribute is set in function.json Else if the script contains a single function Else if main() is present Else throw an error

Function format

Any references to external modules will be included at the beginning of the file using the import keyword. Additional classes and/or helper functions can also be added to the same file.

# main.py
import os
def main():
	pass

Since only a single Python process can exist per function app, it is recommended to implement main() as an asynchronous coroutine using the async def statement.

# Would be run with asyncio directly
async def main():
    await some_nonblocking_socket_io_op()

If the main() function is synchronous (no async qualifier) we automatically run it in an asyncio thread-pool:

# Would be run in an asyncio thread-pool
def main():
    some_blocking_socket_io()

This approach gives the user a choice of what libraries they want to use and async/await becomes an opt-in.

Binding Data

Trigger data and bindings will be communicated to the main() function via method arguments whose names are specified in function.json.

Inputs

There are two types of inputs: 1. trigger input and 2 . additional input. Although they are different in function.json, their usage is identical in the Python code.

# main.py
def main(trigger_input, binding_input):
	# functions logic here

Outputs

Outputs can be expressed both in 1. return value and 2. output parameters. If there is only one output, we recommend using it as the return value. For multiple outputs, you have to use the function arguments.

To use the return value of your function as an output binding, label it using name : $return in function.json.

//function.json
{ 
	"direction": "in", 
	"name": "my_input"
},
{ 
	"direction": "out", 
	"name": "$return"
}

# main.py
def main(my_input):
    # function logic here
    return 'Value of the output binding.'

To produce multiple outputs, use the named function arguments. For example -

// function.json
{
    "name": "trigger_input",
    "direction": "in"
},
{
    "name": "$return",
    "direction": "out"
},
{
    "name": "other_output",
    "direction": "out",
}

# main.py
def main(trigger_input, other_output):
    
    other_output = trigger_input
    return 'Value of the output binding.'

Data Types

We will use function annotations (type hints) to define the data type of the binding arguments. The runtime will use this information to determine how to pass data into and return data from a function execution. For example -

# main.py
def main(my_input : str) -> bool:
	context.logger.info(f'Hello {my_input}')
	return True

Alternately, you can define the dataType attribute in the function.json configuration.

HTTP Triggers and Bindings

HTTP webhook triggers and HTTP output bindings will use request and response objects to represent the HTTP messaging.

Request Object

The request object has the following attributes.

Attribute	Data Type	Description
method	str	HTTP method of the request
url	str	Url of the request
headers	dict	HTTP headers sent with the request
data	dict	Form data parameters
params	dict	Query string parameters
body	bytes	Raw HTTP request body

Response Object

The response object has the following attributes.

Attribute	Data Type	Description
headers	dict	Response headers
status_code	int	HTTP response status code
content	str, bytes, dict	Contents of the response
url	str	Final url location of the response

Similar to any other binding, the HTTP request and response objects can be accessed using the name attribute from function.json.

Context Object

Similar to the JavaScript programming model, the runtime will use a context object to pass data to and from the Python function. It is primarily used to reference metadata and information related to a specific execution of a function. For example -

# main.py
def main(context):
	context.logger.info(context.invocation_id) 

Similarly, enlisted are some other attributes you can access using context:

Attribute	Description
context.invocation_id	Unique identifier of the function execution.
context.function_name	Name of your function.
context.function_directory	Directory path to where your script is located.

Logging

In order to write traces during an execution, we’ll use Python’s logging module. The runtime will instantiate logger as a keyword-only required argument of type logging.Logger.

The logging module provides various convenience methods that let you write to the console log at more than one trace level. For example -

# main.py
from logging import Logger
def main(logger: Logger):
	logger.warning('Something has happened!')

Other methods include:

Method	Description
debug(message)	Detailed information, typically of interest only when diagnosing problems.
info(message)	Confirmation that things are working as expected.
warning(message)	An indication that something unexpected happened, or indicative of some problem in the near future (e.g. ‘disk space low’). The software is still working as expected.
error(message)	Due to a more serious problem, the software has not been able to perform some function.
critical(message)	A serious error, indicating that the program itself may be unable to continue running.

By default, the effective level for logger will be set to INFO. To modify this level, use the setLevel() method provided by the logging module.

Note: Since we use a single Python process per function app, using the inbuilt print utility will output to our system logs. These are not available to the user and hard to distinguish on a per function basis.

Environment variables

To retrieve information about the current process or user environment, use the os.environ method of the os module in Python. For example -

# main.py

import os
def main(context):
	context.logger.info('Python process has begun!')
	
	for name in ('HOME', 'USERPROFILE', 'USERNAME'):
		info = os.environ.get(name)
		context.logger.info(f'{name} : {info}')	

Version & Package management

We’ll stick to Python version 3.6.

Option 1 : Requirements.txt

We want the following experience to load packages in our function app:

1. Go to https://<function_app_name>.scm.azurewebsites.net.
2. Click Debug Console > CMD.
3. Go to D:\home\site\wwwroot, and then drag your requirements.txt file to the wwwroot folder at the top half of the page.

Django==2.0.1
Numpy==1.6.2

1. After the file is uploaded, run the pip install command in the Kudu remote execution console.This action downloads the packages indicated in the requirements.txt file and restarts the function app.

pip install -r requirements.txt

Once the packages are installed, you can import them to your function.

# main.py
import numpy
from django import http
def main():
	pass

Option 2 : Pipenv

Pipenv is the new official recommendation from python.org for installing and managing your Python dependencies.

Workflow/experience TBD.

Note :

As an ideal experience, we’ll be able to imbed the kudu console on the main dev screen in the portal and directly install pip packages there. Refer to the issue here for more details - https://github.com/Azure/azure-functions-ux/issues/2133.

Testing/CI

TBD

Samples

• HTTP Trigger example
• Timer Trigger example
• Queue Trigger example

Out of scope / Nice to have

1. Entry point decorator

In order to indicate an entry point in the script, we can define a @main decorator. For example:

# main.py
@main
def foo_bar():
	pass

Since the behavior is redundant to specifying the “entry point” attribute in function.json, we’ll consider this out of scope for now.

2. Access to bindings via context

TBD

3. Auto-install dependencies in Kudu

TBD

Open questions

TBD

Change log

5/5 : Chris - Add basic list of feature areas 1/10 : Asavari - Elaborating on the feature areas and programming model 1/11 : Asavari - Adding info about package management and environment variables 1/18 - Adding Samples 1/19 - Incorporating community feedback 1/23 - Incorporating community feedback (part 2)

Copied from original issue: Azure/azure-webjobs-sdk-script#335

[asavaritayal]

From @thdeltei on August 19, 2016 10:6

Azure ML Studio currently let you publish Python and R webservices, which work in a similar way as Azure Functions. It would be great if Python in Azure Functions came installed with the Anaconda bundle. That would allow for a simpler integration between Azure ML and Azure Functions.

[asavaritayal]

Closing issue since all comments did not get copied as intended.