Unable to import fiona in AWS Lambda Function

Expected behavior and actual behavior.

I have an AWS Lambda Function that uses a Lambda Layer that contains fiona. It has worked consistently for over a year now. Today I updated from 1.8.20 to 1.8.21 and now my Lambda function raises an ImportError when it starts up.

[ERROR] Runtime.ImportModuleError: Unable to import module 'app': No module named 'fiona.ogrext'

I expected the Lambda to continue working after the update to 1.8.21.

I also attempted to include fiona in the Lambda package without the use of a Lambda Layer and received a different message:

[ERROR] Runtime.ImportModuleError: Unable to import module 'app': libexpat.so.1: cannot open shared object file: No such file or directory

Steps to reproduce the problem.

I use the SAM framework to build and deploy my Lambda functions.

1. Create a new SAM project using sam init and choose the Hello World template
2. Add fiona to the requirements.txt file provided by the template
3. Use the SAM CLI to build: sam build
4. Use the SAM CLI to deploy: sam deploy --guided
5. Once the stack has been created, open the AWS Lambda Console in web browser and open the Lambda function created by the SAM application
6. Test the Lambda function using the Lambda console

Operating system

AWS Lambda python3.9 runtime

AWS Lambda runs in a Linux-like environment but I am not sure of distro or version. I was also able to produce this behavior on Mac OS X 12.2.1

Fiona and GDAL version and provenance

The SAM framework installs the latest fiona version during the sam build command. This also installs the binaries the fiona requires. They are packed in the site-packages/Fiona.libs folder of the Lambda package.

Here are the contents of Fiona.libs:

libcurl-5ae56dbf.so.4.7.0
libgdal-e127aa65.so.30.0.1
libgeos-3097a6d2.so.3.10.2
libgeos_c-a8310b50.so.1.16.0
libjpeg-82e9a1fb.so.9.3.0
libjson-c-b8129721.so.5.1.0
libnghttp2-e183d352.so.14.21.1
libpcre-9513aab5.so.1.2.0
libpng16-8ebcc106.so.16.35.0
libproj-4d96bf55.so.22.2.1
libsqlite3-83998bda.so.0.8.6
libtiff-104e144c.so.5.7.0

[sgillies]

It seems to me that the presence of libexpat in the Python 3.7 Fiona wheels is a quirk and can’t be relied upon. It may go away in future releases.

I’m going to close this one as a wontfix. Auditwheel is excluding libexpat for manylinux_2_17_x86_64.manylinux2014_x86_64 and newer wheels by design. That project would need to change.

[beck3905]

@sgillies I did some experimenting today. I did a pip install fiona on the following docker images:

• amazon/aws-lambda-python:3.7
• amazon/aws-lambda-python:3.8
• amazon/aws-lambda-python:3.9

All 3 used the manylinux2014 wheel. According to the docs, Lambda does support manylinux2014. I found that the libexpat library was included in Fiona.libs when installed with python 3.7, but not for 3.8 and 3.9. I was able to import fiona locally in all 3 docker containers. However, when I built my Lambda layer package using the site packages from the aws-lambda-python:3.9 container, my Lambda function using the Python3.9 runtime failed due to an import error looking for libexpat.