Manylinux wheels for packages like pandas and numpy cannot be replaced by zappa

I was trying to zappa deploy a python 3.8 function to AWS Lambda using MacOSX development environment. The python 3.8 function depends on numerous packages, including pandas. When I was doing development, and cross check the deployment package of zappa, i found that the pandas library in the package is still darwin’s wheel, expected to be manylinux wheel.

Expected Behavior

Expect manylinux wheels for pandas and numpy will be replaced when calling zappa deploy / update.

Actual Behavior

Pandas and numpy inside the virtualenv is still darwin package

Possible Fix

Change the line at https://github.com/zappa/Zappa/blob/1cdbd7f828b643b2a70c32fd334ec4f0662642de/zappa/core.py#L320 from: f’^.{self.manylinux_suffix_start}-manylinux({“|”.join(manylinux_suffixes)})_x86_64.whl$’ to f’^.{self.manylinux_suffix_start}(.*?)(-|.)manylinux({“|”.join(manylinux_suffixes)})_x86_64.whl$’

Steps to Reproduce

1. write a python 3.8 lambda handler
2. create an environment with pip install the latest pandas
3. try zappa deploy in macos, with delete_local_zip = false
4. unzip the generated package, check the wheel tag for pandas
5. Find the wheel tag is cp38-cp38-macosx_10_9_x86_64, which is unexpected

Your Environment

• Zappa version used: latest (0.53)
• Operating System and Python version: Darwin 20.5.0 Darwin Kernel Version 20.5.0: Sat May 8 05:10:33 PDT 2021; root:xnu-7195.121.3~9/RELEASE_X86_64 x86_64 i386, Python 3.8.6
• The output of pip freeze: argcomplete==1.12.3 boto3==1.17.112 botocore==1.20.112 certifi==2021.5.30 cfn-flip==1.2.3 charset-normalizer==2.0.2 click==8.0.1 durationpy==0.5 future==0.18.2 hjson==3.0.2 idna==3.2 jmespath==0.10.0 kappa==0.6.0 numpy==1.21.0 pandas==1.3.0 pep517==0.10.0 pip-tools==6.2.0 placebo==0.9.0 python-dateutil==2.8.2 python-slugify==5.0.2 pytz==2021.1 PyYAML==5.4.1 requests==2.26.0 s3transfer==0.4.2 six==1.16.0 text-unidecode==1.3 toml==0.10.2 tqdm==4.61.2 troposphere==3.0.1 urllib3==1.26.6 Werkzeug==0.16.1 wsgi-request-logger==0.4.6 zappa==0.53.0
• Link to your project (optional):
• Your zappa_settings.json:

[faiyeah]

Confirmed my original reported issue is gone with 0.54.0. Thanks all.

[david-wallace-croft]

I am seeing something similar with psycopg2-binary. It looks like psycopg2 switched to using PEP600: https://www.psycopg.org/docs/news.html#what-s-new-in-psycopg-2-9

So now instead of having a filename like this: psycopg2_binary-2.8.6-cp38-cp38-manylinux1_x86_64.whl

It now looks like this: psycopg2_binary-2.9.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

It looks like the the manylinux_wheel_file_match regular expression almost finds the PEP600-formatted filename using the legacy suffix except that now there is a period before “manylinux” instead of a hyphen. It looks like the possible fix recommended above could work.