question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Can't build sam with missing wheel for PyYaml

See original GitHub issue

Description

I am trying to use sam build to build my lambda and test it locally with a requirements.txt that includes the awscli. This in turn requires pyyaml. pyyaml is missing a wheel, and when sam tries to resolve this by building from the sdist it can’t do it. What appear to be the relevant sections of the build logs are below.

When I omit the awscli dependency, the build succeeds. When I deploy the Lambda to AWS using a cloudformation pipeline, it uses the Pipfile instead of the requirements.txt, and it is able to deploy successfully. But I can’t use sam to build and test my lambda locally.

Steps to reproduce

Create a Lambda that includes a dependency of awscli==1.16.155 and/or pyyaml==3.13 and run sam build

Observed result

2019-05-10 10:21:25 Build missing wheels from sdists (C compiling True): {rbt-utilities==0.0.1(sdist), pyyaml==3.13(sdist), article-data==0.0.1(sdist), awscli-wrapper==0.0.1(sdist), vectorizer==0.0.1(sdist), classifier==0.0.1(sdist), binarizer==0.0.1(sdist), nltk==3.4.1(sdist)}
...
2019-05-10 10:21:27 calling pip wheel --no-deps --wheel-dir /var/folders/cs/xfy7j9k12gvc12l0v9tvt0k1l7nvry/T/tmpx8snqxbs /var/folders/cs/xfy7j9k12gvc12l0v9tvt0k1l7nvry/T/tmpx8snqxbs/PyYAML-3.13.tar.gz
...
2019-05-10 10:21:41 compatible after building wheels (no C compiling): {ply==3.11(wheel), html2text==2018.1.9(wheel), requests==2.21.0(wheel), colorama==0.3.9(wheel), psycopg2==2.7.7(wheel), pyasn1==0.4.5(wheel), decorator==4.4.0(wheel), binarizer==0.0.1(wheel), certifi==2019.3.9(wheel), scikit-learn==0.21.0(wheel), scipy==1.2.1(wheel), s3transfer==0.2.0(wheel), python-dateutil==2.8.0(wheel), chardet==3.0.4(wheel), botocore==1.12.145(wheel), scikit-multilearn==0.2.0(wheel), awscli==1.16.155(wheel), boto3==1.9.145(wheel), rbt-utilities==0.0.1(wheel), article-data==0.0.1(wheel), awscli-wrapper==0.0.1(wheel), vectorizer==0.0.1(wheel), classifier==0.0.1(wheel), urllib3==1.24.3(wheel), nltk==3.4.1(wheel), jmespath==0.9.4(wheel), six==1.12.0(wheel), numpy==1.16.3(wheel), boto==2.49.0(wheel), rsa==3.4.2(wheel), joblib==0.13.2(wheel), docutils==0.14(wheel), idna==2.8(wheel), datadog==0.28.0(wheel)}
2019-05-10 10:21:41 Build missing wheels from sdists (C compiling False): {pyyaml==3.13(sdist)}
2019-05-10 10:21:41 calling pip wheel --no-deps --wheel-dir /var/folders/cs/xfy7j9k12gvc12l0v9tvt0k1l7nvry/T/tmpx8snqxbs /var/folders/cs/xfy7j9k12gvc12l0v9tvt0k1l7nvry/T/tmpx8snqxbs/PyYAML-3.13.tar.gz
...

2019-05-10 10:21:42 Final compatible: {ply==3.11(wheel), html2text==2018.1.9(wheel), requests==2.21.0(wheel), colorama==0.3.9(wheel), psycopg2==2.7.7(wheel), pyasn1==0.4.5(wheel), decorator==4.4.0(wheel), binarizer==0.0.1(wheel), certifi==2019.3.9(wheel), scikit-learn==0.21.0(wheel), scipy==1.2.1(wheel), s3transfer==0.2.0(wheel), python-dateutil==2.8.0(wheel), chardet==3.0.4(wheel), botocore==1.12.145(wheel), scikit-multilearn==0.2.0(wheel), awscli==1.16.155(wheel), boto3==1.9.145(wheel), rbt-utilities==0.0.1(wheel), article-data==0.0.1(wheel), awscli-wrapper==0.0.1(wheel), vectorizer==0.0.1(wheel), classifier==0.0.1(wheel), urllib3==1.24.3(wheel), nltk==3.4.1(wheel), jmespath==0.9.4(wheel), six==1.12.0(wheel), numpy==1.16.3(wheel), boto==2.49.0(wheel), rsa==3.4.2(wheel), joblib==0.13.2(wheel), docutils==0.14(wheel), idna==2.8(wheel), datadog==0.28.0(wheel)}
2019-05-10 10:21:42 Final incompatible: {scikit-learn==0.21.0(wheel), scipy==1.2.1(wheel), pyyaml==3.13(wheel), numpy==1.16.3(wheel), psycopg2==2.7.7(wheel)}
2019-05-10 10:21:42 Final missing wheels: {pyyaml==3.13(sdist)}
2019-05-10 10:21:47 PythonPipBuilder:ResolveDependencies failed
Traceback (most recent call last):
  File "/usr/local/Cellar/aws-sam-cli/0.15.0/libexec/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/actions.py", line 39, in execute
    self.scratch_dir
  File "/usr/local/Cellar/aws-sam-cli/0.15.0/libexec/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 143, in build_dependencies
    requirements_path, artifacts_dir_path, scratch_dir_path)
  File "/usr/local/Cellar/aws-sam-cli/0.15.0/libexec/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 212, in build_site_packages
    raise MissingDependencyError(packages_without_wheels)
aws_lambda_builders.workflows.python_pip.packager.MissingDependencyError: {pyyaml==3.13(sdist)}

(note: the other missing ones are local project specific packages each with their own setup.py file in the same repo)

Expected result

I expect the build to succeed

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

  1. OS: Mac OS Sierra
  2. sam --version: SAM CLI, version 0.15.0

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Reactions:16
  • Comments:7 (2 by maintainers)

github_iconTop GitHub Comments

21reactions
sriram-mvcommented, Feb 8, 2020

Was able to buid pyyaml as well simplejson within a container, so closing this issue.

13reactions
sriram-mvcommented, Sep 23, 2019

Have you tried sam build --use-container

Read more comments on GitHub >

github_iconTop Results From Across the Web

SAM Build only completes successfully with --use-container ...
It appears that some of the libraries you are trying to use are missing wheel files. It actually says so in the debug...
Read more >
Using "sam build" - Build includes non required folders
How can you force 'sam build' commands to not include sub directories ... Am I missing a step to stop these folders being...
Read more >
Tutorial: Deploying a Hello World application
In this guide, you download, build, and deploy a sample Hello World application using AWS SAM. You then test the application in the...
Read more >
Python YAML package documentation - Read the Docs
If your input is inconsistently indented, such indentation cannot be preserved. The first round-trip will make it consistent/normalize it. Here ...
Read more >
How to Handle your Python packaging in Lambda with ...
This will create a Serverless Python 3 template project at the given path ( numpy-test/ ) with ... Installing setuptools, pip, wheel...done.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found