question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

`sam build` fails with fastavro dependency in the requirements file

See original GitHub issue

Description

Sam build fails to build when there’s a dependency of fastavro in the requirements.txt file.

Steps to reproduce

In the requirements file use fastavro as dependency and use sam build or sam build -u

Observed result

Please provide command output with --debug flag set.

Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
'build' command is called
Starting Build inside a container
No Parameters detected in the template
1 resources found in the template
Found Serverless function with name='XandrReadLambda' and CodeUri='.'
Building resource 'XandrReadLambda'

Fetching lambci/lambda:build-python3.7 Docker container image......
Mounting /Users/d441303/Documents/professional/holodeck/xandr as /tmp/samcli/source:ro,delegated inside runtime container
Using the request object from command line argument
Loading workflow module 'aws_lambda_builders.workflows'
Registering workflow 'PythonPipBuilder' with capability 'Capability(language='python', dependency_manager='pip', application_framework=None)'
Registering workflow 'NodejsNpmBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm', application_framework=None)'
Registering workflow 'RubyBundlerBuilder' with capability 'Capability(language='ruby', dependency_manager='bundler', application_framework=None)'
Registering workflow 'GoDepBuilder' with capability 'Capability(language='go', dependency_manager='dep', application_framework=None)'
Registering workflow 'GoModulesBuilder' with capability 'Capability(language='go', dependency_manager='modules', application_framework=None)'
Registering workflow 'JavaGradleWorkflow' with capability 'Capability(language='java', dependency_manager='gradle', application_framework=None)'
Registering workflow 'JavaMavenWorkflow' with capability 'Capability(language='java', dependency_manager='maven', application_framework=None)'
Registering workflow 'DotnetCliPackageBuilder' with capability 'Capability(language='dotnet', dependency_manager='cli-package', application_framework=None)'
Found workflow 'PythonPipBuilder' to support capabilities 'Capability(language='python', dependency_manager='pip', application_framework=None)'
Running workflow 'PythonPipBuilder'
Running PythonPipBuilder:ResolveDependencies
calling pip download -r /tmp/samcli/source/requirements.txt --dest /tmp/samcli/scratch
Full dependency closure: {fastavro==0.22.12(wheel), urllib3==1.25.8(wheel), smart-open==1.9.0(sdist), requests==2.23.0(wheel), boto3==1.12.11(wheel), docutils==0.15.2(wheel), boto==2.49.0(wheel), certifi==2019.11.28(wheel), s3transfer==0.3.3(wheel), python-dateutil==2.8.1(wheel), pytz==2019.3(wheel), botocore==1.15.11(wheel), six==1.14.0(wheel), jmespath==0.9.5(wheel), chardet==3.0.4(wheel), idna==2.9(wheel)}
initial compatible: {urllib3==1.25.8(wheel), requests==2.23.0(wheel), boto3==1.12.11(wheel), docutils==0.15.2(wheel), idna==2.9(wheel), certifi==2019.11.28(wheel), s3transfer==0.3.3(wheel), python-dateutil==2.8.1(wheel), pytz==2019.3(wheel), botocore==1.15.11(wheel), six==1.14.0(wheel), jmespath==0.9.5(wheel), chardet==3.0.4(wheel), boto==2.49.0(wheel)}
initial incompatible: {fastavro==0.22.12(wheel), smart-open==1.9.0(sdist)}
Downloading missing wheels: {fastavro==0.22.12(wheel), smart-open==1.9.0(sdist)}
calling pip download --only-binary=:all: --no-deps --platform manylinux1_x86_64 --implementation cp --abi cp37m --dest /tmp/samcli/scratch fastavro==0.22.12
calling pip download --only-binary=:all: --no-deps --platform manylinux1_x86_64 --implementation cp --abi cp37m --dest /tmp/samcli/scratch smart-open==1.9.0
compatible wheels after second download pass: {urllib3==1.25.8(wheel), requests==2.23.0(wheel), boto3==1.12.11(wheel), docutils==0.15.2(wheel), boto==2.49.0(wheel), certifi==2019.11.28(wheel), s3transfer==0.3.3(wheel), python-dateutil==2.8.1(wheel), pytz==2019.3(wheel), botocore==1.15.11(wheel), six==1.14.0(wheel), jmespath==0.9.5(wheel), chardet==3.0.4(wheel), idna==2.9(wheel)}
Build missing wheels from sdists (C compiling True): {smart-open==1.9.0(sdist)}
calling pip wheel --no-deps --wheel-dir /tmp/samcli/scratch /tmp/samcli/scratch/smart_open-1.9.0.tar.gz
compatible after building wheels (no C compiling): {urllib3==1.25.8(wheel), smart-open==1.9.0(wheel), requests==2.23.0(wheel), boto3==1.12.11(wheel), docutils==0.15.2(wheel), boto==2.49.0(wheel), certifi==2019.11.28(wheel), s3transfer==0.3.3(wheel), python-dateutil==2.8.1(wheel), pytz==2019.3(wheel), botocore==1.15.11(wheel), six==1.14.0(wheel), jmespath==0.9.5(wheel), chardet==3.0.4(wheel), idna==2.9(wheel)}
Build missing wheels from sdists (C compiling False): set()
compatible after building wheels (C compiling): {urllib3==1.25.8(wheel), smart-open==1.9.0(wheel), requests==2.23.0(wheel), boto3==1.12.11(wheel), docutils==0.15.2(wheel), boto==2.49.0(wheel), certifi==2019.11.28(wheel), s3transfer==0.3.3(wheel), python-dateutil==2.8.1(wheel), pytz==2019.3(wheel), botocore==1.15.11(wheel), six==1.14.0(wheel), jmespath==0.9.5(wheel), chardet==3.0.4(wheel), idna==2.9(wheel)}
Final compatible: {urllib3==1.25.8(wheel), smart-open==1.9.0(wheel), requests==2.23.0(wheel), boto3==1.12.11(wheel), docutils==0.15.2(wheel), boto==2.49.0(wheel), certifi==2019.11.28(wheel), s3transfer==0.3.3(wheel), python-dateutil==2.8.1(wheel), pytz==2019.3(wheel), botocore==1.15.11(wheel), six==1.14.0(wheel), jmespath==0.9.5(wheel), chardet==3.0.4(wheel), idna==2.9(wheel)}
Final incompatible: {fastavro==0.22.12(wheel)}
Final missing wheels: {fastavro==0.22.12(wheel)}
PythonPipBuilder:ResolveDependencies failed
Traceback (most recent call last):
  File "/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/actions.py", line 42, in execute
    requirements_path=self.manifest_path,
  File "/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 137, in build_dependencies
    self._dependency_builder.build_site_packages(requirements_path, artifacts_dir_path, scratch_dir_path)
  File "/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 201, in build_site_packages
    raise MissingDependencyError(packages_without_wheels)
aws_lambda_builders.workflows.python_pip.packager.MissingDependencyError: {fastavro==0.22.12(wheel)}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflow.py", line 269, in run
    action.execute()
  File "/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/actions.py", line 45, in execute
    raise ActionFailedError(str(ex))
aws_lambda_builders.actions.ActionFailedError: {fastavro==0.22.12(wheel)}
Builder workflow failed
Traceback (most recent call last):
  File "/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/actions.py", line 42, in execute
    requirements_path=self.manifest_path,
  File "/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 137, in build_dependencies
    self._dependency_builder.build_site_packages(requirements_path, artifacts_dir_path, scratch_dir_path)
  File "/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 201, in build_site_packages
    raise MissingDependencyError(packages_without_wheels)
aws_lambda_builders.workflows.python_pip.packager.MissingDependencyError: {fastavro==0.22.12(wheel)}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflow.py", line 269, in run
    action.execute()
  File "/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflows/python_pip/actions.py", line 45, in execute
    raise ActionFailedError(str(ex))
aws_lambda_builders.actions.ActionFailedError: {fastavro==0.22.12(wheel)}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/lang/lib/python3.7/site-packages/aws_lambda_builders/__main__.py", line 126, in main
    mode=params.get("mode", None),
  File "/var/lang/lib/python3.7/site-packages/aws_lambda_builders/builder.py", line 125, in build
    return workflow.run()
  File "/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflow.py", line 76, in wrapper
    func(self, *args, **kwargs)
  File "/var/lang/lib/python3.7/site-packages/aws_lambda_builders/workflow.py", line 276, in run
    raise WorkflowFailedError(workflow_name=self.NAME, action_name=action.NAME, reason=str(ex))
aws_lambda_builders.exceptions.WorkflowFailedError: PythonPipBuilder:ResolveDependencies - {fastavro==0.22.12(wheel)}
Build inside container returned response {"jsonrpc": "2.0", "id": 1, "error": {"code": 400, "message": "PythonPipBuilder:ResolveDependencies - {fastavro==0.22.12(wheel)}"}}

Build Failed
Sending Telemetry: {'metrics': [{'commandRun': {'awsProfileProvided': False, 'debugFlagProvided': True, 'region': '', 'commandName': 'sam build', 'duration': 28859, 'exitReason': 'BuildError', 'exitCode': 1, 'requestId': '3d44ac3d-9c44-43bf-b28d-6a171b805ce2', 'installationId': '5a21b101-c25d-4cad-847e-9bf24c479324', 'sessionId': '479c0825-1275-449d-a2cd-9837c7d2c177', 'executionEnvironment': 'CLI', 'pyversion': '3.7.6', 'samcliVersion': '0.41.0'}}]}
HTTPSConnectionPool(host='aws-serverless-tools-telemetry.us-west-2.amazonaws.com', port=443): Read timed out. (read timeout=0.1)
Error: PythonPipBuilder:ResolveDependencies - {fastavro==0.22.12(wheel)}

Expected result

Build Succeeded

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

  1. OS: MacOS Mojave 10.14.6 (18G3020)
  2. sam --version: SAM CLI, version 0.41.0

Add --debug flag to command you are running

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:7 (1 by maintainers)

github_iconTop GitHub Comments

2reactions
snagafritzcommented, Jun 23, 2020

I was able to use the last fastavro release that included a manylinux1 build: 0.22.7.

https://github.com/fastavro/fastavro/releases/tag/0.22.7

0reactions
sriram-mvcommented, Jul 20, 2020

Support in manylinux wheels is in the works: https://github.com/awslabs/aws-lambda-builders/commit/09f837ea4626cd076ab1f527e2bec8a12de57c90

Closing the issue for now, as the original dependency now builds.

Read more comments on GitHub >

github_iconTop Results From Across the Web

sam build fails with fastavro dependency in the requirements ...
Sam build fails to build when there's a dependency of fastavro in the requirements.txt file. Steps to reproduce. In the requirements file use ......
Read more >
aws sam build not able to build packages which require ...
However, I seem to have run into a problem when trying to deploy a lambda which relies on some specific dependencies. Here is...
Read more >
conda/conda-build - Gitter
I have a meta.yaml with these dependencies: requirements: build: - python >=3 - setuptools - setuptools_scm - gitpython run: - python >=3 -...
Read more >
psutil legacy-install-failure | The AI Search Engine You Control
I'm having trouble installing Odoo in file requirements ... [dev]' Obtaining file:///Users/davidhyc/aws-sam-cli Installing build dependencies ... done ...
Read more >
Package List — Spack 0.20.0.dev0 documentation
Versions: develop, 2.2.3, 2.2.2, 2.2.1, 2.2.0; Build Dependencies: elpa, ... that contains several programs that perform operations on SAM/BAM files.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found