Requirements for building the lambda function
See original GitHub issueI’m having issues being able to successfully build the ./build-s3-dist.sh source-bucket-base-name
. The README doesn’t specify what the base OS requirements are for building the solution, but given yum
is mentioned I assumed that it was built on an Amazon Linux AMI or the like. To get around this I’m working on a Dockerfile
that bases the image off of the amazonlinux
based image.
pip
The first issue I ran into is that the latest version of pip
does not expose req
, so for pip
> version 10
it needs to say pip._internal.req
rather than pip.req
. Here is what I’ve updated both setup.py
to contain to get around it.
try: # for pip >= 10
from pip._internal.req import parse_requirements
except ImportError: # for pip <= 9.0.3
from pip.req import parse_requirements
However, when tornado_botocore
is included, it suffers from the same problem:
Collecting tornado_botocore==1.0.2 (from image-handler==2.0)
Downloading https://files.pythonhosted.org/packages/35/35/65434dd70a524c9bffc8c24c3c549573887c741f82bf33b8dc2bed696d88/tornado-botocore-1.0.2.tar.gz
Complete output from command python setup.py egg_info:
WARNING:root:It looks like some requirements are missing.
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-RY9v9C/tornado-botocore/setup.py", line 35, in <module>
from pip.req import parse_requirements
ImportError: No module named req
To get around this, I had to update 'tornado_botocore==1.3.2'
as well as 'botocore==1.8'
as those versions have updates around the pip
issue.
pycurl
The next issue that I found was when installing pycurl
. Here is the output:
Collecting pycurl<7.44.0,>=7.19.0 (from thumbor->thumbor-plugins==0.2.1->-r source/image-handler/requirements.txt (line 1))
Downloading https://files.pythonhosted.org/packages/e8/e4/0dbb8735407189f00b33d84122b9be52c790c7c3b25286826f4e1bdb7bde/pycurl-7.43.0.2.tar.gz (214kB)
100% |################################| 215kB 17.4MB/s
Complete output from command python setup.py egg_info:
Using curl-config (libcurl 7.51.0)
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-a6Ym8A/pycurl/setup.py", line 913, in <module>
ext = get_extension(sys.argv, split_extension_source=split_extension_source)
File "/tmp/pip-install-a6Ym8A/pycurl/setup.py", line 582, in get_extension
ext_config = ExtensionConfiguration(argv)
File "/tmp/pip-install-a6Ym8A/pycurl/setup.py", line 99, in __init__
self.configure()
File "/tmp/pip-install-a6Ym8A/pycurl/setup.py", line 316, in configure_unix
specify the SSL backend manually.''')
__main__.ConfigurationError: Curl is configured to use SSL, but we have not been able to determine which SSL backend it is using. Please see PycURL documentation for how to specify the SSL backend manually.
So it’d seem that the amazonlinux
base image isn’t setup to be able to build pycurl
. In order to get around that issue, I had to do the following:
yum install -y python27-devel openssl-devel
Success? ✅
…or so I thought. I was at least able to build the ZIP and deploy the function, however when I went to convert an image I got a 502
. Here are a couple of things from the CloudWatch logs:
13:29:27 [WARNING] It looks like some requirements are missing.
13:29:27 [ERROR] start_thumbor error: cannot import name Botocore
So, the bump for botocore
seemed to have broken some things with tc_aws
, so in order to get past that I had to use 'tc_aws==6.2.10'
per thumbor-community/aws#100.
Where I’m At
After bumping tc_aws
and deploying the latest built package, now all I’m relegated is a 502
still and simply a message that says:
[WARNING] It looks like some requirements are missing.
At this point, I’m at a loss at what to look into, so any help would be greatly appreciated to point me in the right direction as to what I might be doing wrong. I’ll probably try loading an actual AWS AMI that Lambda is using and see if I get the same results or not from there.
Ultimately, all I’m trying to do is bump thumbor to 6.5.2
but I’ve thus far been unsuccessful at even building a package that I can start with.
Here is the Dockerfile
that I’m currently attempting this with:
FROM amazonlinux:2017.03.1.20170812
MAINTAINER Levi Wilson <levi@leviwilson.com>
# lock yum to the same repository version
RUN sed -i 's/releasever=.*/releasever=2017.03/g' /etc/yum.conf
# base requirements
RUN yum install yum-utils zip -y && \
yum-config-manager --enable epel && \
yum install git libpng-devel libcurl-devel gcc python27-devel libjpeg-devel -y
# enable epel on Amazon Linux 2
RUN yum install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
# pip
RUN alias sudo='env PATH=$PATH' && \
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py && \
python get-pip.py && rm get-pip.py && \
pip install --upgrade setuptools && \
pip install --upgrade virtualenv
# pycurl
RUN yum install -y nss-devel
ENV PYCURL_SSL_LIBRARY=nss
# ImageMagick
RUN yum install -y ImageMagick-devel
Issue Analytics
- State:
- Created 5 years ago
- Comments:9
Top GitHub Comments
@rromanchuk this is what I’d done the last time we needed to build it
https://github.com/northwoodspd/serverless-image-handler/blob/sharp-output-format/.gitlab-ci.yml
@leviwilson do you have a recent workflow (current Dockerfile) for building this? I want to avoid a rabbit hole if possible.