Chrome not reachable with Selenium Python

I’m using serverless-chrome with Python in Lambda, following this repo’s instructions.

It works great locally in a Docker container, but not when deployed to Lambda. This error comes up. It seems the webdriver starts up fine but the binary is unreachable when asked to parse the DOM with Selenium.

(Session info: headless chrome=65.0.3325.146)
(Driver info: chromedriver=2.37.544315 (730aa6a5fdba159ac9f4c1e8cbc59bf1b5ce12b7),platform=Linux 4.9.85-38.58.amzn1.x86_64 x86_64)
: WebDriverException
Traceback (most recent call last):
File "/var/task/src/lambda_function.py", line 284, in lambda_handler
driver.find_element_by_xpath('//*[@id="searchForm"]/div[4]/ul/li[1]/input').click()
File "/var/task/lib/selenium/webdriver/remote/webdriver.py", line 385, in find_element_by_xpath
return self.find_element(by=By.XPATH, value=xpath)
File "/var/task/lib/selenium/webdriver/remote/webdriver.py", line 955, in find_element
'value': value})['value']
File "/var/task/lib/selenium/webdriver/remote/webdriver.py", line 312, in execute
self.error_handler.check_response(response)
File "/var/task/lib/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: chrome not reachable
(Session info: headless chrome=65.0.3325.146)
(Driver info: chromedriver=2.37.544315 (730aa6a5fdba159ac9f4c1e8cbc59bf1b5ce12b7),platform=Linux 4.9.85-38.58.amzn1.x86_64 x86_64)

This is the code leading up to the line that fails:

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--disable-gpu')
chrome_options.add_argument('--window-size=1280x1696')
chrome_options.add_argument('--user-data-dir=/tmp/user-data')
chrome_options.add_argument('--hide-scrollbars')
chrome_options.add_argument('--enable-logging')
chrome_options.add_argument('--log-level=0')
chrome_options.add_argument('--v=99')
chrome_options.add_argument('--single-process')
chrome_options.add_argument('--data-path=/tmp/data-path')
chrome_options.add_argument('--ignore-certificate-errors')
chrome_options.add_argument('--homedir=/tmp')
chrome_options.add_argument('--disk-cache-dir=/tmp/cache-dir')
chrome_options.add_argument('user-agent=Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36')
chrome_options.binary_location = '/var/task/bin/headless-chromium'

driver = webdriver.Chrome(chrome_options=chrome_options)
driver.implicitly_wait(3)
driver.get(URL)

Any advice would be fantastic.

[robroc]

After a lot of tooling around, I got it to work on Lambda. The problem was with incompatible versions of serverless-chrome, chromedriver, and Selenium. These are the versions that play well together in Lambda. Why is beyond me:

chromedriver v.2.37 severless-chrome v.0.0-37 selenium 2.53.6 (for Python)

[umihico]

My script worked well with

• serverless-chrome v1.0.0-37
• chromedriver 2.37
• latest selenium
• Python 3.7 (also confirmed not working with 3.8)

These binaries could be one lambda layer with this script. The test code is here. Please remove unnecessary arguments such as region in the script.