chrome crash on aws lambda (running python code)

hi, thanks for this awesome project. i moving on python because i need to use https://github.com/lightbody/browsermob-proxy . in a python virtual environment i locally installed selenium and i locally move chromedriver and you “compiled for lambda” chromium-driver (headless-chromium). i set up code in order to use the chromedriver binary and the headless-browser binary of the folder running the code. everything works well in ubuntu 18.04.

on aws lambda this will not work

this is the error

Message: unknown error: Chrome failed to start: exited abnormally
(unknown error: DevToolsActivePort file doesn't exist)
(The process started from chrome location /var/task/chromium-browser is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
(Driver info: chromedriver=2.45.615279 (12b89733300bd268cff3b78fc76cb8f3a7cc44e5),platform=Linux 4.14.77-70.59.amzn1.x86_64 x86_64)
: WebDriverException
Traceback (most recent call last):
File "/var/task/scraper.py", line 16, in handle
driver = webdriver.Chrome(executable_path=os.path.abspath(".")+os.path.sep+'chromedriver', chrome_options=chrome_options)
File "/var/task/selenium/webdriver/chrome/webdriver.py", line 81, in __init__
desired_capabilities=desired_capabilities)
File "/var/task/selenium/webdriver/remote/webdriver.py", line 157, in __init__
self.start_session(capabilities, browser_profile)
File "/var/task/selenium/webdriver/remote/webdriver.py", line 252, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "/var/task/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/var/task/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally
(unknown error: DevToolsActivePort file doesn't exist)
(The process started from chrome location /var/task/chromium-browser is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
(Driver info: chromedriver=2.45.615279 (12b89733300bd268cff3b78fc76cb8f3a7cc44e5),platform=Linux 4.14.77-70.59.amzn1.x86_64 x86_64)

any suggestion?

thanks

p.s i can move everything to https://www.npmjs.com/package/browsermob-proxy but i like to use python (and i think the problem is not python related).

[ghost]

you need to make sure all the versions are compatible. I’ve got it working using:

chromedriver 2.41 headless-chromium-68.0.3440.84 selenium==3.141.0

The latest binaries on the release page don’t seem to respect the '--disable-dev-shm-usage' flag

set permissions on /dev/shm to readonly and then try and run locally to test before pushing to lambda.

Lastly, if you having different problems, set a logger in selenium driver:

driver = webdriver.Chrome(
        CHROMEDRIVER_PATH,
        chrome_options=options,
        service_log_path='/tmp/chromedriver.log'
    )

then just do a try/except all and read the log in the lambda if it fails:

try:
    your_headless_chrome_function()
except:
        with open('/tmp/chromedriver.log', 'r') as logfile:
            data = logfile.readlines()
        return data

[kremen]

@x00x70, thanks a lot! I use Ruby and my requests to chrome were timed out on AWS Lambda ({“message”: “Endpoint request timed out”}), when I was using the latest binaries from the release page (v1.0.0-55).

And it really works with: chromedriver 2.41 headless-chromium-68.0.3440.84 selenium==3.141.0

Cool tip to set permissions on /dev/shm to readonly and test it locally!