question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

page.waitForSelector timeout in Docker container

See original GitHub issue

page.waitForSelectior() is working pretty welll on my dev machine (mac os) but it stacks on Docker container.

Here is my code:

async function scrape (url) {
  debug('Starting Chromium')

  const browser = await puppeteer.launch({args: ['--no-sandbox', '--disable-setuid-sandbox']})
  const page = await browser.newPage()
  await page.goto(url)
  await page.waitForSelector('#search') // Timeout on this line

  let result = await page.evaluate(() => {
    let results = []
    const items = document.querySelectorAll('#search')
    items.forEach(item => {
      // ...
      results.push({
       // ...
      })
    })

    return results
  })

  await browser.close()
  return result
}

scrape('https://www.google.com/search?q=puppeteer')

I have tried to use timout, like await page.waitForSelector('#search', {timeout: 30000}) but it is still not working.

Anybody know what is the reason of this issue ?

UPDATE: here is my Dockerfile

FROM node:8

RUN apt-get update && \
apt-get install -yq gconf-service libasound2 libatk1.0-0 libc6 libcairo2 libcups2 libdbus-1-3 \
libexpat1 libfontconfig1 libgcc1 libgconf-2-4 libgdk-pixbuf2.0-0 libglib2.0-0 libgtk-3-0 libnspr4 \
libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 \
libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 \
ca-certificates fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils wget

WORKDIR /app

COPY package*.json ./

RUN npm install

COPY . .

EXPOSE 80

CMD ["npm", "start"]

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Comments:12 (2 by maintainers)

github_iconTop GitHub Comments

2reactions
us24mancommented, May 7, 2021

Why has this been closed if there is no solution yet ? I am facing the exact same issue. Script works fine on mac os but gives this error when running in docker

1reaction
mechanoidcommented, Aug 16, 2021

For me it turned out, that waitFor wasn’t the problem. Instead it was.click() not submitting a form in the dockerized chromium, resulting in the timeout. Maybe it is something similar for you.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Puppeteer: waitForSelector works in local Docker container ...
I have built a nodeJS service that takes a screenshot of a Single Page Application using Puppeteer. When the SPA is finished loading, ......
Read more >
Scraping dynamic content - Apify Documentation
Wait for dynamically loaded content when web scraping. See code examples and a detailed breakdown for setting timeouts and custom wait functions.
Read more >
How to use the pyppeteer.errors.TimeoutError function ... - Snyk
Use Snyk Code to scan source code in minutes - no build needed - and fix issues ... assertRaises(TimeoutError): await self.page.goto(self.url + 'long')....
Read more >
puppeteer waitforselector - Unisa
I'm 2 to 3 hours into The Witcher 3 and drowners are impossible to kill. tear down of request listeners after page.close, Download...
Read more >
Playwright makes end-to-end browser testing fun - dflate.io
The way both Puppeteer and Playwright solve it is by offering timeout based APIs which wait until whatever you are looking for enters...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found