page.waitForSelector timeout in Docker container
See original GitHub issuepage.waitForSelectior()
is working pretty welll on my dev machine (mac os) but it stacks on Docker container.
Here is my code:
async function scrape (url) {
debug('Starting Chromium')
const browser = await puppeteer.launch({args: ['--no-sandbox', '--disable-setuid-sandbox']})
const page = await browser.newPage()
await page.goto(url)
await page.waitForSelector('#search') // Timeout on this line
let result = await page.evaluate(() => {
let results = []
const items = document.querySelectorAll('#search')
items.forEach(item => {
// ...
results.push({
// ...
})
})
return results
})
await browser.close()
return result
}
scrape('https://www.google.com/search?q=puppeteer')
I have tried to use timout, like await page.waitForSelector('#search', {timeout: 30000})
but it is still not working.
Anybody know what is the reason of this issue ?
UPDATE: here is my Dockerfile
FROM node:8
RUN apt-get update && \
apt-get install -yq gconf-service libasound2 libatk1.0-0 libc6 libcairo2 libcups2 libdbus-1-3 \
libexpat1 libfontconfig1 libgcc1 libgconf-2-4 libgdk-pixbuf2.0-0 libglib2.0-0 libgtk-3-0 libnspr4 \
libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 \
libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 \
ca-certificates fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils wget
WORKDIR /app
COPY package*.json ./
RUN npm install
COPY . .
EXPOSE 80
CMD ["npm", "start"]
Issue Analytics
- State:
- Created 6 years ago
- Comments:12 (2 by maintainers)
Top Results From Across the Web
Puppeteer: waitForSelector works in local Docker container ...
I have built a nodeJS service that takes a screenshot of a Single Page Application using Puppeteer. When the SPA is finished loading, ......
Read more >Scraping dynamic content - Apify Documentation
Wait for dynamically loaded content when web scraping. See code examples and a detailed breakdown for setting timeouts and custom wait functions.
Read more >How to use the pyppeteer.errors.TimeoutError function ... - Snyk
Use Snyk Code to scan source code in minutes - no build needed - and fix issues ... assertRaises(TimeoutError): await self.page.goto(self.url + 'long')....
Read more >puppeteer waitforselector - Unisa
I'm 2 to 3 hours into The Witcher 3 and drowners are impossible to kill. tear down of request listeners after page.close, Download...
Read more >Playwright makes end-to-end browser testing fun - dflate.io
The way both Puppeteer and Playwright solve it is by offering timeout based APIs which wait until whatever you are looking for enters...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
Why has this been closed if there is no solution yet ? I am facing the exact same issue. Script works fine on mac os but gives this error when running in docker
For me it turned out, that waitFor wasn’t the problem. Instead it was
.click()
not submitting a form in the dockerized chromium, resulting in the timeout. Maybe it is something similar for you.