requestHandlerTimeoutSecs works ambiguously

Which package is this bug report for? If unsure which one to select, leave blank

@crawlee/puppeteer (PuppeteerCrawler)

Issue description

If it set a value for requestHandlerTimeoutSecs in the crawler config, it seems to be adding on to a default value of 70. According to the docs, the default value is 60 and I would think requestHandlerTimeoutSecs should override the default value.

Setting it to 0 times out at 70. Setting it to 10 times out at 80. Setting it to 61 times out at 131.

Initially I tested it because I wanted to run my handler without a timeout and thought maybe 0 would do it.

Tested it with both Playwright and Puppeteer with the same result.

Code sample

const crawler = new PuppeteerCrawler({
        requestHandler: router,
        maxConcurrency: 1,
        requestHandlerTimeoutSecs: 10,
        headless: false,
        requestList
    })

Package version

^3.0.0

Node.js version

v19.0.0

Operating system

Ubuntu

Apify platform

• Tick me if you encountered this issue on the Apify platform

Priority this issue should have

Low (slightly annoying)

I have tested this on the next release

No response

Other context

No response

[B4nan]

Yeah, that’s the same as OP - the final timeout is a sum of a few things, including additional 10s as a safety check. Which makes some sense for the default, but not so much when a user sets the value explicitly.

https://github.com/apify/crawlee/blob/master/packages/http-crawler/src/internals/http-crawler.ts#L333 https://github.com/apify/crawlee/blob/master/packages/browser-crawler/src/internals/browser-crawler.ts#L357

[mnmkng]

Oops.