Puppeteer option 'timeout' being ignored in rendering

🐞 Bug report

We have a Scully configuration for one of our company’s project. As there have been some changes on the underlying API some pages are taking more time to get all the information so we began getting a: Puppeteer error while rendering "/some_url/final_url" TimeoutError: Navigation timeout of 30000 ms exceeded we will retry rendering this page up to 3 times.

We tried adding the timeout option in the puppeteerLaunchOptions section of our Scully config without any results.

🔬 Minimal Reproduction

Scully config file extract:

export const config: ScullyConfig = {
  puppeteerLaunchOptions: {
    // slowMo: 1750, // Relentiza la ejecucion de Puppeteer
    args: [
      '--disable-gpu',
      '--renderer',
      '--no-sandbox',
      '--no-service-autorun',
      '--no-experiments',
      '--no-default-browser-check',
      '--disable-dev-shm-usage',
      '--disable-setuid-sandbox',
      '--no-first-run',
      '--no-zygote',
      '--single-process',
      '--disable-extensions',
      '--user-agent=\'Scully Navigator\''
    ],
    executablePath: '/usr/local/bin/chromium',
    timeout: 120000
  },
  projectRoot: './apps/frontend/project/src',
// other configurations...
};

Execute: npx scully --prod --showGuessError --showBrowser=false --pluginsError=false --logSeverity=normal --project frontend-project

After a while many endpoints still launch the same error pointing to a Navigation timeout of 30000 ms exceeded even though the configuration states a timeout of 120000 ms. I think this might be a Puppeteer issue and not Scully’s fault, but anyway Scully lacks an option that allows increasing the navigation timeout.

Workaround

I edited directly the node_modules/@scullyio/scully/src/lib/renderPlugins/puppeteerRenderPlugin.js in order to increase the timeout when navigating the pages.

        // enter url in page
        await page.goto(path, , { waitUntil: 'domcontentloaded', timeout: 120000 });

Some sites suggest using { waitUntil: 'load', timeout: 0 } . See https://github.com/puppeteer/puppeteer/blob/main/docs/api.md#pagegotourl-options for all available options.

💻Your Environment

Angular Version:

[12:19:39] → ng --version

     _                      _                 ____ _     ___
    / \   _ __   __ _ _   _| | __ _ _ __     / ___| |   |_ _|
   / △ \ | '_ \ / _` | | | | |/ _` | '__|   | |   | |    | |
  / ___ \| | | | (_| | |_| | | (_| | |      | |___| |___ | |
 /_/   \_\_| |_|\__, |\__,_|_|\__,_|_|       \____|_____|___|
                |___/
    

Angular CLI: 11.2.10
Node: 14.15.5
OS: darwin x64

Angular: 11.2.11
... animations, common, compiler, compiler-cli, core, elements
... forms, language-service, localize, platform-browser
... platform-browser-dynamic, router, service-worker
Ivy Workspace: <error>

Package                         Version
---------------------------------------------------------
@angular-devkit/architect       0.1102.2
@angular-devkit/build-angular   0.1102.10
@angular-devkit/core            11.2.10
@angular-devkit/schematics      11.2.10
@angular/cdk                    11.2.10
@angular/cli                    11.2.10
@angular/flex-layout            10.0.0-beta.32
@angular/material               11.2.10
@schematics/angular             10.2.3
@schematics/update              0.1102.10
ng-packagr                      11.2.4
rxjs                            6.6.7
typescript                      4.0.3

Scully Version:

    "@scullyio/init": "1.0.1",
    "@scullyio/ng-lib": "1.0.0",
    "@scullyio/scully": "^1.1.1-BETA.94615"

We are using the @develop version of Scully as @SanderElias suggested on https://github.com/scullyio/scully/issues/1348#issuecomment-831397694

I hope you can help us with this. We have a CI pipeline for our deploys and I’ll have to make a special version with the workaround to make things work again (the cleanest option is to make a pull request and submit the fix, but … ¯_(ツ)_/¯

Thanks in advance for any help.

[hunhavoc]

Hi @jsanta ,

No need to say sorry, you just made me a favour 😃 Thank you for sharing, it definitely helps a lot!

Stay safe

[SanderElias]

@jsanta that is the reason the enterprise plugins are there, so you can override whatever you need. I designed Scully so, that almost everything is a plugin, that can be swapped out when needed. You even can replace existing plugins if you like.