Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Ramp Up phase produces spike in VUs

See original GitHub issue

Version info: 

Artillery Core: 2.0.0-23

With these phases and scenarios defined in my config:

  phases:
    - duration: 60
      arrivalCount: 1
      name: Start Up
    - duration: 900
      arrivalRate: 1
      rampTo: 25
      name: Ramp Up

scenarios:
  - name: HomePage
    engine: playwright
    flowFunction: 'homePage'
    flow: []
    weight: 5
  - name: PlpPage
    engine: playwright
    flowFunction: 'plpPage'
    flow: []
    weight: 3
  - name: PdpPage
    engine: playwright
    flowFunction: 'pdpPage'
    flow: []
    weight: 2

I expected to see this happen:

Linear-ish ramp up from 1 to 25 users over 15 minutes

Instead, a large spike in users was created at the beginning of the ramp up phase, followed by timeout errors and most of the time the worker dies (some metrics and errors removed for brevity):

Running scenarios...
Phase started: Start Up (index: 0, duration: 60s) 11:15:52(-0400)

Phase completed: Start Up (index: 0, duration: 60s) 11:16:52(-0400)

Phase started: Ramp Up (index: 1, duration: 900s) 11:16:52(-0400)

--------------------------------------
Metrics for period to: 11:17:00(-0400) (width: 7.948s)
--------------------------------------

engine.browser.http_requests: .................................................. 1874
engine.browser.memory_used_mb:
  min: ......................................................................... 13.4
  max: ......................................................................... 21.3
  median: ...................................................................... 15.6
  p95: ......................................................................... 20.3
  p99: ......................................................................... 20.3
user.status_check_ok.: .......................... 8
user.status_check_ok.shop/mens: ................ 1
vusers.completed: .............................................................. 8
vusers.created: ................................................................ 71
vusers.created_by_name.HomePage: ............................................... 33
vusers.created_by_name.PdpPage: ................................................ 8
vusers.created_by_name.PlpPage: ................................................ 30
vusers.failed: ................................................................. 0
vusers.session_length:
  min: ......................................................................... 2419.9
  max: ......................................................................... 6329.4
  median: ...................................................................... 2671
  p95: ......................................................................... 6187.2
  p99: ......................................................................... 6187.2


--------------------------------------
Metrics for period to: 11:17:10(-0400) (width: 9.919s)
--------------------------------------

engine.browser.http_requests: .................................................. 2121
engine.browser.memory_used_mb:
  min: ......................................................................... 16.5
  max: ......................................................................... 24.3
  median: ...................................................................... 20.3
  p95: ......................................................................... 21.5
  p99: ......................................................................... 21.5
user.status_check_ok.: .......................... 8
user.status_check_ok....  1
user.status_check_ok....  1
user.status_check_ok....  1
user.status_check_ok....  2
user.status_check_ok.shop/mens: ................ 4
user.status_check_ok.shop/womens: .............. 3
vusers.completed: .............................................................. 19
vusers.created: ................................................................ 81
vusers.created_by_name.HomePage: ............................................... 38
vusers.created_by_name.PdpPage: ................................................ 12
vusers.created_by_name.PlpPage: ................................................ 31
vusers.failed: ................................................................. 0
vusers.session_length:
  min: ......................................................................... 6924.8
  max: ......................................................................... 16522.3
  median: ...................................................................... 10407.3
  p95: ......................................................................... 12968.3
  p99: ......................................................................... 12968.3

Errors like the below start to occur (10 total):

page.goto: Timeout 30000ms exceeded.
=========================== logs ===========================
navigating to "shop/womens", waiting until "load"
============================================================
    at visitPage (support/playwright/visit-storefront-page.js:2:31)
    at plpPage (support/playwright/visit-storefront-page.js:16:9)
    at Array.scenario (node_modules/artillery-engine-playwright/index.js:120:15) {
  name: 'TimeoutError'
}

And one:

worker error, id: 3 page.evaluate: Target closed
    at Page.<anonymous> (node_modules/artillery-engine-playwright/index.js:106:60) {
  name: 'Error'
}

--------------------------------------
Metrics for period to: 11:17:20(-0400) (width: 9.897s)
--------------------------------------

engine.browser.http_requests: .................................................. 1679
engine.browser.memory_used_mb:
  min: ......................................................................... 16.1
  max: ......................................................................... 20.2
  median: ...................................................................... 17.6
  p95: ......................................................................... 19.9
  p99: ......................................................................... 19.9
user.status_check_ok.: .......................... 6
user.status_check_ok....  1
user.status_check_ok.shop/mens: ................ 5
user.status_check_ok.shop/womens: .............. 5
vusers.completed: .............................................................. 14
vusers.created: ................................................................ 61
vusers.created_by_name.HomePage: ............................................... 32
vusers.created_by_name.PdpPage: ................................................ 8
vusers.created_by_name.PlpPage: ................................................ 21
vusers.failed: ................................................................. 0
vusers.session_length:
  min: ......................................................................... 14787.7
  max: ......................................................................... 24202.2
  median: ...................................................................... 18588.1
  p95: ......................................................................... 23630.3
  p99: ......................................................................... 23630.3

More page.goto: Timeout 30000ms exceeded. errors

--------------------------------------
Metrics for period to: 11:17:30(-0400) (width: 9.669s)
--------------------------------------

engine.browser.http_requests: .................................................. 1832
engine.browser.memory_used_mb:
  min: ......................................................................... 15.8
  max: ......................................................................... 21.7
  median: ...................................................................... 16.9
  p95: ......................................................................... 19.9
  p99: ......................................................................... 19.9
user.status_check_ok.: .......................... 6
user.status_check_ok....  1
user.status_check_ok....  1
user.status_check_ok.shop/mens: ................ 1
user.status_check_ok.shop/womens: .............. 2
vusers.completed: .............................................................. 12
vusers.created: ................................................................ 71
vusers.created_by_name.HomePage: ............................................... 36
vusers.created_by_name.PdpPage: ................................................ 10
vusers.created_by_name.PlpPage: ................................................ 25
vusers.failed: ................................................................. 2
vusers.session_length:
  min: ......................................................................... 21222.9
  max: ......................................................................... 31406.1
  median: ...................................................................... 27730.6
  p95: ......................................................................... 31266.3
  p99: ......................................................................... 31266.3

More worker error, id: 6 page.evaluate: Target closed type errors

Last metrics output:

--------------------------------------
Metrics for period to: 11:17:40(-0400) (width: 9.866s)
--------------------------------------

engine.browser.http_requests: .................................................. 478
engine.browser.memory_used_mb:
  min: ......................................................................... 18.1
  max: ......................................................................... 19.8
  median: ...................................................................... 18
  p95: ......................................................................... 18
  p99: ......................................................................... 18
user.status_check_ok....  1
vusers.completed: .............................................................. 2
vusers.created: ................................................................ 16
vusers.created_by_name.HomePage: ............................................... 7
vusers.created_by_name.PdpPage: ................................................ 6
vusers.created_by_name.PlpPage: ................................................ 3
vusers.failed: ................................................................. 3
vusers.session_length:
  min: ......................................................................... 35093.5
  max: ......................................................................... 37909.4
  median: ...................................................................... 35252.8
  p95: ......................................................................... 35252.8
  p99: ......................................................................... 35252.8

Playwright test file being used:

async function visitPage(page, url, events) {
  const response = await page.goto(url);
  if (response.status() !== 200) {
    return events.emit('counter', `user.status_check_failed.${url}`, 1);
  }
  return events.emit('counter', `user.status_check_ok.${url}`, 1);
}

async function pdpPage(page, userContext, events) {
  const url = userContext.vars.pdpUrl;
  await visitPage(page, url, events);
}

async function plpPage(page, userContext, events) {
  const url = userContext.vars.plpUrl;
  await visitPage(page, url, events);
}

async function homePage(page, userContext, events) {
  const url = userContext.vars.homeUrl;
  await visitPage(page, url, events);
}

module.exports = { pdpPage, plpPage, homePage };

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:6 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
juanig1commented, Oct 31, 2022

Hey, these two recent prs should have fixed the rampup behavior (#1621 and #1633). They are live on release 2.0.0-24.

I also need to use maxVusers in addition to rampTo?

If your scenarios take longer than 1s to complete, that means your test target will end up serving more users than what you set as rampTo/arrivalRate values. To avoid that you can set maxVusers to ensure artillery only has that amount of users at any specific time in the test

0reactions
tchambers97commented, Oct 31, 2022

Ok, I’ll upgrade and give that a try. Thanks!

Read more comments on GitHub >

github_iconTop Results From Across the Web

JMeter Ramp-Up Period: The Ultimate Guide - BlazeMeter
In this article, we are going to focus on applying JMeter ramp-up usage best practices to mimic different load cases for our application....
Read more >
Ramp-Up: Definition, How It Works, Business Examples
A ramp-up is a significant increase in the level of output of a company's products or services in anticipation of an imminent increase...
Read more >
The Importance of Ramp Up and Ramp Down User Load
Ramp up and ramp down helps us to test server auto scaling, as when server sees requests are on an increasing trend, it...
Read more >
Dosing Modifications for CLL/SLL
Interrupt dosing or reduce dose for select adverse reactions. For patients having a dosage interruption lasting more than 1 week during the ramp-up...
Read more >
Ramping up at warp speed | McKinsey
Semiconductor manufacturers are in an aggressive race to win major new designs and meet their customers' demands for rapid time to market.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

Lambda: Cannot read properties of null (reading 'toString')

Next page

worker error, id: 1 Error: Cookie failed to parse