Incorrect number of requests in summary

Hi! I have the following configuration file:

config:
  http:
    pool: 2
    timeout: 240
  target: 'http://localhost:8080'
  phases:
    - duration: 10
      arrivalRate: 10
scenarios:
  - flow:
      - log: "New user"
      - post:
          url: '/v1/links'
          json:
            foo: 'bar'

I expected to have 100 requests in the summary, but I got a different number:

How is this possible?

[andrewdibiasio6]

Working on versions:

VERSION INFO:
Artillery Core: 2.0.0-11
Node.js: v16.2.0
OS:      darwin

So, I believe that you have multiple tickets open for the same type of issue. I think the issue boils down to: the final summary is not a complete aggregate of all other summary slices. Here is another example that I run on my local machine, not distributed.:

config:
  target: "removed"
  processor: "../bin/my-functions.js"
  phases:
    - duration: 10
      arrivalRate: 3
      name: Warm up

scenarios:
  - name: "Basic Cycle"
    flow:
      - post:
          headers:
            Content-Type: "application/json"
          url: "/cars"
          beforeRequest:
            - "setBody"
          capture:
            - json: "$.id"
              as: "id"
      - think: 60
      - delete:
          url: "/path/{{ id }}"

      ... more rest calls ...

    weight: 1
  - name: "Navigate UI"
    flow:
      - get:
          url: "/path/{{ id }}"
      - think: 10
       ... more rest calls ...
    weight: 10

I expect 30 requests to be created, and I believe they are created, but the final summary is not complete.

Here is an example output:

## SLICE 1 ##
--------------------------------------
Metrics for period to: 16:15:40(-0500) (width: 6.593s)  
-------------------------------------- 

http.codes.200: ................................................................ 82
http.codes.202: ................................................................ 2
http.request_rate: ............................................................. 12/sec
http.requests: ................................................................. 84
http.response_time:
  min: ......................................................................... 2
  max: ......................................................................... 90
  median: ...................................................................... 3
  p95: ......................................................................... 61
  p99: ......................................................................... 70.1
http.responses: ................................................................ 84
vusers.created: ................................................................ 21
vusers.created_by_name.Basic Cycle: ............................................ 3
vusers.created_by_name.Navigate UI: ............................................ 18


## SLICE 2 ##
--------------------------------------
Metrics for period to: 16:15:50(-0500) (width: 9.598s)
--------------------------------------

http.codes.200: ................................................................ 53
http.codes.202: ................................................................ 3
http.request_rate: ............................................................. 11/sec
http.requests: ................................................................. 56
http.response_time:
  min: ......................................................................... 1
  max: ......................................................................... 61
  median: ...................................................................... 5
  p95: ......................................................................... 58.6
  p99: ......................................................................... 59.7
http.responses: ................................................................ 56
vusers.completed: .............................................................. 11
vusers.created: ................................................................ 9
vusers.created_by_name.Basic Cycle: ............................................ 2
vusers.created_by_name.Navigate UI: ............................................ 7
vusers.session_length:
  min: ......................................................................... 13594.2
  max: ......................................................................... 13644.6
  median: ...................................................................... 13497.6
  p95: ......................................................................... 13497.6
  p99: ......................................................................... 13497.6

## SLICE 3 ##
--------------------------------------
Metrics for period to: 16:16:00(-0500) (width: 5.003s)
--------------------------------------

vusers.completed: .............................................................. 14
vusers.session_length:
  min: ......................................................................... 13592.7
  max: ......................................................................... 13601.3
  median: ...................................................................... 13497.6
  p95: ......................................................................... 13497.6
  p99: ......................................................................... 13497.6

## SLICE 4 ##
--------------------------------------
Metrics for period to: 16:16:40(-0500) (width: 2.213s)
--------------------------------------

http.codes.204: ................................................................ 4
http.request_rate: ............................................................. 4/sec
http.requests: ................................................................. 4
http.response_time:
  min: ......................................................................... 3
  max: ......................................................................... 211
  median: ...................................................................... 4
  p95: ......................................................................... 210.6
  p99: ......................................................................... 210.6
http.responses: ................................................................ 4
vusers.completed: .............................................................. 2
vusers.session_length:
  min: ......................................................................... 61284.2
  max: ......................................................................... 61284.9
  median: ...................................................................... 61717.2
  p95: ......................................................................... 61717.2
  p99: ......................................................................... 61717.2

## SLICE 5 ##
--------------------------------------
Metrics for period to: 16:16:50(-0500) (width: 3.206s)
--------------------------------------

http.codes.204: ................................................................ 6
http.request_rate: ............................................................. 2/sec
http.requests: ................................................................. 6
http.response_time:
  min: ......................................................................... 3
  max: ......................................................................... 225
  median: ...................................................................... 4
  p95: ......................................................................... 210.6
  p99: ......................................................................... 210.6
http.responses: ................................................................ 6
vusers.completed: .............................................................. 3
vusers.session_length:
  min: ......................................................................... 61279.3
  max: ......................................................................... 61289.8
  median: ...................................................................... 61717.2
  p95: ......................................................................... 61717.2
  p99: ......................................................................... 61717.2

## Final Summary ##
All VUs finished. Total time: 1 minute, 12 seconds

--------------------------------
Summary report @ 16:16:44(-0500)
--------------------------------

http.codes.200: ................................................................ 94
http.codes.202: ................................................................ 2
http.codes.204: ................................................................ 4
http.request_rate: ............................................................. 5/sec
http.requests: ................................................................. 100
http.response_time:
  min: ......................................................................... 1
  max: ......................................................................... 211
  median: ...................................................................... 4
  p95: ......................................................................... 61
  p99: ......................................................................... 210.6
http.responses: ................................................................ 100
vusers.completed: .............................................................. 20
vusers.created: ................................................................ 20
vusers.created_by_name.Basic Cycle: ............................................ 2
vusers.created_by_name.Navigate UI: ............................................ 18
vusers.session_length:
  min: ......................................................................... 13592.7
  max: ......................................................................... 61284.9
  median: ...................................................................... 13497.6
  p95: ......................................................................... 61717.2
  p99: ......................................................................... 61717.2

You can see there are no failure codes, all requests passed. In slice 1, I created 21 vusers, which makes sense because ~ 7 seconds went by. In slice 2, I create the final 9 vusers. So 30 vusers are runing. If you go back and count, it looks like 30 vusers also end up completed.

So I think that the issue is how you calculate the final summary.

An important thing to note is I would expect the http.max to be 225 not 211 and the number of http.request to be over 100.

Is it possible that failures are happening and that could be why it is incorrect?

[hassy]

thanks for the extra info @spinlud! you’re right, requests that timeout are still meant to be included in the report (as ETIMEOUT errors)