Cast phase config strings to numbers in order to support ENV vars

The Issue

When using ENV-based config for phases, for example:

config:
  phases:
    - duration: "{{ $processEnvironment.DURATION }}"
      arrivalRate: "{{ $processEnvironment.ARRIVAL_RATE }}"
      rampTo: "{{ $processEnvironment.RAMP_TO }}" # The troublemaker

The scenario does not run as expected (finishes early). In my anecdotal setup the rampTo bit is what introduces the issue.

Why does this happen?

The process environment variables which become the spec entry values are strings, and the various mathematical operations on these here: https://github.com/artilleryio/artillery/blob/74baeac42a7bfb9ad9378ce7b9a01f4535eb1490/core/lib/phases.js#L85-L92

And here: https://github.com/artilleryio/artillery/blob/74baeac42a7bfb9ad9378ce7b9a01f4535eb1490/core/lib/phases.js#L118

Are likely the culprit.

The Solution

PR introduces a conversion of defined phase config entries (based on a keys whitelist) expected to have numerical values.

[idpbond]

Hi @xyngfei

Sorry for not double-checking, but it looks like the env-casting feature #978 + #1025 was not yet merged into a tagged release.

I’ll have to defer to @hassy as I’m not sure how the folks at artillery do master -> x.y.z release merging

However, if you must use this feature and can’t wait for a tagged version, you can do the following:

# Install artillery@master from github directly
npm i -D git://github.com/artilleryio/artillery#master

# demo.yml
config:
  target: "https://example.com"
  # --- Commenting out the plugin ---
  # plugins:
  #   expect: {}
  phases:
    - name: "ramp up"
      duration: "{{ $processEnvironment.art_sustain_duration }}"
      rampTo: "{{ $processEnvironment.art_rampto }}"
      arrivalRate: "{{ $processEnvironment.art_arrivalRate }}"
scenarios:
  - name: "test demo site"
    flow:
      - post:
          url: "/api"
          capture:
            - json: "$.success"
              as: success
            - json: "$.errors"
              as: errors
          expect:
            - statusCode: 404

art_arrivalRate=1 art_rampto=20 art_sustain_duration=30 ./node_modules/.bin/artillery run --output report.json demo.yml

Which, for me, produced a successful (in that it finished) test:

************************************************************

All virtual users done. Summary report:

Total time: 31 seconds
http.request_rate: .................................. 12/sec
http.response_rate: ................................. 12/sec
vusers.created_by_name.test demo site: .............. 160
vusers.created.total: ............................... 160
vusers.failed: ...................................... 161
http.requests: ...................................... 160
http.codes.404: ..................................... 161
http.responses: ..................................... 161
http.response_time:
  min: .............................................. 88.2
  max: .............................................. 354.4
  median: ........................................... 133
  p95: .............................................. 206.5
  p99: .............................................. 262.5

[idpbond]

@xyngfei

1. Does this happen on artillery@1.7.2 ?
2. If so, can you provide a stripped down way (CLI or yml) to reproduce the issue?