Summary mode
See original GitHub issueThanks again for the excellent work. Here’s a suggestion to hopefully make the CLI more versatile.
Goals
- focus on only high level info
- efficiently go through many requests
- use in continuous monitoring and trigger alarms
Proposal
There is a helpful --verbose
mode, but there doesn’t seem to be a way to make httpyac more concise.
For just one request being tested in the normal mode, with a tiny body response:
https://the.server.dev/checks/health => 200
GET https://the.server.dev/checks/health
accept: */*
accept-encoding: gzip, deflate, br
user-agent: httpyac
HTTP/1.1 200 OK
alt-svc: h3-27=":443"; ma=86400, h3-28=":443"; ma=86400, h3-29=":443"; ma=86400, h3=":443"; ma=86400
cf-ray: 66b9e0e55d873469-NRT
connection: close
content-length: 15
content-type: application/json
date: Thu, 08 Jul 2021 14:07:22 GMT
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
nel: {"report_to":"cf-nel","max_age":604800}
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v2?s=u2dylFQnhzyetY8zUENOcv4l0%2FvmxRW8%2FmAKXfC0Kyna8uVZddEaWx6oSfQBJfM%2FwDfKchWeiwRdwhXSP7VykaQnTuM0mNTl0HCkd4%2BWhjtUB%2FcmtTroVQnfMEe%2Bs%2F8dw8Lh"}],"group":"cf-nel","max_age":604800}
server: cloudflare
vary: Accept-Encoding
{
"status": "OK"
}
Tests for /checks/health
✓ Request succeeded
✖ Schema is valid (AssertionError [ERR_ASSERTION]: Response schema is incorrect: does not exactly match expected constant: ok - t.IntellijHttpClient.assert (/Users/name/.nvm/versions/node/v14.15.5/lib/node_modules/httpyac/dist/index.js:1:32228)
This is very helpful a lot of the time, but it is a bit difficult to use when running large collections of tests, or tests with large bodies, or if trying to use it in a continuous monitoring situation.
We could imagine something like this as an extra output mode:
Summary line
The summary line would be of some simple format like:
? requests tested (? succeeded, ? failed with ? CRITICAL)
So that an evaluation script may look at the last line and check that 0 failed
or 0 CRITICAL
is found, or do something if not (like trigger alerts, save or forward the full log…).
Options
Perhaps something like:
--summary summary as above, with all requests included
--summary --failed same format but showing just requests involving at least one failed test
--summary --critical same format but showing just requests which have CRITICAL failures
--summary --json outputs everything in json format for easy parsing
Issue Analytics
- State:
- Created 2 years ago
- Comments:20 (12 by maintainers)
Top GitHub Comments
Yes, your guess is right. The Name of a request is used as variable name (used with
@ref
). I wanted to test, if I can reuse the title as name, but I guess not:-) I will change it to title. You don’t have to feel obligated to test. I think it’s great, but it should be fun for us first and foremost.updated npm to 2.20.0-beta.2
Thanks again for all the ideas. I think I currently have everything covered in this ticket. I will deal with anweber/vscode-httpyac#50 next