[📜] Spec for declaration of vulns to help scanners check find rate

Back in 2016 an idea of having a __vulns.json file in vulnerable applications came up and was prepared by members of the OWASP ZAP, VWAD and Juice Shop teams. It was supposed to allow scanners/tools to assess their success rate by

1. iterating over the vulnerabilities array
2. looking for one or more matches from each flags array in their own reported finding descriptions/root causes etc.

The Juice Shop version of this file looked like this back then:

{
  "application" : "juice-shop",
  "vulnerabilities" : [
    {"type" : "xss", "flags" : ["/#/search", "q="] },
    {"type" : "sqli", "flags" : ["/#/search", "q="] },
    {"type" : "sqli", "flags" : ["/#/login", "email"] },
    {"type" : "sqli", "flags" : ["/#/login", "password"] },
    {"type" : "crypto", "flags" : ["/ftp/eastere\\.gg", "base64", "L2d1ci9xcmlmL25lci9mYi9zaGFhbC9ndXJsL3V2cS9uYS9ybmZncmUvcnR0L2p2Z3V2YS9ndXIvcm5mZ3JlL3J0dA=="] },
    {"type" : "access", "flags" : ["/#/administration"] },
    {"type" : "access", "flags" : ["/#/score-board"] },
    {"type" : "hash", "flags" : ["/api/Users", "md5"] },
    {"type" : "hash", "flags" : ["/api/Users/[0-9]*", "md5"] },
    {"type" : "access", "flags" : ["/api/Feedbacks/[0-9]*", "delete"] },
    {"type" : "access", "flags" : ["/api/BasketItems/[0-9]*", "put", "quantity"] },
    {"type" : "session", "flags" : ["sessionStorage", "bid" ] },
    {"type" : "csrf", "flags" : ["/rest/user/change-password", "get" ] },
    {"type" : "trustbound", "flags" : ["/rest/user/change-password.*^((?!current).)*$", "new", "repeat" ] },
    {"type" : "trustbound", "flags" : ["/#/complain", "/file-upload", "size" ] },
    {"type" : "trustbound", "flags" : ["/#/complain", "/file-upload", "type" ] },
    {"type" : "session", "flags" : ["/#/contact", "userId", "hide" ] },
    {"type" : "xss", "flags" : ["/#/contact", "comment" ] },
    {"type" : "xss", "flags" : ["/#/contact", "comment" ] },
    {"type" : "crypto", "flags" : ["/#/basket", "coupon", "(z85|base85)" ] },
    {"type" : "access", "flags" : ["/the/devs/are/so/funny/they/hid/an/easter/egg/within/the/easter/egg" ] },
    {"type" : "access", "flags" : ["css/geo-bootstrap/swatch/bootstrap\\.css" ] },
    {"type" : "access", "flags" : ["/i18n/tlh\\.json" ] },
    {"type" : "access", "flags" : ["/ftp/.*%2500\\.md" ] },
    {"type" : "access", "flags" : ["/ftp/.*%2500\\.pdf" ] },
    {"type" : "access", "flags" : ["/ftp/.*?md_debug=.*\\.pdf" ] },
    {"type" : "access", "flags" : ["/ftp/.*?md_debug=.*\\.md" ] },
    {"type" : "redirect", "flags" : ["/redirect?to=.*?.*=https://github.com/bkimminich/juice-shop" ] },
    {"type" : "redirect", "flags" : ["/redirect?to=.*?.*=https://blockchain.info/address/1FXJq5yVANLzR6ZWfqPKhJU3zWT3apnxmN" ] },
    {"type" : "redirect", "flags" : ["/redirect?to=.*?.*=https://gratipay.com/juice-shop" ] },
    {"type" : "redirect", "flags" : ["/redirect?to=.*?.*=http://flattr.com/thing/3856930/bkimminichjuice-shop-on-GitHub" ] },
    {"type" : "xss", "flags" : ["/#/administration", "post", "/api/Users", "email" ] },
    {"type" : "xss", "flags" : ["/#/search", "post", "/api/Products", "description" ] },
    {"type" : "xss", "flags" : ["/#/search", "put", "/api/Products/[0-9]*", "description" ] },
    {"type" : "crypto", "flags" : ["/#/score-board", "continueCode", "hashid" ] }
  ]
}

The JSON spec was the following (see https://github.com/OWASP/OWASP-VWAD/blob/6a8b54004db5b48278e82f2cc856fdc229c80ca3/src/owasp-wiki/deprecated/vulns-json/schema.json)

{
    "$schema": "http://json-schema.org/draft-04/schema#",
    "title": "Vulnerability matchers for an intentionally vulnerable web application",
    "type": "object",
    "properties": {
        "application": { "description": "The unique name of the vulnerable web application", "type": "string" },
        "vulnerabilities": {
                "type": "array",
                "items": {
                    "type": "object",
                    "properties": {
                        "type" : { "enum": [ "xss", "sqli", "crypto", "access", "hash", "session", "csrf", "trustbound", "redirect", "leakage" ] },
                        "flags": {
                          "type": "array",
                          "items": {
                              "type": "string"
                          },
                          "minItems": 1,
                          "uniqueItems": true
                        }
                    },
                    "required": ["type", "flags"]
                },
                "uniqueItems": true
            }
    },
    "required": ["application", "vulnerabilities"]
}

Because the idea was not pursued further and it saw no adoption, it was dropped in 2017.

Now, 3 years later, let’s try that again, shall we?

In this issue ideas can be collected and the original schema can be taken apart. Personally, I still think a light-weight approach is a good idea, but some meta data could be added (like URL of the application repository as a unique identifier) or existing one improves (like the - in hindsight - seemingly arbitrary enumeration of type entries) - Any constructive criticism is welcome.

To express if you think this idea is good (👍) or worthless (👎), please use the reactions on this issue instead of posting +1/-1 comments. Thanks!

[bkimminich]

Okay, actually we could throw the flags into different arrays, as already done with type. Then each scanner vendor can still decide which of the lists they want to use. Like:

• flags
  • source
  • runtime
  • classifiers
  • …

[preetkaran20]

But URL might remain the same and that can be mandatory and in case of minified versions we need not to qualify the parameter as mandatory because it changes at runtime. If we think about server side vulnerabilities like SQLInjection, it can be very important like say 2 levels are there sqli/level1?q=<something> and sqli/level2?q=<something> even if scanner has not found the vulnerability in second endpoint still it can claim that it has found the vulnerability (just a way to reduce the false positives).