question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Double encoded URLs

See original GitHub issue

Describe the bug After migrating to cloud 2020.0.0-M6 from Hoxton.SR8 I started to have problems with double encoded URLs. It most probably only affects URL containing multiple = charactors in one query parameter. Problem most probably is that ServerWebExchangeUtils#containsEncodedParts reports false - taking URL as not encoded.

Sample

Exception is trapped in:

catch (IllegalArgumentException ignore) {
  if (log.isTraceEnabled()) {
    log.trace("Error in containsEncodedParts", ignore);
  }
}

java.lang.IllegalArgumentException: Invalid character '=' for QUERY_PARAM in "key==%22value%22"

For URL:

/filter?query=key=="value"&page=0&size=30

This “kind” of URL should be valid and is used by https://github.com/jirutka/rsql-parser or FIQL (https://tools.ietf.org/html/draft-nottingham-atompub-fiql-00#section-4)

Seems the validation is too strict. As it cannot be replaced with custom implementation, it would essentially mean rewriting the RouteToRequestUrlFilter filter (and potentially others, that use ServerWebExchangeUtils#containsEncodedParts).

Issue Analytics

  • State:open
  • Created 3 years ago
  • Reactions:13
  • Comments:15 (3 by maintainers)

github_iconTop GitHub Comments

8reactions
maresja1commented, Feb 4, 2021

@spencergibb I apologize for approaching you directly, but is there a chance, someone else could look at this? I am confident this is a bug (please see the unit test I provided) and I think there will be more users affected by it. As SGC should be a generic tool, it shouldn’t restrict the form of URLs more than the HTTP specification does.

I considered creating PR that fixes the issue, but I think it needs deeper analysis and considerations.

We really like SGC and we would like to upgrade to the newer version, but this prevents us from doing so. Thank you in advance.

2reactions
flyoncecommented, Jan 7, 2021

@ctlove0523
I had meet same problem, in my case the gateway incoming uri encoded as:

/get_rptlist?client=011&rptType=[%22PY%22]

the detail debug info show as belows invalid char

Read more comments on GitHub >

github_iconTop Results From Across the Web

Double Encoding | OWASP Foundation
This attack technique consists of encoding user request parameters twice in hexadecimal format in order to bypass security controls or cause unexpected behavior ......
Read more >
Double URL Encoding | Imperva
Double URL Encoding Summary SecureSphere has detected an HTTP request that has at least one double URL encoded character in it. Detailed Description...
Read more >
Double encoding | Mastering Modern Web Penetration Testing
Double percent encoding is the same as percent encoding with a twist that each character is encoded twice instead of once. This technique...
Read more >
html - How do I avoid double URL encoding when rendering ...
If you want to avoid double encoding the links you can just use urldecode() on both links, and then urlencode() afterwards, as decoding...
Read more >
Usage of Double Encoding - Application Security
Double Encoding is an attack technique that can be used to bypass certain blacklisting-filter mechanism. Suppose an application is trying to ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found