Should URLSearchParams be used in node.js 8 instead of querystring?

Node >= 8.0 introduced an issue in querystring.parse (see here https://github.com/nodejs/node/issues/13773) that will affect all users of body-parser. It occurs when there is a trailing whitespace in a query string parameter:

Before require('querystring').parse('a=%20+&') was { a: ' ' }

In node 8.x require('querystring').parse('a=%20+&') is { a: '%20 ' }

This applies to any parameter value ending with a space.

Even though it is not body-parser’s fault, this issue can cause non trivial problems that are very hard to track down. By updating any app that uses body-parser on older node versions to node 8.x, any url-encoded form input with a trailing space will start to have the encoding of special characters broken. It took us a long time to discover the issue and to find out where the problem was. In the issue mentioned above, it is suggested that the new URLSearchParams class is a faster way of parsing query strings and more similar to native browser implementations. Does it make sense to consider it?

[zemlanin]

blows the dust off

I was trying to match the behavior of bodyParser.urlencoded with node’s url.searchParams and stumbled upon this issue…

Is it time to revisit this discussion?

[TimothyGu]

Is URLSearchParams just a drop-in replacement for the querystring module or are there other changes we’ll need to make in order to use it correctly.

No, URLSearchParams has a different API. It has the exact same interface as the URLSearchParams exposed in browsers.

• params[key] = value is params.set(key, value)
• Array.isArray(params[key]) ? params[key].push(value) : params[key] = [params[key], value] is params.append(key, value)
• delete params[key] is params.delete(key)
• key in params is params.has(key)
• params[key] is params.get(key) or params.getAll(key)
• Object.keys(params) is Array.from(params.keys()) (with the caveat that the latter would return duplicate keys)
• querystring.stringify(params) is params.toString()

---

If you intend for that module to be used for urlencoded bodies, can you at least add a mode switch to the module to allow users to indicate they are trying to parse a urlencoded body, rather than people are supposed to know they need to prepend a ? to the string before parsing? I also don’t see any support for maxKeys in URLSearchParams, which we use in this module.

It would not be possible for us to add any options not already in the spec to the URLSearchParams constructor. But we might make a custom API like url.parseSearchParams() that takes options, much like the custom url.format() we have for URL. Even if we do add such an API, in general I would still encourage the use of URLSearchParams class if possible as it has a more defined API in the entire JS world.

---

it would be a great time to move towards the custom charset support (or, at least the path to add _charset_ is clear)

I assume you are talking about decoding/parsing with custom charsets here.

Currently, the Node.js core only has Latin-1/UTF-8/UTF-16 decoders built-in. Based on this and WHATWG’s stance

A legacy server-oriented implementation might have to support encodings other than UTF-8 as well as have special logic for tuples of which the name is _charset [sic]. Such logic is not described here as only UTF-8 is conforming.

the URLSearchParams constructor will probably never support _charset_. But when https://github.com/nodejs/node/pull/13644 lands, and if we do decide to add the aforementioned custom parser API, supporting _charset_ should be easy enough on our side to add an option for.