Feature Request: Enable JSON serialization in Request and JSON parsing in Response separately

Summary

Note: This is a Feature Request. I’m willing to discuss the 2 solutions I suggest, and already implemented both of them with success; I will submit the PR as soon as discussion on this Feature Request is able to settle on one of the 2 solutions

I think that it would be useful to be able to enable separately the JSON serialization of a Request’s data, and the JSON parsing of the Response’s data. Currently, both of this parts are handled by the same json param which makes it a pain for people who might want one and not the other.

Simplest Example to Reproduce

This example will be based on https://hellojson.horgix.eu is the most simple API of the world and will just answer this with application/json Content-Type wether you GET or POST on / :

{
    "hello": "world"
}

So, let’s send a simple POST to it:

$ cat test.js
var request = require("request");
request(
        {
          method: 'POST',
          url: 'https://hellojson.horgix.eu',
          body: '"feature request"',
          json: true
        },
        function(err, httpResponse, body){
          console.log(body)
          console.log('hello' in body)
        }
)

and the result:

$ node test.js
{ hello: 'world' }
true

This is fine. Now just switch the json: true to json: false in order to not encode the request body ("feature request") in JSON, and we get this:

$ node test.js
{
  "hello": "world"
}

./test.js:12
          console.log('hello' in body)
                              ^

TypeError: Cannot use 'in' operator to search for 'hello' in {
  "hello": "world"
}

    at Request._callback (./test.js:12:31)
    at Request.self.callback (./node_modules/request/request.js:188:22)
    at emitTwo (events.js:106:13)
    at Request.emit (events.js:194:7)
    at Request.<anonymous> (./node_modules/request/request.js:1171:10)
    at emitOne (events.js:96:13)
    at Request.emit (events.js:191:7)
    at IncomingMessage.<anonymous> (./node_modules/request/request.js:1091:12)
    at Object.onceWrapper (events.js:293:19)
    at emitNone (events.js:91:20)

And this is what I want to talk about.

Current Behavior

This request lib exposes the json parameter for requests, allowing the user to enable a set of things:

From https://github.com/request/request#requestoptions-callback :

body - entity body for PATCH, POST and PUT requests. Must be a Buffer, String or ReadStream. If json is true, then body must be a JSON-serializable object.

[…]

json - sets body to JSON representation of value and adds Content-type: application/json header. Additionally, parses the response body as JSON.

The usecase that I’m concerned with is the case where we need to do a request that is not a JSON request, but where the expected response is a JSON response, so where we want to be able to use the “Additionally, parses the response body as JSON” part despite this json param being set to false.

Actually, as told in the documentation quoted some lines ago, the JSON parsing of the response is only enabled if the request itself is declared as json:

From https://github.com/request/request/blob/master/request.js#L1152 :

if (self._json) {
  try {
    response.body = JSON.parse(response.body, self._jsonReviver)

Expected Behavior

I’m feeling like it would make sense to be able to enable this response parsing without depending on the json parameter; this is actually a need that I find myself with, and which is described in details below.

Possible Solution

I see 2 possibles solutions:

1. Add a json_response parameter that enables response parsing even if json is set to false
   • Advantages: quite simple, just implies adding a new parameter and documenting it
   • Disadvantages: it would have to be clear what the default behavior is in case it’s not defined
2. Add something a bit smarter, which parses the JSON in the Response body if the Response comes with application/json as Content-Type, which would totally make sense
   • Advantages: cleaner, imho
   • Disadvantages: might be a breaking change for people who are relying on response not being parsed right now

Context

I’m using chakram (site, github) to test a REST API. This is just an extension for chai (site, github) that uses request for requests, and which is run by mocha (site, github).

Among my tests, I would like to test the edge case where I POST an invalid JSON to the API on purpose, but with a Content-Type set correctly to application/json; the goal being to ensure that my API correctly detects the invalid JSON data and handles it propertly by sending back an HTTP 400 error.

Concretely, this result in the following test: https://gist.github.com/Horgix/6a612f2a86c7507663012f530ed66c59#file-test_invalid_json-js

The "json": false is here to disable the JSON serialization of my data by request lib (it is automatically enabled by chakram), and the manual definition of the Content-Type header is here due to the fact that this is usually done by the json param, which I just explicitely set to false.

My API returns an error looking like this:

HTTP/1.0 400 BAD REQUEST
Content-Type: application/json

{
  "error": "400 Bad Request: Failed to decode JSON object: Unterminated string starting at: line 1 column 9 (char 8)"
}

Until this point, everything is fine.

Except that, as explained in the Current Behavior part, I’m setting the json param to false (since I explicitely want to avoid my request’s body from being serialized into JSON), so I don’t benefit from the “Additionally, parses the response body as JSON.” part, despite the fact that I really want to use it since I’m going to verify this response against a JSON schema (see test linked above). This is actually on this part that my test fails, since the response body is no longer parsed and is only a plain string, which obviously doesn’t match whatever JSON schema that expects a dict (which would be got from parsing the reponse body).

That’s at this point that I took a look at how this json think is handled and that I felt the need of this Feature Request. Of course I could parse the response afterward by myself in a single line, but this is something I would expect request being able to do.

Your Environment

Not really relevant. Anyway I’m using nodejs v7.10.0 with request v2.81.0.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[newhouse]

+1 for this. While for me it’s not “common”, I have just now come across a scenario where I need to send over non-JSON and expect JSON in the response.