Update typings for functions that wrap `request()` to return `Response` when using `rawResponse`

As I commented on #462, the current typings for e.g. queryFeatures say that the method returns Promise<IQueryFeaturesResponse | IQueryResponse>. The method actually returns a Promise<Response> when the rawResponse option is true.

It would be nice if the method could return the correct promise type directly based on the shape of the options object passed, either via implied generics or using a (granted, pretty verbose) overload. Overloading would be verbose but I don’t think especially difficult – see this Angular class for an example. Much like this library, the Angular HTTPClient has a request method that changes its return type based on an options-object, and multiple helpers that call request and also vary the return type based on arguments.

[patrickarlt]

@gavinr @tomwayson and I discussed this in an internal meeting. We all agreed that this is pretty important for some methods (queryFeatures, request) but really pointless for most methods createItem, geocode.

There are 2 main downsides to this:

1. In order to do this each call to request has to be wrapped in an if statement for each format in order for TypeScript to trace the types properly. Istanbul will treat each if statement as something we have to write tests for to get test coverage. Which really isn’t needed.
2. It really is a ton of work when you consider all 100+ methods in Rest JS. Here a quick before/after:

   /**
   * Before
   */
   function create(options: CreateOptions): Promise<CreateResponse> {
     const baseRequestOptions = {
       url: "..."
     }
     
     return request<CreateResponse>({
       ...baseRequestOptions, format: "json"
     });
   }
   
   /**
   * After
   */
   // adding overrides are fairly easy...
   function create(options: CreateOptions & { format?: "json" , raw?: false}): Promise<CreateResponse>;
   function create(options: CreateOptions & { format?: "html", raw?: false }): Promise<string>;
   function create(options: CreateOptions & { raw?: true , format?: "json" | "html"}): Promise<Response>;
   function create(options: Partial<CreateOptions> = {format: "json", raw: false}): Promise<Response> | Promise<string> | Promise<CreateResponse> {
     const baseRequestOptions = {
       url: "..."
     }
   
     // conditionalizing all possible returns types feels really silly.
     if (options.raw) {
       return request({
         ...baseRequestOptions, raw: true, format: options.format
       });
     }
   
     if (options.format === "html") {
       return request({
         ...baseRequestOptions, format: "html"
       });
     }
     
     return request<CreateResponse>({
       ...baseRequestOptions, format: "json"
     });
   }
   

I want to sit on this for a bit. I think there definitely needs to be some trimming down of the API surface area. For example:

1. The /addItem endpoint only supports json, pjson and html as valid f params. My preference is to actually drop support for both rawResponse and force f=json. I’m having a hard time justifying use cases where f=html or f=pjson would really be valid OR where I would need to stream a tiny JSON response. This would really reduce the surface area of the API and make most of these cases obsolete.
2. Do we need rawResponse anymore? The main motivation for rawResponse comes from https://github.com/Esri/arcgis-rest-js/issues/373 when our recommended fetch implementation didn’t support response.blob(). With 4.0 we will ship the latest version of node-fetch which DOES support response.blob(). The main motivation for rawResponse now seems to be streaming responses, which is useful for working with large responses like huge queries or files OR for parsing formats we don’t support in REST JS for whatever reason.

I’m leaning towards the following:

1. We should drop support for f=html everywhere unless the endpoint REALLY returns a valid HTML page that developers would want to use. In its current state f=html is really only used for the forms that exist on the server itself. I don’t think there is anywhere in REST JS currently where f=html makes any sense for someone to want to use so we should drop it.
2. Remove rawResponse. Instead REST JS should intelligently handle things like getItemData(), getItemResource() and the missing getAttachment() methods or any other methods that return files/blobs by doing the following:
   1. Make a request with f=json first. This will return JSON data immediately for things like web maps. If you do not pass f=json you get an HTML error by default so this request should be done first to check for errors.
   2. If the item data/resource is NOT JSON then you get a Specified output format not supported. and you can retry the request without f=json and return a Response object that the user can decide how they want to handle with response.blob() or steam response.body.
   3. The response types and doc for methods that return files should be used to point out that this will return an object for JSON data and a Response object for everything else and users should either handle both cases OR check the type of item up front so they know how to parse it.
3. For methods like queryFeatures that support additional response types like geojson or pbf we can write intelligent overrides like I described above. I think queryFeatures might be the only one at this point. queryFeatures also has lots of other special cases like returnCentroids which would be good to handle as well.
4. We should update request like I described above because that is much better for TypeScript.

This REALLY minimizes the surface area of the API to the point where the original issue simply disappears. The only thing we really loose is the ability to stream any request which for 99.9% of requests really doesn’t make sense anyway. Streaming responses also bypasses all error checking authentication management anyway.

If we REALLY feel like specific methods need the option for streaming responses then we could add a stream option in request that could optionally be reflected in parent methods with the caveat that it turns off most of fancy REST JS features.

[tomwayson]

Sorry, I updated my comment for clarity. I do think we should still provide:

the option of getting a raw W3C or Node stream and handling the decode myself.