application/json default charset is not UTF-8 when parsing request

This is a follow up to #80. Apparently, responses were fixed, but there are still issues with the requests.

Request with Content-Type: application/json is decoded not as UTF-8, while request with Content-Type: application/json; charset=utf-8 is decoded correctly. Both has to behave the same and be decoded as UTF-8.

[functionaldude]

Use this function if you just want to “have it work”:

/**
 * Receive the request as String.
 * If there is no Content-Type in the HTTP header specified use ISO_8859_1 as default charset, see https://www.w3.org/International/articles/http-charset/index#charset.
 * But use UTF-8 as default charset for application/json, see https://tools.ietf.org/html/rfc4627#section-3
 */
private suspend fun ApplicationCall.receiveTextWithCorrectEncoding(): String {
  fun ContentType.defaultCharset(): Charset = when (this) {
    ContentType.Application.Json -> Charsets.UTF_8
    else -> Charsets.ISO_8859_1
  }
  
  val contentType = request.contentType()
  val suitableCharset = contentType.charset() ?: contentType.defaultCharset()
  return receiveStream().bufferedReader(charset = suitableCharset).readText()
}

@cy6erGn0m Maybe you want to change the default implementation of receiveText to the implementation of receiveTextWithCorrectEncoding()?

[clydebarrow]

I have run into this problem as well. The receiveTextWithCorrectEncoding() work-around solved it for now, but it seems like it is still an issue.