Handling a BOM with .json()

Anybody know how I should be handling a BOM with a returned json payload? Right now i’m getting this error when calling res.json() Exception: FetchError: invalid json response body at https://api_that_returns_json_with_bom reason: Unexpected token ? in JSON at position 0

I tried adding an Accept header of application/json; charset=utf-8 but that didn’t resolve the issue.

[github-actions[bot]]

🎉 This issue has been resolved in version 3.2.3 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀

[jimmywarting]

Hmm, chrome seems to be able to decode BOM

await new Response(String.fromCharCode(0xFEFF) + '1').json() // 1

What we are essentially doing is:

const buffer = Buffer.from(String.fromCharCode(0xFEFF) + '1'
JSON.parse(buffer.toString()) // Error

We should stop using buffer and turn towards using TextDecoder instead, this removes the BOM

JSON.parse(new TextDecoder().decode((Buffer.from(String.fromCharCode(0xFEFF) + '1')))) // 1

While we are at it:

async text() {
  let text = ''
  const decoder = new TextDecoder()
  for await (const uint8 of this.body) {
    text += decoder.decode(uint8, { stream: true })
  }
  text += decoder.decode() // flush
  return text
}

json() {
  return this.text().then(JSON.parse)
}