req.body format and webhook signature verification

SDK you’re using (please complete the following information):

• Version 4.17.0

Describe the bug As per the various tutorials by Xero about how to catch webhook calls using Express – e.g. https://devblog.xero.com/keeping-your-integration-in-sync-implementing-xero-webhooks-using-node-express-and-ngrok-6d2976baac6d and https://devblog.xero.com/using-xero-webhooks-with-node-express-hapi-examples-7c607b423379 – I think I might have found a “bug” with the way that the x-xero-signature header is generated, or at least why there’s lots of posts on Stack Overflow etc as to why verification isn’t working.

If you use the examples as-is everything works, because req.body.toString() when output via console.log looks something like:

{"events":[],"firstEventSequence": 0,"lastEventSequence": 0, "entropy": "MIXBSNRWXBPLZQMKKQRS"}

The problem is though that this might have spaces it shouldn’t have, i.e. if you parse this and then stringify it, like so:

JSON.stringify(JSON.parse('{"events":[],"firstEventSequence": 0,"lastEventSequence": 0, "entropy": "MIXBSNRWXBPLZQMKKQRS"}')

then you get:

{"events":[],"firstEventSequence":0,"lastEventSequence":0,"entropy":"MIXBSNRWXBPLZQMKKQRS"}

Note the missing spaces before the number and string values that were present in the original req.body.toString() version.

This causes the equality comparison of Xero’s signature and the one generated by the Node crypto.createHmac method from the tutorials to not match and thus the received webhook data to fail verification.

I found that this can also be a problem if using middleware like express.json() or if you’re using a host that manipulates the req.body (like it seems Firebase Cloud Functions does).

My suggested solution would be to make it so that the x-xero-signature does not contain these spaces so that users can use JSON.parse/stringify to get the correct strings for verification.

[Youdaman]

For anyone else using Firebase (which changes req.body so that webhook validation will fail) you can use req.rawBody:

  const computedSignature = crypto.createHmac('sha256', xeroConfig.webhooksKey).update(req.rawBody).digest('base64')

See docs here https://cloud.google.com/functions/docs/writing/http#handling_content_types and thanks to @nevostruev and @enu-kuro via SO https://stackoverflow.com/a/50338756/67675

[Youdaman]

I also just noticed that it’s not just the numbers and strings that have spaces before them, but the only key that has a space before it is “entropy” – there’s no space after the commas for the others.

I just managed to get a successful verification by doing the following (on Firebase Cloud Hosting):

  const data = JSON.stringify(req.body).split(':').join(': ').split(': [').join(':[').split(',"entropy"').join(', "entropy"')

and then using this data in the signature generation as per the tutorials:

  const computedSignature = crypto.createHmac('sha256', xeroConfig.webhooksKey).update(data).digest('base64')

And one of the requests that came through gave me a match with x-xero-signature and successful webhook validation.

This could probably be done better with regex instead of split/join but I just wanted to see if I could generate a match.

So yeah I reckon you’d get rid of most if not all of the complaints about validation not working if the x-xero-signature was generated from proper JSON of the payload, i.e. no spaces before values or commas, just like JSON.stringify produces.