Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

UTF8 decoding differs from Node

See original GitHub issue

Code

var salt = 'mnemonicメートルガバヴァぱばぐゞちぢ十人十色'
var buffer = new Buffer(salt, 'utf8')

console.log(buffer.toString('base64'))

Results

node v6.5.0

bW5lbW9uaWPjg6Hjg7zjg4jjg6vjgqvjgpnjg4/jgpnjgqbjgpnjgqHjga/jgprjga/jgpnjgY/jgpnjgp3jgpnjgaHjgaHjgpnljYHkurrljYHoibI=

Chromium 53.0.2785.101

bW5lbW9uaWPDo8aSwqHDo8aSwrzDo8aSy4bDo8aSwqvDo+KAmsKrw6PigJrihKLDo8aSwo/Do+KAmuKEosOj4oCawqbDo+KAmuKEosOj4oCawqHDo8KBwq/Do+KAmsWhw6PCgcKvw6PigJrihKLDo8KBwo/Do+KAmuKEosOj4oCawp3Do+KAmuKEosOjwoHCocOjwoHCocOj4oCa4oSiw6XCjcKBw6TCusK6w6XCjcKBw6jigLDCsg==

Issue Analytics

State:
Created 7 years ago
Comments:10 (6 by maintainers)

Top GitHub Comments

1reaction

jessetanecommented, Sep 14, 2016

Just to be precise about why this happened: the browser can only guess what encoding a source file is in unless you specify it via meta tag or http header. Of course everyone stores text files as utf-8 these days, but for backward compatibility the browser probably assumed iso-8859-1 and so OP just experienced this mismatch.

1reaction

ferosscommented, Sep 14, 2016

Sent a PR to resolve this issue: https://github.com/feross/buffer/pull/134

Top Results From Across the Web

How do I capture utf-8 decode errors in node.js?

8.23, current git: v0. 11.3-pre) ignores any decoding errors in its Buffer handling, silently replacing any non-utf8 characters with '\ufffd' ( ...

String decoder | Node.js v19.3.0 Documentation

The node:string_decoder module provides an API for decoding Buffer objects into strings in a manner that preserves encoded multi-byte UTF-8 and UTF-16 ...

UTF-8 decoding is order of magnitude slower than necessary ...

When Node's UTF-8 decoder encounters even a single non-ASCII character, its performance drops by an order of magnitude (from 0.7 ms to 3.3ms ......

Node util: 274% faster UTF8 decoding : r/programming - Reddit

Better way would be to say 274% more throughput in UTF-8 decoding. "Faster" can mean too many different things.

How to Base64 Encode/Decode a Value in Node.js

Base64 is a binary-to-text encoding scheme used to transport data. The encoding is necessary when the transfer medium is not able to handle ......