question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

"\u0000" in sql will lead to invalid message format error

See original GitHub issue
INSERT INTO "content"(
    "name"
)
VALUES
    (
        '"1234\u0000"',
    );

OK with command line. not OK with ng.

Issue Analytics

  • State:closed
  • Created 7 years ago
  • Comments:13 (8 by maintainers)

github_iconTop GitHub Comments

2reactions
vitaly-tcommented, Oct 31, 2018

@brianc , @charmander

Guys, I’m afraid this issue was closed prematurely. There is indeed a bug in the driver related specifically to Unicode symbol \u0000, which results in throwing error invalid message format.

This one works fine:

select E'\u0001' as value

And these ones throw that error:

select E'\u0000' as value

select '\0' as value

This needs to be re-opened, and escalated into a bug.

1reaction
sehropecommented, Oct 31, 2018

The error message difference is due to the lack of escaping the backslash in the JS code so the \u0000 is evaluated in the client, not by the server. This makes the query string itself sent to the backend contain an invalid 0-byte and thus gets rejected differently.

Here’s the two situations:

// Single backslash evaluated to a zero byte locally which is rejected at message level by server:
> client.query("SELECT E'\u0000'").catch((err) => console.error('%s', err)).then(console.log)
error: invalid message format

// Double backslash gets sent to the server correctly
> client.query("SELECT E'\\u0000'").catch((err) => console.error('%s', err)).then(console.log)
error: invalid Unicode escape value at or near "E'\u0000"

You can see it in the lengths of the strings too:

// Single backslash
> console.log('%s', "SELECT E'\u0000'".length);
11
// Double backslash
> console.log('%s', "SELECT E'\\u0000'".length);
16

This does indicate a bug but not directly related to these error messages. The driver shouldn’t be sending strings with zero bytes to the backend as that’s invalid and a potential security issue. I’m going to open a separate issue for rejecting those queries in the client as they violate the FEBE protocol.

Read more comments on GitHub >

github_iconTop Results From Across the Web

"\u0000" in sql will lead to invalid message format error
INSERT INTO "content"( "name" ) VALUES ( '"1234\u0000"', );. OK with command line. not OK with ng. See More.
Read more >
Handling Unicode sequences in postgresql - json
\u0000 is the one Unicode code point which is not valid in a string. I see no other way than to sanitize the...
Read more >
unsupported Unicode escape sequence, \u0000 cannot be ...
4), it would report the following error: ERROR: unsupported Unicode escape sequence DETAIL: \u0000 cannot be converted to text. CONTEXT: JSON ...
Read more >
Removing Null Characters (0x00) From Values in SQL Server
Casting to varbinary returns the character values in the column as hexadecimal. Once the values were converted to hexadecimal I could see the ......
Read more >
I am trying to load data in a table with variant column and I am ...
davies (Snowflake)​ Another thing that I can not use JSON format cause I have to load data in more than one column and...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found