question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Support non-UTF8 location

See original GitHub issue

When investigating https://github.com/axios/axios/issues/1804, I found that follow-redirects didn’t handle non-UTF8 location. Code example,

const axios = require('axios')

;(async () => {
  const url = "http://www.smun.spb.ru"
  const head_response = await axios.head(url)

  console.log(head_response.headers)
})()
.then(() => { console.log("Success")})
.catch(() => { console.log("Error")})

The solution can be inspired by https://github.com/sindresorhus/got/pull/214, and the original discussion was in https://github.com/nodejs/node/issues/7928. With following patches, it can be fixed. If confirmed and without regression problems, I can provide a PR.

    // Perform the redirected request
-   var redirectUrl = url.resolve(this._currentUrl, location);
+   var redirectUrl = Buffer.from(location, 'binary').toString();
+   redirectUrl = (new url.URL(redirectUrl, this._currentUrl)).toString();
    debug("redirecting to", redirectUrl);

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:5

github_iconTop GitHub Comments

1reaction
RubenVerborghcommented, Mar 16, 2020

@dinnouti We have that now in the latest version thanks to 137c8cf467c52c0583502bb611aab7e65704a929.

0reactions
dinnouticommented, Mar 16, 2020

The ability to handle the error would be great. I am processing 10k urls and just crashes without giving me a chance to investigate and clean the faulty URL from my list of URLs.

Read more comments on GitHub >

github_iconTop Results From Across the Web

How to Remove Non UTF-8 Characters From a File - Baeldung
We can break down the command above to find out what each part is doing: -f: Represents the original file format.
Read more >
Support Non-UTF-8 terminal · Issue #384 · stan-dev/cmdstanpy
Description: The terminal encoding is normally sys.stdin.encoding. But CmdnStanPy hardcords 'utf-8'. Here is the workaround. This workaround use ...
Read more >
Unable to support non-UTF-8 characters in XML file generated ...
Where AbsoluteFilesPath is an absolute reference to a location on the server. The method I'm using to change the encoding of the XML...
Read more >
Quick Tip: Replacing non-UTF-8 Characters - Vertica
Vertica 10.1.x introduces the MAKEUTF8 built-in function that coerces a string to UTF-8 by removing or replacing non-UTF-8 characters.
Read more >
Find non-UTF-8 data in mysql
You can check for the existence of (non-)UTF-8 data by comparing byte length to character length on a column, e.g.:
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found