question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Can't write wget snapshots of URL with query-string

See original GitHub issue

Describe the bug

Most of Linux filesystems allow to use ? and other special characters as part of folder and filenames. But if you run this using ExFAT as the storage, it will throw the following error:

Got wget response code 3:
Cannot write to ‘www.youtube.com/watch?v=BkW1xQgrSPQ.html’ (No such file or directory).

ExFAT are common filesystems for portable USB drives, specially if there is the intention to use it with Linux, Windows and MacOS. It is not the best archival filesystem (eg: zfs), but it is the most portable filesystem.

I’m not sure how to do that yet, but it would be nice to convert special characters into safer ones when writing it down to the disk.

Steps to reproduce

  1. Create a ExFAT partition, cd into it
  2. Execute archivebox using a YouTube URL: echo 'www.youtube.com/watch?v=BkW1xQgrSPQ' | ./archive

Software versions

  • OS: Debian
  • ArchiveBox version: docker image: 296aa767078f

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:5 (5 by maintainers)

github_iconTop GitHub Comments

0reactions
bltavarescommented, Apr 12, 2019

I’ve noticed that the Dockerfile clones the latest commit on Github, instead of copying the project with their changes into the image. This could cause some confusion on why a change is not being built, as it is being cached by Docker automated build layers, and given the text don’t change it will not update the image.

I’ll send a PR soon (unless its intentional) 😃

Read more comments on GitHub >

github_iconTop Results From Across the Web

Can't write wget snapshots of URL with query-string · Issue #210
Got wget response code 3: Cannot write to 'www.youtube.com/watch?v=BkW1xQgrSPQ.html' (No such file or directory). ExFAT are common filesystems ...
Read more >
Why does wget ignore the query string in the URL?
No matter what comes after page= , it always downloads the first page of the listing. Do I have to escape some characters...
Read more >
WGET and escaping URL with query string - SitePoint
I meant to share my solution to the problem. WGet has a switch that allows you to read URLs from a file as...
Read more >
Using Query Parameters (AWS Signature Version 4)
Using query parameters to authenticate requests is useful when you want to express a request entirely in a URL. This method is also...
Read more >
Downloading Multiple Records Using Query Strings
In this lesson, we will write a program that will download a series of records from the Old Bailey Online using custom search...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found