Can't write wget snapshots of URL with query-string
See original GitHub issueDescribe the bug
Most of Linux filesystems allow to use ?
and other special characters as part of folder and filenames. But if you run this using ExFAT as the storage, it will throw the following error:
Got wget response code 3:
Cannot write to ‘www.youtube.com/watch?v=BkW1xQgrSPQ.html’ (No such file or directory).
ExFAT are common filesystems for portable USB drives, specially if there is the intention to use it with Linux, Windows and MacOS. It is not the best archival filesystem (eg: zfs), but it is the most portable filesystem.
I’m not sure how to do that yet, but it would be nice to convert special characters into safer ones when writing it down to the disk.
Steps to reproduce
- Create a ExFAT partition,
cd
into it - Execute archivebox using a YouTube URL:
echo 'www.youtube.com/watch?v=BkW1xQgrSPQ' | ./archive
Software versions
- OS: Debian
- ArchiveBox version: docker image:
296aa767078f
Issue Analytics
- State:
- Created 4 years ago
- Comments:5 (5 by maintainers)
Top Results From Across the Web
Can't write wget snapshots of URL with query-string · Issue #210
Got wget response code 3: Cannot write to 'www.youtube.com/watch?v=BkW1xQgrSPQ.html' (No such file or directory). ExFAT are common filesystems ...
Read more >Why does wget ignore the query string in the URL?
No matter what comes after page= , it always downloads the first page of the listing. Do I have to escape some characters...
Read more >WGET and escaping URL with query string - SitePoint
I meant to share my solution to the problem. WGet has a switch that allows you to read URLs from a file as...
Read more >Using Query Parameters (AWS Signature Version 4)
Using query parameters to authenticate requests is useful when you want to express a request entirely in a URL. This method is also...
Read more >Downloading Multiple Records Using Query Strings
In this lesson, we will write a program that will download a series of records from the Old Bailey Online using custom search...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
Done https://github.com/pirate/ArchiveBox/commit/4f599c0b0b07c842b1a2d0ec31f229d8fa0d6294
I’ve noticed that the Dockerfile clones the latest commit on Github, instead of copying the project with their changes into the image. This could cause some confusion on why a change is not being built, as it is being cached by Docker automated build layers, and given the text don’t change it will not update the image.
I’ll send a PR soon (unless its intentional) 😃