question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[QUESTION] Make archive faster after changing file archive list structure

See original GitHub issue

Checklist

Question

So i changed file archive list structure from lines in '_make_archive_id' function

from return f'{extractor} {video_id}'

to return f'{extractor.lower()} {uploader} {title} ({video_id})'

it makes the archiving changed

from iwara 4jqbmudrwcwmoqwl

to Iwara 000MMD クソガキでCHUNG HA - Dream of You (18kqetlxjt3w8wd3)

Now instead of fast loading archive like this

F:\downloader\iwara.tv\dl>ytdl-patched -a 000MMD_links.txt --conf config_iwara
[Iwara] 03yedh9lbizq1kq8: has already been recorded in the archive
[Iwara] 18kqetlxjt3w8wd3: has already been recorded in the archive
[Iwara] 25xbni92aixod0wo: has already been recorded in the archive
[Iwara] 4jqbmudrwcwmoqwl: has already been recorded in the archive
[iwara] 6a8als43auo2aezk: Downloading webpage

I get a slow loading archive like this

F:\downloader\iwara.tv\dl>ytdl-patched` -a 000MMD_links.txt --conf config_iwara
[iwara] 03yedh9lbizq1kq8: Downloading webpage
[iwara] 03yedh9lbizq1kq8: Downloading JSON metadata
WARNING: [iwara] unable to extract upload_date; please report this issue on  https://github.com/ytdl-patched/ytdl-patched/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
[download] ミニスカポリスなミクたちでSOLAR - Spit it out has already been recorded in the archive
[iwara] 18kqetlxjt3w8wd3: Downloading webpage
[iwara] 18kqetlxjt3w8wd3: Downloading JSON metadata
WARNING: [iwara] unable to extract upload_date; please report this issue on  https://github.com/ytdl-patched/ytdl-patched/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
[download] クソガキでCHUNG HA - Dream of You has already been recorded in the archive

The question is What other lines i need to change to make it load archive faster like it used to be?

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:7 (1 by maintainers)

github_iconTop GitHub Comments

2reactions
pukkandancommented, May 25, 2022

You are not supposed to change the archive format! If you want a file with other metadata, use --print-to-file

As for why it slow, it is because yt-dlp cannot determine title and uploader without extracting the video information. When it is just the id, we can determine it from the URL without making any network requests (not for all sites, but works here) and so the entire extraction process is skipped

2reactions
gamer191commented, May 25, 2022

Please do not report issues or ask questions related to ytdl-patched in the yt-dlp repo, they should be reported here

Read more comments on GitHub >

github_iconTop Results From Across the Web

Moving one large archive, or many individual files and folders?
1 Answer 1 ... If contents and size are equal, then transferring one file should be faster due to only having one header/index...
Read more >
Archive infostructure customizing when active - SAP Community
When we try to read from archive through transaction SARI we can see the list, but it is not possible to open object...
Read more >
Find Files Faster: How to Organize Files and Folders - Zapier
Then save every new file according to that folder structure. Consider archiving all old files by moving them into an "Old Files" or...
Read more >
Compressing directory using shutil.make_archive() while ...
cd into dir1 and run shutil.make_archive(base_name=base_name, format=format) ; it will create an archive dir1_arc.zip inside dir1 ; the only ...
Read more >
Copy the current Mailbox folder structure to a new pst-file
The trick to copy the folder structure of your current mailbox is to archive the mailbox with a date so far in the...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found