Stuck on an issue?
Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
ValueError: too many file descriptors in select()

See original GitHub issue
Windows with at least DVC>=2.6.0 (perhaps earlier versions) and azure remote I have large number of directories (say 200) each with say 10 files in, the directories are tracked directly with DVC. I’m trying to run dvc pull to download these and I get ValueError: too many file descriptors in select() and CMD hangs forever.
Another related but non-critical issue is that DVC seems to query the remote separately for every *.dir file it needs to download and download them individually which is really slow - can’t the remote be queried once and all ~200 dir files be downloaded at once?
>dvc pull -v
2021-09-15 10:54:54,190 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 10:54:54,192 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 10:54:54,193 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 10:54:54,197 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 10:54:54,198 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 10:54:54,297 DEBUG: Querying 230 hashes via object_exists
2021-09-15 10:55:36,775 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:55:36,863 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:55:37,225 DEBUG: Downloading 'azure://my-remote/db/b51231e80a0dbd0f1fe5786242c766.dir' to '.dvc\cache\db\b51231e80a0dbd0f1fe5786242c766.dir'
2021-09-15 10:55:37,376 DEBUG: state save (2912436261517855907, 1631699737330944256, 603) dbb51231e80a0dbd0f1fe5786242c766.dir
2021-09-15 10:55:37,454 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 10:55:37,457 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 10:55:37,460 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 10:55:37,465 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 10:55:37,466 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 10:55:37,570 DEBUG: Querying 230 hashes via object_exists
2021-09-15 10:55:39,880 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:55:39,944 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:55:40,349 DEBUG: Downloading 'azure://my-remote/39/1169727bee5ec91ef388cd0004917c.dir' to '.dvc\cache\39\1169727bee5ec91ef388cd0004917c.dir'
2021-09-15 10:55:40,484 DEBUG: state save (1881485164717332997, 1631699740477027584, 603) 391169727bee5ec91ef388cd0004917c.dir
2021-09-15 10:55:40,578 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 10:55:40,580 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 10:55:40,582 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 10:55:40,588 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 10:55:40,589 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 10:55:40,717 DEBUG: Querying 230 hashes via object_exists
2021-09-15 10:55:42,733 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:55:42,804 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:55:43,293 DEBUG: Downloading 'azure://my-remote/85/1c6738d144c116f8bc9dc8595a3a5a.dir' to '.dvc\cache\85\1c6738d144c116f8bc9dc8595a3a5a.dir'
2021-09-15 10:55:43,414 DEBUG: state save (2664473120733366070, 1631699743407263488, 603) 851c6738d144c116f8bc9dc8595a3a5a.dir
2021-09-15 10:55:43,522 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 10:55:43,525 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 10:55:43,527 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 10:55:43,535 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 10:55:43,537 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 10:55:43,643 DEBUG: Querying 230 hashes via object_exists
2021-09-15 10:55:47,130 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:55:47,194 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:55:47,722 DEBUG: Downloading 'azure://my-remote/d9/1dc90286ad34727eda0faffb9c0529.dir' to '.dvc\cache\d9\1dc90286ad34727eda0faffb9c0529.dir'
2021-09-15 10:55:47,827 DEBUG: state save (2664473120739191315, 1631699747821704704, 603) d91dc90286ad34727eda0faffb9c0529.dir
2021-09-15 10:55:47,894 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 10:55:47,896 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 10:55:47,898 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 10:55:47,902 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 10:55:47,903 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 10:55:47,998 DEBUG: Querying 230 hashes via object_exists
2021-09-15 10:55:51,779 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:55:51,853 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:55:52,267 DEBUG: Downloading 'azure://my-remote/f1/32b5218127af55aec353761c252dee.dir' to '.dvc\cache\f1\32b5218127af55aec353761c252dee.dir'
2021-09-15 10:55:52,376 DEBUG: state save (7103619552507012856, 1631699752370092288, 603) f132b5218127af55aec353761c252dee.dir
2021-09-15 10:55:52,467 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 10:55:52,469 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 10:55:52,471 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 10:55:52,476 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 10:55:52,477 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 10:55:52,595 DEBUG: Querying 230 hashes via object_exists
2021-09-15 10:55:54,545 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:55:54,624 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:55:55,109 DEBUG: Downloading 'azure://my-remote/05/6d7e6c19902301fd3891c864b94ee0.dir' to '.dvc\cache\05\6d7e6c19902301fd3891c864b94ee0.dir'
2021-09-15 10:55:55,229 DEBUG: state save (3904281985119155855, 1631699755222837248, 603) 056d7e6c19902301fd3891c864b94ee0.dir
2021-09-15 10:55:55,301 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 10:55:55,303 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 10:55:55,305 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 10:55:55,310 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 10:55:55,311 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 10:55:55,415 DEBUG: Querying 230 hashes via object_exists
2021-09-15 10:55:57,894 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:55:57,969 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:55:58,476 DEBUG: Downloading 'azure://my-remote/99/461c4482257e0a2da7420844a28f92.dir' to '.dvc\cache\99\461c4482257e0a2da7420844a28f92.dir'
2021-09-15 10:55:58,594 DEBUG: state save (3121291592579061681, 1631699758587884544, 603) 99461c4482257e0a2da7420844a28f92.dir
2021-09-15 10:55:58,693 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 10:55:58,695 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 10:55:58,698 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 10:55:58,702 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 10:55:58,704 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 10:55:58,825 DEBUG: Querying 230 hashes via object_exists
2021-09-15 10:56:00,662 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:56:00,731 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:56:01,263 DEBUG: Downloading 'azure://my-remote/a7/051ea171af7508476638e2f0e4f588.dir' to '.dvc\cache\a7\051ea171af7508476638e2f0e4f588.dir'
2021-09-15 10:56:01,395 DEBUG: state save (6053281630900451211, 1631699761387491584, 603) a7051ea171af7508476638e2f0e4f588.dir
2021-09-15 10:56:01,495 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 10:56:01,497 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 10:56:01,500 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 10:56:01,506 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 10:56:01,507 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 10:56:01,639 DEBUG: Querying 230 hashes via object_exists
2021-09-15 10:56:04,192 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:56:04,268 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:56:04,829 DEBUG: Downloading 'azure://my-remote/11/5a245027722b301a2056ecc9849b4b.dir' to '.dvc\cache\11\5a245027722b301a2056ecc9849b4b.dir'
2021-09-15 10:56:05,001 DEBUG: state save (2333858505471818690, 1631699764994374400, 603) 115a245027722b301a2056ecc9849b4b.dir
2021-09-15 10:56:05,107 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 10:56:05,110 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 10:56:05,112 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 10:56:05,118 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 10:56:05,120 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 10:56:05,231 DEBUG: Querying 230 hashes via object_exists
2021-09-15 10:56:07,172 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:56:07,250 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:56:07,759 DEBUG: Downloading 'azure://my-remote/78/eaf8fff631709ae58eb11aeacf0fe1.dir' to '.dvc\cache\78\eaf8fff631709ae58eb11aeacf0fe1.dir'
2021-09-15 10:56:07,876 DEBUG: state save (2333858505498614817, 1631699767869417216, 603) 78eaf8fff631709ae58eb11aeacf0fe1.dir
2021-09-15 10:56:07,985 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 10:56:07,986 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 10:56:07,989 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 10:56:07,995 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 10:56:07,996 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 10:56:08,098 DEBUG: Querying 230 hashes via object_exists
2021-09-15 10:56:10,331 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:56:10,396 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:56:10,824 DEBUG: Downloading 'azure://my-remote/a8/85916d3638d54407b8e61b2267daff.dir' to '.dvc\cache\a8\85916d3638d54407b8e61b2267daff.dir'
2021-09-15 10:56:10,963 DEBUG: state save (1672630496652891751, 1631699770957171456, 603) a885916d3638d54407b8e61b2267daff.dir
2021-09-15 10:56:11,020 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 10:56:11,022 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 10:56:11,023 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 10:56:11,027 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 10:56:11,028 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 10:56:11,117 DEBUG: Querying 230 hashes via object_exists
2021-09-15 10:56:13,739 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:56:13,814 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:56:14,280 DEBUG: Downloading 'azure://my-remote/4f/d0d2a6ede96cf455b261ac5fb998e2.dir' to '.dvc\cache\4f\d0d2a6ede96cf455b261ac5fb998e2.dir'
2021-09-15 10:56:14,402 DEBUG: state save (2581820230443326816, 1631699774395614208, 603) 4fd0d2a6ede96cf455b261ac5fb998e2.dir
2021-09-15 10:56:14,494 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 10:56:14,496 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 10:56:14,498 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 10:56:14,503 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 10:56:14,505 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 10:56:14,620 DEBUG: Querying 230 hashes via object_exists
2021-09-15 10:56:18,920 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:56:18,995 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:56:19,469 DEBUG: Downloading 'azure://my-remote/d7/309d4d492f82ae93cb89f5c6f81990.dir' to '.dvc\cache\d7\309d4d492f82ae93cb89f5c6f81990.dir'
2021-09-15 10:56:19,597 DEBUG: state save (2333858505485799278, 1631699779590052608, 603) d7309d4d492f82ae93cb89f5c6f81990.dir
2021-09-15 10:56:19,703 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 10:56:19,705 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 10:56:19,707 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 10:56:19,714 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 10:56:19,715 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 10:56:19,818 DEBUG: Querying 230 hashes via object_exists
2021-09-15 10:56:22,260 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:56:22,332 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:56:22,808 DEBUG: Downloading 'azure://my-remote/03/f6ceb80a8279c2cbb8c15de4ac113d.dir' to '.dvc\cache\03\f6ceb80a8279c2cbb8c15de4ac113d.dir'
2021-09-15 10:56:22,917 DEBUG: state save (1755284303052416028, 1631699782911220992, 603) 03f6ceb80a8279c2cbb8c15de4ac113d.dir
2021-09-15 10:56:23,024 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 10:56:23,027 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 10:56:23,030 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 10:56:23,036 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 10:56:23,037 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 10:56:23,141 DEBUG: Querying 230 hashes via object_exists
2021-09-15 10:56:25,200 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:56:25,273 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:56:25,739 DEBUG: Downloading 'azure://my-remote/54/e4604be059efcb5efddaaec038c1d3.dir' to '.dvc\cache\54\e4604be059efcb5efddaaec038c1d3.dir'
2021-09-15 10:56:25,854 DEBUG: state save (846091016842337172, 1631699785848288256, 603) 54e4604be059efcb5efddaaec038c1d3.dir
2021-09-15 10:56:25,958 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 10:56:25,961 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 10:56:25,963 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 10:56:25,969 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 10:56:25,970 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 10:56:26,096 DEBUG: Querying 230 hashes via object_exists
2021-09-15 10:56:28,178 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:56:28,255 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:56:28,721 DEBUG: Downloading 'azure://my-remote/d3/6c623c5a9636efdfd0bf9ed55223a9.dir' to '.dvc\cache\d3\6c623c5a9636efdfd0bf9ed55223a9.dir'
2021-09-15 10:56:28,841 DEBUG: state save (2581820230444491865, 1631699788833970688, 603) d36c623c5a9636efdfd0bf9ed55223a9.dir
2021-09-15 10:56:28,943 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 10:56:28,945 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 10:56:28,947 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 10:56:28,954 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 10:56:28,955 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 10:56:29,059 DEBUG: Querying 230 hashes via object_exists
2021-09-15 10:56:30,883 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:56:30,961 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:56:31,476 DEBUG: Downloading 'azure://my-remote/33/7f31e963911b2e28b50fa814510228.dir' to '.dvc\cache\33\7f31e963911b2e28b50fa814510228.dir'
2021-09-15 10:56:31,603 DEBUG: state save (2581820230325656867, 1631699791597101568, 603) 337f31e963911b2e28b50fa814510228.dir
2021-09-15 10:56:31,703 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 10:56:31,706 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 10:56:31,708 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 10:56:31,715 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 10:56:31,716 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 10:56:31,824 DEBUG: Querying 230 hashes via object_exists
2021-09-15 10:57:13,260 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:57:13,338 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:57:13,700 DEBUG: Downloading 'azure://my-remote/ca/f884d145de4c7fb865b2a1da5a6d4c.dir' to '.dvc\cache\ca\f884d145de4c7fb865b2a1da5a6d4c.dir'
2021-09-15 10:57:13,815 DEBUG: state save (2475006299367402635, 1631699833808065536, 603) caf884d145de4c7fb865b2a1da5a6d4c.dir
2021-09-15 10:57:13,917 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 10:57:13,919 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 10:57:13,920 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 10:57:13,925 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 10:57:13,928 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 10:57:14,012 DEBUG: Querying 230 hashes via object_exists
2021-09-15 10:58:35,016 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:58:35,100 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:58:35,451 DEBUG: Downloading 'azure://my-remote/60/473b8c82a39d9d54639b35b812312c.dir' to '.dvc\cache\60\473b8c82a39d9d54639b35b812312c.dir'
2021-09-15 10:58:35,570 DEBUG: state save (3986935207190684280, 1631699915563839488, 603) 60473b8c82a39d9d54639b35b812312c.dir
2021-09-15 10:58:35,659 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 10:58:35,661 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 10:58:35,665 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 10:58:35,670 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 10:58:35,671 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 10:58:35,757 DEBUG: Querying 230 hashes via object_exists
2021-09-15 10:59:16,493 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:59:16,568 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:59:16,995 DEBUG: Downloading 'azure://my-remote/ef/09bee90c2730c82ac9a8befe0ae8d9.dir' to '.dvc\cache\ef\09bee90c2730c82ac9a8befe0ae8d9.dir'
2021-09-15 10:59:17,117 DEBUG: state save (3077744340208349071, 1631699957110445056, 603) ef09bee90c2730c82ac9a8befe0ae8d9.dir
2021-09-15 10:59:17,224 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 10:59:17,226 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 10:59:17,228 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 10:59:17,234 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 10:59:17,235 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 10:59:17,324 DEBUG: Querying 230 hashes via object_exists
2021-09-15 10:59:19,167 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:59:19,239 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:59:19,665 DEBUG: Downloading 'azure://my-remote/6d/8237994f47b0484e6127651c1515d0.dir' to '.dvc\cache\6d\8237994f47b0484e6127651c1515d0.dir'
2021-09-15 10:59:19,787 DEBUG: state save (2995088957491702549, 1631699959780902144, 603) 6d8237994f47b0484e6127651c1515d0.dir
2021-09-15 10:59:19,891 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 10:59:19,893 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 10:59:19,896 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 10:59:19,901 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 10:59:19,902 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 10:59:19,988 DEBUG: Querying 230 hashes via object_exists
2021-09-15 10:59:22,084 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:59:22,156 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:59:22,701 DEBUG: Downloading 'azure://my-remote/64/4e2ec0cf7d1e65f9220b6a196d2fb8.dir' to '.dvc\cache\64\4e2ec0cf7d1e65f9220b6a196d2fb8.dir'
2021-09-15 10:59:22,815 DEBUG: state save (4813473565748760867, 1631699962806881280, 603) 644e2ec0cf7d1e65f9220b6a196d2fb8.dir
2021-09-15 10:59:22,899 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 10:59:22,900 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 10:59:22,902 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 10:59:22,905 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 10:59:22,907 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 10:59:23,025 DEBUG: Querying 230 hashes via object_exists
2021-09-15 10:59:25,958 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:59:26,016 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:59:26,472 DEBUG: Downloading 'azure://my-remote/c5/9f75077d7acf285d29aed1b0f9334d.dir' to '.dvc\cache\c5\9f75077d7acf285d29aed1b0f9334d.dir'
2021-09-15 10:59:26,593 DEBUG: state save (1755284303036105342, 1631699966583963648, 603) c59f75077d7acf285d29aed1b0f9334d.dir
2021-09-15 10:59:26,683 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 10:59:26,685 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 10:59:26,687 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 10:59:26,693 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 10:59:26,694 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 10:59:26,802 DEBUG: Querying 230 hashes via object_exists
2021-09-15 10:59:29,461 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:59:29,534 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:59:30,068 DEBUG: Downloading 'azure://my-remote/65/bc354e70d30f4866bfc8a0d6750dab.dir' to '.dvc\cache\65\bc354e70d30f4866bfc8a0d6750dab.dir'
2021-09-15 10:59:30,187 DEBUG: state save (2581820230344297651, 1631699970180367616, 603) 65bc354e70d30f4866bfc8a0d6750dab.dir
2021-09-15 10:59:30,290 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 10:59:30,293 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 10:59:30,296 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 10:59:30,305 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 10:59:30,306 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 10:59:30,403 DEBUG: Querying 230 hashes via object_exists
2021-09-15 10:59:33,466 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:59:33,537 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:59:34,030 DEBUG: Downloading 'azure://my-remote/d2/7738c6447163c6bd0a13ed81b0e386.dir' to '.dvc\cache\d2\7738c6447163c6bd0a13ed81b0e386.dir'
2021-09-15 10:59:34,515 DEBUG: state save (2581820230340802504, 1631699974506218240, 603) d27738c6447163c6bd0a13ed81b0e386.dir
2021-09-15 10:59:34,611 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 10:59:34,613 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 10:59:34,616 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 10:59:34,624 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 10:59:34,626 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 10:59:34,755 DEBUG: Querying 230 hashes via object_exists
2021-09-15 10:59:37,321 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:59:37,393 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:59:37,827 DEBUG: Downloading 'azure://my-remote/b1/213bc6b64940a26a96e8d5df9dbf18.dir' to '.dvc\cache\b1\213bc6b64940a26a96e8d5df9dbf18.dir'
2021-09-15 10:59:37,947 DEBUG: state save (5367889193221302641, 1631699977939902976, 603) b1213bc6b64940a26a96e8d5df9dbf18.dir
2021-09-15 10:59:38,052 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 10:59:38,055 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 10:59:38,057 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 10:59:38,063 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 10:59:38,064 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 10:59:38,166 DEBUG: Querying 230 hashes via object_exists
2021-09-15 10:59:40,251 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:59:40,325 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:59:40,707 DEBUG: Downloading 'azure://my-remote/f5/9ee58b42029535211ca7758fc02681.dir' to '.dvc\cache\f5\9ee58b42029535211ca7758fc02681.dir'
2021-09-15 10:59:40,826 DEBUG: state save (2747126927141045690, 1631699980819972096, 603) f59ee58b42029535211ca7758fc02681.dir
2021-09-15 10:59:40,920 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 10:59:40,923 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 10:59:40,925 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 10:59:40,931 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 10:59:40,933 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 10:59:41,026 DEBUG: Querying 230 hashes via object_exists
2021-09-15 10:59:43,252 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:59:43,322 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:59:43,865 DEBUG: Downloading 'azure://my-remote/43/92b922ae627318c210d47af8492dd7.dir' to '.dvc\cache\43\92b922ae627318c210d47af8492dd7.dir'
2021-09-15 10:59:43,985 DEBUG: state save (3243050682512142733, 1631699983978857472, 603) 4392b922ae627318c210d47af8492dd7.dir
2021-09-15 10:59:44,083 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 10:59:44,085 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 10:59:44,088 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 10:59:44,094 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 10:59:44,098 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 10:59:44,182 DEBUG: Querying 230 hashes via object_exists
2021-09-15 10:59:46,380 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:59:46,457 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:59:46,943 DEBUG: Downloading 'azure://my-remote/b4/a498637ff04e3a9c60f9cc416b014f.dir' to '.dvc\cache\b4\a498637ff04e3a9c60f9cc416b014f.dir'
2021-09-15 10:59:47,059 DEBUG: state save (2829781039059524580, 1631699987053569792, 603) b4a498637ff04e3a9c60f9cc416b014f.dir
2021-09-15 10:59:47,142 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 10:59:47,144 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 10:59:47,145 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 10:59:47,150 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 10:59:47,151 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 10:59:47,241 DEBUG: Querying 230 hashes via object_exists
2021-09-15 10:59:49,403 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:59:49,478 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:59:49,949 DEBUG: Downloading 'azure://my-remote/40/5c9cd1e6983b64fbd9092fefde873b.dir' to '.dvc\cache\40\5c9cd1e6983b64fbd9092fefde873b.dir'
2021-09-15 10:59:50,073 DEBUG: state save (2747126927144540837, 1631699990065082368, 603) 405c9cd1e6983b64fbd9092fefde873b.dir
2021-09-15 10:59:50,179 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 10:59:50,181 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 10:59:50,183 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 10:59:50,190 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 10:59:50,191 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 10:59:50,278 DEBUG: Querying 230 hashes via object_exists
2021-09-15 10:59:52,754 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:59:52,834 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:59:53,245 DEBUG: Downloading 'azure://my-remote/fe/7576f34edf661a64fdec39859d73e1.dir' to '.dvc\cache\fe\7576f34edf661a64fdec39859d73e1.dir'
2021-09-15 10:59:53,362 DEBUG: state save (2829781039057194482, 1631699993355135232, 603) fe7576f34edf661a64fdec39859d73e1.dir
2021-09-15 10:59:53,457 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 10:59:53,460 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 10:59:53,462 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 10:59:53,467 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 10:59:53,469 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 10:59:53,553 DEBUG: Querying 230 hashes via object_exists
2021-09-15 10:59:55,661 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:59:55,733 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:59:56,369 DEBUG: Downloading 'azure://my-remote/9a/5f58a954a2f4f3c6c6a92e836fcc11.dir' to '.dvc\cache\9a\5f58a954a2f4f3c6c6a92e836fcc11.dir'
2021-09-15 10:59:56,485 DEBUG: state save (2581820230334977259, 1631699996478923520, 603) 9a5f58a954a2f4f3c6c6a92e836fcc11.dir
2021-09-15 10:59:56,595 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 10:59:56,597 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 10:59:56,600 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 10:59:56,606 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 10:59:56,607 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 10:59:56,691 DEBUG: Querying 230 hashes via object_exists
2021-09-15 10:59:58,854 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:59:58,916 DEBUG: Querying 1 hashes via object_exists
2021-09-15 10:59:59,483 DEBUG: Downloading 'azure://my-remote/ae/3f4fa3bced5b5e1c546c53400d6ef2.dir' to '.dvc\cache\ae\3f4fa3bced5b5e1c546c53400d6ef2.dir'
2021-09-15 10:59:59,605 DEBUG: state save (2829781039060689629, 1631699999597140480, 603) ae3f4fa3bced5b5e1c546c53400d6ef2.dir
2021-09-15 10:59:59,688 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 10:59:59,690 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 10:59:59,693 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 10:59:59,698 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 10:59:59,700 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 10:59:59,786 DEBUG: Querying 230 hashes via object_exists
2021-09-15 11:00:02,419 DEBUG: Querying 1 hashes via object_exists
2021-09-15 11:00:02,496 DEBUG: Querying 1 hashes via object_exists
2021-09-15 11:00:02,984 DEBUG: Downloading 'azure://my-remote/6d/ca5a39e1b5a14e60474a0e5bdf72ca.dir' to '.dvc\cache\6d\ca5a39e1b5a14e60474a0e5bdf72ca.dir'
2021-09-15 11:00:03,101 DEBUG: state save (1755284303054746126, 1631700003096579840, 603) 6dca5a39e1b5a14e60474a0e5bdf72ca.dir
2021-09-15 11:00:03,186 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 11:00:03,188 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 11:00:03,190 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 11:00:03,195 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 11:00:03,198 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 11:00:03,284 DEBUG: Querying 230 hashes via object_exists
2021-09-15 11:00:05,364 DEBUG: Querying 1 hashes via object_exists
2021-09-15 11:00:05,431 DEBUG: Querying 1 hashes via object_exists
2021-09-15 11:00:05,903 DEBUG: Downloading 'azure://my-remote/cf/8b5dfe8dd0a460e6aa3d8701441620.dir' to '.dvc\cache\cf\8b5dfe8dd0a460e6aa3d8701441620.dir'
2021-09-15 11:00:06,031 DEBUG: state save (2664473120613366023, 1631700006024068352, 603) cf8b5dfe8dd0a460e6aa3d8701441620.dir
2021-09-15 11:00:06,131 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 11:00:06,134 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 11:00:06,136 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 11:00:06,142 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 11:00:06,143 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 11:00:06,236 DEBUG: Querying 230 hashes via object_exists
2021-09-15 11:00:08,548 DEBUG: Querying 1 hashes via object_exists
2021-09-15 11:00:08,620 DEBUG: Querying 1 hashes via object_exists
2021-09-15 11:00:09,131 DEBUG: Downloading 'azure://my-remote/1f/3380373b1942c8af2839219180a0fd.dir' to '.dvc\cache\1f\3380373b1942c8af2839219180a0fd.dir'
2021-09-15 11:00:09,252 DEBUG: state save (2581820230364103484, 1631700009246375680, 603) 1f3380373b1942c8af2839219180a0fd.dir
2021-09-15 11:00:09,345 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 11:00:09,347 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 11:00:09,350 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 11:00:09,355 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 11:00:09,356 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 11:00:09,452 DEBUG: Querying 230 hashes via object_exists
2021-09-15 11:00:12,460 DEBUG: Querying 1 hashes via object_exists
2021-09-15 11:00:12,555 DEBUG: Querying 1 hashes via object_exists
2021-09-15 11:00:13,201 DEBUG: Downloading 'azure://my-remote/22/c0f4bb8bee735ceecc32177c45344a.dir' to '.dvc\cache\22\c0f4bb8bee735ceecc32177c45344a.dir'
2021-09-15 11:00:13,327 DEBUG: state save (2747126927137550543, 1631700013320934144, 603) 22c0f4bb8bee735ceecc32177c45344a.dir
2021-09-15 11:00:13,411 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 11:00:13,413 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 11:00:13,415 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 11:00:13,419 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 11:00:13,421 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 11:00:13,516 DEBUG: Querying 230 hashes via object_exists
2021-09-15 11:00:19,441 DEBUG: Querying 1 hashes via object_exists
2021-09-15 11:00:19,512 DEBUG: Querying 1 hashes via object_exists
2021-09-15 11:00:19,970 DEBUG: Downloading 'azure://my-remote/2a/06bac5b388a8014131037d220f64df.dir' to '.dvc\cache\2a\06bac5b388a8014131037d220f64df.dir'
2021-09-15 11:00:20,103 DEBUG: state save (2581820230361773386, 1631700020096268544, 603) 2a06bac5b388a8014131037d220f64df.dir
2021-09-15 11:00:20,202 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 11:00:20,205 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 11:00:20,209 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 11:00:20,215 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 11:00:20,216 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 11:00:20,327 DEBUG: Querying 230 hashes via object_exists
2021-09-15 11:00:23,389 DEBUG: Querying 1 hashes via object_exists
2021-09-15 11:00:23,460 DEBUG: Querying 1 hashes via object_exists
2021-09-15 11:00:23,890 DEBUG: Downloading 'azure://my-remote/d5/45853d3814c7a8d8664d711114ac92.dir' to '.dvc\cache\d5\45853d3814c7a8d8664d711114ac92.dir'
2021-09-15 11:00:24,011 DEBUG: state save (2581820230362938435, 1631700024004060672, 603) d545853d3814c7a8d8664d711114ac92.dir
2021-09-15 11:00:24,088 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 11:00:24,090 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 11:00:24,092 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 11:00:24,096 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 11:00:24,097 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 11:00:24,211 DEBUG: Querying 230 hashes via object_exists
2021-09-15 11:00:28,645 DEBUG: Querying 1 hashes via object_exists
2021-09-15 11:00:28,723 DEBUG: Querying 1 hashes via object_exists
2021-09-15 11:00:29,248 DEBUG: Downloading 'azure://my-remote/c2/85769d49c3d064c25a5f9cb125e744.dir' to '.dvc\cache\c2\85769d49c3d064c25a5f9cb125e744.dir'
2021-09-15 11:00:29,362 DEBUG: state save (2829781039088650805, 1631700029355005952, 603) c285769d49c3d064c25a5f9cb125e744.dir
2021-09-15 11:00:29,454 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 11:00:29,457 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 11:00:29,460 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 11:00:29,464 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 11:00:29,465 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 11:00:29,595 DEBUG: Querying 230 hashes via object_exists
2021-09-15 11:00:32,232 DEBUG: Querying 1 hashes via object_exists
2021-09-15 11:00:32,306 DEBUG: Querying 1 hashes via object_exists
2021-09-15 11:00:32,830 DEBUG: Downloading 'azure://my-remote/eb/fc4729c7e7e13a2decb64103967e56.dir' to '.dvc\cache\eb\fc4729c7e7e13a2decb64103967e56.dir'
2021-09-15 11:00:32,971 DEBUG: state save (2581820230360608337, 1631700032963369472, 603) ebfc4729c7e7e13a2decb64103967e56.dir
2021-09-15 11:00:33,055 DEBUG: Preparing to transfer data from 'azure://my-remote' to '.dvc\cache'
2021-09-15 11:00:33,057 DEBUG: Preparing to collect status from '.dvc\cache'
2021-09-15 11:00:33,060 DEBUG: Collecting status from '.dvc\cache'
2021-09-15 11:00:33,065 DEBUG: Preparing to collect status from 'azure://my-remote'
2021-09-15 11:00:33,066 DEBUG: Collecting status from 'azure://my-remote'
2021-09-15 11:00:33,183 DEBUG: Querying 230 hashes via object_exists
  0% Querying remote cache|                                                                    |0/230 [00:00<?,     ?file/s]Exception in thread fsspecIO:
Traceback (most recent call last):
  File "...\lib\threading.py", line 926, in _bootstrap_inner
    self.run()
  File "...\lib\threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "...\lib\asyncio\base_events.py", line 541, in run_forever
    self._run_once()
  File "...\lib\asyncio\base_events.py", line 1750, in _run_once
    event_list = self._selector.select(timeout)
  File "...\lib\selectors.py", line 323, in select
    r, w, _ = self._select(self._readers, self._writers, [], timeout)
  File "...\lib\selectors.py", line 314, in _select
    r, w, x = select.select(r, w, w, timeout)
ValueError: too many file descriptors in select()
Issue Analytics

State:
Created 2 years ago
Comments:8 (8 by maintainers)
Top GitHub Comments

1reaction
efiopcommented, Jul 20, 2022
@friggog It really depends on particular filesystem. Right now we are working on not using default jobs number at all and relying on the underlying fsspec filesystem’s async functionality to throttle as needed (e.g. we use adlfs for azure). So please stay tuned for that.
1reaction
efiopcommented, Oct 20, 2021
@friggog THanks for the feedback! The fetching will be addressed in the future, we are working on pre-requisites right now.
Top Results From Across the Web

ValueError: too many file descriptors in select() on Windows
This is a limitation of underlying select() API call. To increase the limit please use ProactorEventLoop , you can use the code below....
In Python using asyncio and aiohttp, what does 'ValueError
In Python using asyncio and aiohttp, what does "ValueError: too many file descriptors in select()" mean on a Windows environment? All related (32)....
ValueError: too many file descriptors in select() #5873 - GitHub
Hi I have 3-5 instances of Electrum (3.3.8, BTC) with hundreds of transactions in each open for days and weeks on my Win10...
ValueError: too many file descriptors in select() Rasa Error
I'm testing rasa on windows local setup with 15000 concurrent users and it's getting this error and stoped. ValueError: too many file ......
HELP! too many file descriptors in select()!!!!
Next message: [medusa] Re: HELP! too many file descriptors in select()! ... select.select (r,w,e, timeout) > > ValueError: too many file ...