question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Recursive operations always fail

See original GitHub issue

What happened:

  • If you call copy(path1, path2, recursive=True) it will always fail on copying the root of your copy command (it tries to copy a “prefix” which will fail with FileNotFoundError
  • If you call rm(path, recursive=True) it will fail with “backend error”, which I think is caused by a concurrency which is to high, if I increase the batchsize, this will go away. I also noticed the cleanup of directories doesn’t take into account the batchsize currently. The test directory contains about 1200 files, which will cause 60 requests concurrently.

Environment:

  • Python version: 3.7
  • gcsfs: 0.7.1
  • fsspec: 0.8.4

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:6 (6 by maintainers)

github_iconTop GitHub Comments

1reaction
martindurantcommented, Nov 19, 2020

For copy, there would only be “raise” or “ignore” options - and it would make sense for recursive copy to ignore by default, if the exception is not-found.

0reactions
martindurantcommented, Jan 18, 2021

This is probably fixed with fsspec master

Read more comments on GitHub >

github_iconTop Results From Across the Web

What is Recursion? A Recursive Function Explained with ...
A recursive function must always have at least one base case to make it stop calling itself or it will cause an error....
Read more >
Mastering recursive programming - IBM Developer
For new computer science students, the concept of recursive programming is often difficult. Recursive thinking is difficult because it almost ...
Read more >
Reading 14: Recursion - MIT
A recursive implementation always has two parts: base case, which is the simplest, smallest instance of the problem, that can't be decomposed any...
Read more >
Recursive return error, Java - Stack Overflow
The if branch always does, but the else branch doesn't. What happens if this.value equals "Invalid", or something else which isn't in your ......
Read more >
Recursive CTE call in FromSqlRaw always fails with ... - GitHub
The call always ends in an InvalidOperationException stating that is was called with non-composable SQL and with a query composing over it.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found