question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

datalakefileclient.upload_data without overwrite

See original GitHub issue

It seems that I cannot use upload_data without setting the overwrite flag to true. Maybe I am doing something wrong:

file1 = my_file_system_client.get_file_client("path/to/NONEXISTING_file.txt")
file1.upload_data(data) # Trying to upload to a non-existing file
# Raises: azure.core.exceptions.ResourceNotFoundError: (PathNotFound) The specified path does not exist.

file1 = my_file_system_client.get_file_client("path/to/NONEXISTING_file.txt")
file1.upload_data(data, overwrite=True) # Trying to upload to a non-existing file, with overwrite
# Works!

file1 = my_file_system_client.get_file_client("path/to/existing.txt")
file1.upload_data(data) # Trying to upload to a existing file, without overwrite
# Raises: azure.storage.filedatalake._generated.models._models_py3.StorageErrorException: Operation returned an invalid status 'The condition specified using HTTP conditional header(s) is not met.'

file1 = my_file_system_client.get_file_client("path/to/existing.txt")
file1.upload_data(data, overwrite=True) # Trying to upload to a existing file, with overwrite
# Works!
 

If this is the case for everyone, I am questioning the use of the overwrite flag… or at least its default value of False?

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Reactions:1
  • Comments:7

github_iconTop GitHub Comments

3reactions
isaacnorman82commented, Aug 3, 2022

is anything happening with this issue? I just ran into it on trying to use azure.storage.filedatalake.

@navba-MSFT I don’t see how the documentation you linked to helps in any way. Yes there is a parameter called overwrite, the problem is the API seems to fail if that parameter isn’t set to True, even if you’re not overwriting. Also none of the errors thrown are suitable for the situation - it should be something that clearly states the cause of the error is trying to overwrite a file that already exists with overwrite set to false (which is not the case here anyway).

0reactions
jinlowcommented, Aug 30, 2022

Also, experiencing the exact same issue still with the DataLakeFileClient class.

Read more comments on GitHub >

github_iconTop Results From Across the Web

azure.storage.filedatalake.DataLakeFileClient class
Returns all user-defined metadata, standard HTTP properties, and system properties for the file. It does not return the content of the file. query_file....
Read more >
How can I upload a .parquet file from my local machine to ...
DataLakeServiceClient has an upload() method but for .NET, not for Python, and DataLakeFileClient now has an upload_data() method. – MisterJT.
Read more >
Class DataLakeFileClient | Azure SDK for Net
If the file already exists, it will be overwritten. If you don't intent to overwrite an existing file, consider using the CreateIfNotExists(PathHttpHeaders, ...
Read more >
Put Blob Without Overwrite - Steve Marx's blog - smarx.com
A question came up recently of how to store a Windows Azure blob without overwriting the blob if it already exists. This quick...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found