question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[BUG] Synapse LibraryClient AppendAsync results in 409 conflict instead of 201 created

See original GitHub issue

Describe the bug When trying to upload a synapse workspace library using the following calls, the AppendAsync fails with 409 conflict: LibraryAlreadyExists. However it looks like it is missing the query string comp=appendblock parameter here which if supplied, allows a library to be uploaded directly against REST API (i.e. not via SDK).

Expected behavior Expect 201 Created response.

Actual behavior (include Exception or Stack Trace)

Azure.RequestFailedException: Service request failed.
Status: 409 (Conflict)

Content:
{"code":"LibraryAlreadyExists","message":"The LibraryArtifact my_library-0.0.32-py3-none-any.whl already exists."}

Headers:
Server: Microsoft-HTTPAPI/2.0
Strict-Transport-Security: REDACTED
x-ms-request-id: REDACTED
x-ms-client-request-id: 36fa6ca1-08ec-4124-b90c-de1b71661a1c
Date: Tue, 04 May 2021 14:35:23 GMT
Content-Length: 132
Content-Type: application/json; charset=utf-8

   at Azure.Analytics.Synapse.Artifacts.LibraryRestClient.AppendAsync(String libraryName, Stream content, Nullable`1 xMsBlobConditionAppendpos, CancellationToken cancellationToken)
   at Azure.Analytics.Synapse.Artifacts.LibraryClient.AppendAsync(String libraryName, Stream content, Nullable`1 xMsBlobConditionAppendpos, CancellationToken cancellationToken)

To Reproduce Steps to reproduce the behavior (include a code snippet, screenshot, or any additional information that might help us reproduce the issue)

var uri = new Uri($"https://{workspace}.dev.azuresynapse.net");
var options = new ArtifactsClientOptions() {                                
};

var filename = Path.GetFileName(file);

var cred = new ClientSecretCredential(
    Environment.GetEnvironmentVariable("AZURE_TENANT_ID"),
    Environment.GetEnvironmentVariable("AZURE_CLIENT_ID"),
    Environment.GetEnvironmentVariable("AZURE_CLIENT_SECRET"));

var client = new LibraryClient(uri, cred, options);

var startResponse = await client.StartCreateAsync(filename, cancellationToken);
await startResponse.WaitForCompletionAsync();

using (Stream s = File.OpenRead(file)) {
    var response = await client.AppendAsync(filename, s);
    Console.WriteLine("Append status code is " + response.Status);
}

var flush = await client.StartFlushAsync(filename, cancellationToken);            
var flushWaitResponse = await flush.WaitForCompletionAsync();

Environment:

  • Name and version of the Library package used: Azure.Analytics.Synapse.Artifacts 1.0.0-preview.8
  • Hosting platform or OS and .NET runtime version (dotnet --info output for .NET Core projects): [e.g. Azure AppService or Windows 10 .NET Framework 4.8]
.NET Core SDK (reflecting any global.json):
Version:   3.1.406
Commit:    f42e6b201a

Runtime Environment:
OS Name:     Windows
OS Version:  10.0.19042
OS Platform: Windows
RID:         win10-x64
Base Path:   C:\Program Files\dotnet\sdk\3.1.406\

Host (useful for support):
 Version: 3.1.12
 Commit:  0267ad09c6

.NET Core SDKs installed:
 2.1.602 [C:\Program Files\dotnet\sdk]
 2.1.700 [C:\Program Files\dotnet\sdk]
 2.1.803 [C:\Program Files\dotnet\sdk]
 3.1.101 [C:\Program Files\dotnet\sdk]
 3.1.406 [C:\Program Files\dotnet\sdk]

.NET Core runtimes installed:
 Microsoft.AspNetCore.All 2.1.9 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
 Microsoft.AspNetCore.All 2.1.11 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
 Microsoft.AspNetCore.All 2.1.15 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
 Microsoft.AspNetCore.App 2.1.9 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
 Microsoft.AspNetCore.App 2.1.11 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
 Microsoft.AspNetCore.App 2.1.15 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
 Microsoft.AspNetCore.App 3.1.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
 Microsoft.AspNetCore.App 3.1.12 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
 Microsoft.NETCore.App 2.1.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
 Microsoft.NETCore.App 2.1.11 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
 Microsoft.NETCore.App 2.1.15 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
 Microsoft.NETCore.App 3.1.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
 Microsoft.NETCore.App 3.1.12 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
 Microsoft.WindowsDesktop.App 3.1.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
 Microsoft.WindowsDesktop.App 3.1.12 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  • IDE and version : [e.g. Visual Studio 16.3] n/a

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:5 (2 by maintainers)

github_iconTop GitHub Comments

1reaction
LittleColincommented, Sep 24, 2021

Thanks, just updated to Azure.Analytics.Synapse.Artifacts 1.0.0-preview.10 and can confirm the issue is fixed, the following code worked for upload:

using (var s = File.OpenRead(file))
{
    var response = await client.AppendAsync(filename, s);
}
1reaction
lmazuelcommented, Sep 16, 2021

Hi @LittleColin , I believe this issue has been fixed by regeneration of the code that happened recently, could you confirm more recent releases of this SDK solve the problem? Thanks!

Read more comments on GitHub >

github_iconTop Results From Across the Web

How To Fix the “409 Conflict” Error (5 Methods)
The "409 conflict" error can be frustrating, but it is actually easy to fix. Learn how to fix the error using 5 different...
Read more >
FIX: Error 409 occurs when you back up databases by ...
A nonrecoverable I/O error occurred on file FilePath Backup to URL received an exception from the remote endpoint. Exception Message: The remote server...
Read more >
Reasons for a 409/Conflict HTTP error when uploading ...
When several files are uploaded (method is called several times), some requests fail with a 409 Conflict HTTP error. I've googled, and it...
Read more >
409 Conflict - HTTP - MDN Web Docs
The HTTP 409 Conflict response status code indicates a request conflict with the current state of the target resource.
Read more >
Uploading with CMIS results in HTTP 409 Conflict
Hello,. I'm creating a Talend job where I upload PDF documents. This is going fine but. I uploaded and removed 40ish testfiles.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found