question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

ListObjectsAsync does not work 50% of the time in asynchronous code

See original GitHub issue

ListObjectsAsync does not work 50% of the time in asynchronous code.

Thas was my method to get list of objects from folder

private List<string> GetObjectFiles(string id)
{
    var observable = _client.ListObjectsAsync(_bucketName, id + "/");
    var files = new List<string>();
    try
    {
        var subscription = observable.Subscribe(
            item => { files.Add(item.Key); },
            ex => throw ex);
        observable.Wait();
        subscription.Dispose();
    }
    catch (InvalidOperationException ex)
    {
        if (!ex.Message.Contains("Sequence contains no elements"))
            throw;
    }

    return files;
}

I also tried this

private async Task<List<string>> GetObjectFiles(string id)
{
    var observable = _client.ListObjectsAsync(_bucketName, id + "/");
    var files = new List<string>();
    try
    {
        var subscription = observable.Subscribe(
            item => { files.Add(item.Key); },
            ex => throw ex);
        await observable.ToTask();
        subscription.Dispose();
    }
    catch (InvalidOperationException ex)
    {
        if (!ex.Message.Contains("Sequence contains no elements"))
            throw;
    }

    return files;
}

But out of 100 or 1000 runs, 30-50% returned an empty list of files

 for (int i = 0; i < 100; i++)
            {
                await InScopeAsync(async (scoped) =>
                {
                    var res = await scoped.Provider.GetRequiredService<IFileStorageClient>()
                        .GetObjectFiles("sign-b7f4a21dbd2648fa8b854df98818c41b");
                    Assert.True(res.Count() == 6);
                });    
            }

Then I had to go into the source code of the client I found GetObjectListAsync and made this method which just call http method

private async Task<List<string>> GetObjectFiles(string id)
        {
            MethodInfo meth = _client.GetType().GetMethod("GetObjectListAsync", BindingFlags.NonPublic | BindingFlags.Instance);
            Task<Tuple<ListBucketResult, List<Item>>> res = (Task<Tuple<ListBucketResult, List<Item>>>)meth.Invoke(_client, new object[] {_bucketName, id + "/", "", null, default(CancellationToken) });
            var res1 = await res;
            return res1.Item2
                .Select( c => c.Key)
                .ToList();
        }

Can you make GetObjectListAsync public? Because i think a lot of developers don’t want to use observable, for simple http request

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:5 (1 by maintainers)

github_iconTop GitHub Comments

1reaction
zolotarevandrewcommented, Jan 13, 2022

use this

private async Task<List<string>> GetObjectFiles(string id)
        {
            MethodInfo meth = _client.GetType().GetMethod("GetObjectListAsync", BindingFlags.NonPublic | BindingFlags.Instance);
            Task<Tuple<ListBucketResult, List<Item>>> res = (Task<Tuple<ListBucketResult, List<Item>>>)meth.Invoke(_client, new object[] {_bucketName, id + "/", "", null, default(CancellationToken) });
            var res1 = await res;
            return res1.Item2
                .Select( c => c.Key)
                .ToList();
        }

This happened to me too. Any news?

0reactions
nop00commented, Feb 10, 2023

I’m running into the same issue, even with the latest release (4.0.7).

If anyone needs it, here’s an updated version of @zolotarevandrew’s workaround that works with release 4.0.7:

private async Task<List<string>> GetObjectFiles(MinioClient client, string bucket, string prefix) {
	MethodInfo meth = client.GetType().GetMethod("GetObjectListAsync", BindingFlags.NonPublic | BindingFlags.Instance, new Type[] {
		typeof(string),
		typeof(string),
		typeof(string),
		typeof(string),
		typeof(CancellationToken)
	});
	Task<Tuple<ListBucketResult, List<Item>>> res = (Task<Tuple<ListBucketResult, List<Item>>>)meth.Invoke(client, new object[] { bucket, prefix + "/", "", null, default(CancellationToken) });
	var res1 = await res;
	return res1.Item2
		.Select(c => c.Key)
		.ToList();
}
Read more comments on GitHub >

github_iconTop Results From Across the Web

[not our bug] how to use ListObjectsAsync? · Issue #294
I uploaded two files and expected to get a list with the two bucket keys. I received an empty list. In order to...
Read more >
How to get the result from IObservable subscription without ...
I'm trying to get filenames within a bucket of my MinIO server using the ListObjectsAsync method. Here is the relevant code: public async...
Read more >
AmazonS3Client Class | AWS SDK for .NET V3
Asynchronous operations (methods ending with Async) in the table below are ... To verify that all parts have been removed, so you don't...
Read more >
What is the best approach to call asynchronous method ...
The point is simply saying "don't do sync over async" is insufficient. The better approach would be to make MyMethod3 async..
Read more >
Static async function returns before its finished
Any time i run the code it skips 60% of it. And i dont know why that is happening. I think it has...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found