question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Does (Queue) CreateIfNotExistsAsync slow things down when doing lots of storage calls? e.g. lots of Add/Get/Delete messages?

See original GitHub issue

Hi 👋

Question: I’m wondering if calling CreateIfNotExistsAsync does a single request to Azure Storage each time that is called. Leading to: wasted HTTP requests after the first time it’s checked.


Nearly all the examples I keep seeing generally follow the following pattern:

// Retrieve storage account from connection string.
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(
    CloudConfigurationManager.GetSetting("StorageConnectionString"));

// Create the queue client.
CloudQueueClient queueClient = storageAccount.CreateCloudQueueClient();

// Retrieve a reference to a queue.
CloudQueue queue = queueClient.GetQueueReference("myqueue");

// Create the queue if it doesn't already exist.
queue.CreateIfNotExists();

// Now do stuff with the queue, like AddMessage, etc.

This is great if it’s a single operation (e.g. Add 1x message). But if we need to keep adding a heap of messages, over time (ie. not all via one call to this method, but as people “do stuff” with your website, like buy things, signup, whatever) … then these calls get called a lot and if feels like it’s wasted cycles and can slow things down … especially if the CreateIfNotExists() is doing HTTP (or TCP?) calls to the Storage.

So, it’s pretty easy to change things around … like say …

// Inject the storage account credentials.
public QueueService(CloudStorageAccount cloudStorageAccount)
{
    _cloudStorageAccount = cloudStorageAccount ?? throw new ArgumentNullException(nameof(cloudStorageAccount));

    // Create a queue ONCE and only when first required...
    _queueClient = new Lazy<CloudQueue>( () =>
    {
        var queueClient = _cloudStorageAccount.CreateCloudQueueClient();

        var queue = queueClient.GetQueueReference("my-queue");
        queue.CreateIfNotExists(); // DOESN'T EXIST ANYMORE. it's CreateIfNotExistsAsync();
        return queue;
    })
}

So now we can create the queue ONCE when the queue if first ‘requested’. But there are now no sync methods on the CloudQueue. They are all async. which means i need to do an async/await inside a Lazy (urgh, not easy, need 3rd party nuget) and then inside a ctor (urgh. death…).

So I’m stuck. I’m not sure what other people do/are doing.

Sure it’s easy for me to just add the CreateIfNotExistsAsync just before i do my action (AddMessageAsync, etc) but that comes down the crux of my issue -> If i call CreateIfNotExistsAsync each time, that does a single HTTP/TCP request … which is expensive and-or an overkill after i’ve checked for the very first time).

So - any hints?

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Reactions:1
  • Comments:7 (2 by maintainers)

github_iconTop GitHub Comments

1reaction
seanmcc-msftcommented, Mar 14, 2019

Thank you for the discussion, we are closing the issue.

0reactions
xt0rtedcommented, Aug 5, 2018

@PureKrome my project isn’t core yet so I’m running the sync versions real early in my owin startup.

To run asynchronous methods synchronously you should call .GetAwaiter().GetResult() (this ensures any exceptions that are thrown are not wrapped in an AggregateException).

In your webjobs you can call these asynchronously by making your Main method async like so:

public static async Task Main(string[] args)
{
   // ...
}
Read more comments on GitHub >

github_iconTop Results From Across the Web

Performance impact of calling CreateIfNotExistsAsync() on ...
This results in two separate requests to the storage system for every message they want to enqueue, with the create queue failing.
Read more >
Azure Queues Transport speed very slow with zero load
Yes, even when deployed out to Azure the azure queues performed very slowly (even with nagling turned off and the connection count bumped...
Read more >
Remember to Async
By making a call to an external service synchronously it leaves itself open to anything transient in the system slowing you down. If...
Read more >
Forcing an Azure Function to process Queue Storage ...
newBatchThreshold defines our function to check the queue for new messages only when the number of processed messages drops down to this value...
Read more >
QueueClient.CreateIfNotExistsAsync Method
operation creates a new queue under the specified account. If the queue already exists, it is not changed. For more information, see Create...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found