Support for semaphore

This may be a bit out there for a “lightweight” library but have you thought of implementing a distributed counting semaphore?

In my scenario, I have a sql azure database with many customers. Customers can have a lot of data. The .net code is hosted in several azure web instances.

I regularly want to process every customer data. However, I don’t want to process all customers at the same time because it would overload the database. I also don’t want to process each customer serially (1 by 1) because it would take too long. Ideally I would use a distributed semaphore with a count of N to process N customers at a time…

It could be built on top of SqlDistributedLock and I wonder if you’ve come across this scenario before? Do you see that fitting within the vision of your library?

[madelson]

@alastairtree the algorithm I ended up with is essentially this (implemented in SQL):

// the idea of the preamble is to make sure we never hit a case where a lock acquisition with a timeout
// (e. g. TryAcquire) returns false due to timing out on the busy wait lock (below) even though there
// were tickets available
preambleLock = name + '_preamble'
lock (preambleLock) // no blocking inside this; so we can wait for it regardless of timeout
{
     waiterCount = SELECT COUNT(*) FROM tempdb.sys.tables WHERE name like '##' + name + 'marker%';
     var markerTable = '##' + name + 'marker' + GUID;
     CREATE TABLE markerTable;

     if (waiterCount < maxCount) { // space available; just take it
         for (i = 0; i < maxCount; ++i) {
             ticketLock = name + i;
             if (tryAcquire(ticketLock)) { return (ticketLock, markerTable); }
         }
         throw "should have acquired";
     }
}

// the idea of the busy wait lock is that threads queue up here to do a spin wait.
// This both ensures fairness (one spinner at a time) and should prevent too much CPU
// load with many waiters (all but one blocked)
busyWaitLock = name + '_busyWait'
if (tryLock(busyWaitLock, timeout)) {
    while (!timeout expired) {
           for (var i = 0; i < maxCount; ++i) {
                 ticketLock = name + i;
                 if (tryAcquire(ticketLock)) { return (ticketLock, markerTable); }
           }
    }
}

// failed acquire
DROP TABLE markerTable;

To clean up, you just drop the marker temp table along with the ticket lock you took. The full code is in https://github.com/madelson/DistributedLock/blob/master/DistributedLock/Sql/SqlSemaphore.cs#L347

There are some additional complexities there dealing with cancellation and lock reentrance.

Give it a try and let me know how it goes.

[madelson]

This is available as of version 1.4