[QUERY] Azure Function Socket Exhaustion issues when using table storage frequently

Library name and version

Azure.Data.Tables 12.3.0

Query/Question

With the below implementation we are storing the status of a job as it goes through different service bus triggers in azure functions. This gets registered as a singleton in the function and the client is reused. Within azure function when job tracking is turned on and this service is heavily used we run into socket exhaustion issues in azure functions reaching the 600 connection limit. With job tracking turned off we get no where near the connection limit. I’m a bit at a loss as to why this is happening when using table storage. We use a lot of blob storage calls as well and we don’t hit this issue so it seems to be related to table storage but I’m not sure what could be going on here.

public class TableStorageService : ITableStorageService
{
        private readonly AzureStorageConfiguration _azureStorageConfiguration;
        
        private readonly ConcurrentDictionary<string, Lazy<TableClient>> _tableClientDirectory;
        
        public TableStorageService(IOptions<AzureStorageConfiguration> azureStorageConfiguration)
        {
	        _azureStorageConfiguration = azureStorageConfiguration.Value;
	        _tableClientDirectory = new ConcurrentDictionary<string, Lazy<TableClient>>();
        }
        
        public async Task AddValue<T>(string tableName, TableItem<T> item)
        {
	        item.PartitionKey = _azureStorageConfiguration.PartitionKey;
	        var client = GetTableClient(tableName);
	        await client.CreateIfNotExistsAsync();
	        using var _ = await client.AddEntityAsync(item);
        }
        
        public async Task UpdateValue<T>(string tableName, TableItem<T> item)
        {
	        item.PartitionKey = _azureStorageConfiguration.PartitionKey;
	        var client = GetTableClient(tableName);
	        await client.CreateIfNotExistsAsync();
	        using var _ = await client.UpdateEntityAsync(item, ETag.All);
        }
        public async Task UpsertValue<T>(string tableName, TableItem<T> item)
        {
	        item.PartitionKey = _azureStorageConfiguration.PartitionKey;
	        var client = GetTableClient(tableName);
	        await client.CreateIfNotExistsAsync();
	        using var _ = await client.UpsertEntityAsync(item);
        }
        public async Task<TableItem<T>> GetSingleValueAsync<T>(string tableName, string key)
        {
	        TableClient client = GetTableClient(tableName);
	        var pageableItems = client.QueryAsync<TableItem<T>>(tableItem => tableItem.Key == key);
	        return pageableItems != null ? await pageableItems.FirstOrDefaultAsync() : null;
        }
        
        private TableClient GetTableClient(string tableName)
        {
	        var connectionString = _azureStorageConfiguration.ConnectionString;
	        
	        return _tableClientDirectory.GetOrAdd(tableName, 
		        (key, conn) => new Lazy<TableClient>(
		        () => new TableClient(conn, key)), connectionString).Value;
        }
}

Environment

Target Framework: net5.0 Azure Function Version: V3

[christothes]

Do you know how can I pass custom http client with MaxConnectionPerServer set to the Azure.Data.Tables especially to “TableClient” or “TableServiceClient”?

See this sample for details on how to configure a custom HttpClient https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/core/Azure.Core/samples/Configuration.md#user-provided-httpclient-instance

[christothes]

Hi @chris-skuvault Although I can’t think of a reason that this port exhaustion would be occurring as a result of using the TableClient (all instances of the client should be reusing the same instance of HttpClient) I do see a change you could make that would make things a bit more efficient.

I’d recommend changing your TableClient initialization slightly as follows:

• In your service initialization, assuming the connection string will always be the same, create a single instance of TableServiceClient
• In your implementation of GetTableClient, instead of constructing a TableClient directly, use TableServiceClient.GetTableClient. This will reuse resources already allocated for the TableServiceClient.