WebJob: Setting CosmosDBTrigger connection string(s) at runtime

I would like to be able to configure the CosmosDBTrigger connection string at runtime so that I can build the connection string dynamically, for example so I can use secrets stored in Key Vault.

Repro steps

I have the following method defined, using the CosmosDBTrigger:

public Task ProcessChangeFeed(
            [CosmosDBTrigger(
                databaseName: "mydatabase",
                collectionName: "mycollection",
                ConnectionStringSetting = "CosmosDB:ConnectionString",
                LeaseCollectionName = "leases",
                LeaseDatabaseName = "changefeed",
                LeaseConnectionStringSetting = "CosmosDB:ConnectionString",
                CreateLeaseCollectionIfNotExists = true)]
            IReadOnlyList<Document> documents,
            Microsoft.Extensions.Logging.ILogger logger)
        {

I’ve tried the following approaches to set the value of the setting CosmosDB:ConnectionString.

Hard-code the value in my appsettings.json file

This is the only approach that works but requires the full connection string, including the AccountKey to be in the config file, or in the Azure Portal (to avoid checking in secrets) - only really an option for local development.

[
  "CosmosDB":
  {
    "ConnectionString" : "AccountEndpoint=https://localhost:8081/;AccountKey=C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw=="
  }
}

Using IPostConfigureOptions

This runs, but since the trigger doesn’t appear to use IOptions<CosmosDBOptions> it doesn’t help. I’m noting it here for completeness.

configurationBuilder.AddAzureKeyVault("https://mykeyvault.vault.azure.net/");

    public class PostConfigureCosmosDBOptions : IPostConfigureOptions<CosmosDBOptions>
    {
        private readonly IConfiguration configuration;

        public PostConfigureCosmosDBOptions(IConfiguration configuration)
        {
            this.configuration = configuration;
        }

        public void PostConfigure(string name, CosmosDBOptions cosmosDbOptions)
        {
            string cosmosDbAccountKey = this.configuration["CosmosDB:AccountKey"];

            string serviceEndpoint = this.configuration["CosmosDB:ServiceEndpoint"];

            string connectionString = $"AccountEndpoint={serviceEndpoint}‌​;AccountKey={cosmosDbAccountKey}";

            cosmosDbOptions.ConnectionString = connectionString;
        }
    }

Adding the setting that the trigger is looking for using AddInMemoryCollection

Again, this code runs and stepping through I can see it adding the correct value but the trigger doesn’t seem to use it.

string cosmosDbAccountKey = configurationRoot[KeyVaultSecretKeys.CosmosDbAuthKey];

string serviceEndpoint = configurationRoot[ConfigurationKeys.Data.CosmosDb.ServiceEndpoint];

string connectionString = $"AccountEndpoint={serviceEndpoint}‌​;AccountKey={cosmosDbAccountKey}";

configurationBuilder.AddInMemoryCollection(
    new Dictionary<string, string>
    {
        ["CosmosDB:ConnectionString"] = connectionString
    });

Error information

The error I get if the connection string isn’t configured is Request url is invalid which makes sense since it’s presumably null/empty.

System.InvalidOperationException: Cannot create Collection Information for Entity in database hrcm with lease leases in database changefeed : Request url is invalid.
ActivityId: 5d22add4-973b-4685-bee0-38b9c08f3e90, Microsoft.Azure.Documents.Common/2.0.0.0, Windows/10.0.17763 documentdb-netcore-sdk/2.1.3 ---> Microsoft.Azure.Documents.DocumentClientException: Request url is invalid.
ActivityId: 5d22add4-973b-4685-bee0-38b9c08f3e90, Microsoft.Azure.Documents.Common/2.0.0.0, Windows/10.0.17763 documentdb-netcore-sdk/2.1.3
   at Microsoft.Azure.Documents.Client.ClientExtensions.ParseResponseAsync(HttpResponseMessage responseMessage, JsonSerializerSettings serializerSettings)
   at Microsoft.Azure.Documents.Client.GatewayServiceConfigurationReader.GetDatabaseAccountAsync(Uri serviceEndpoint)
   at Microsoft.Azure.Documents.Routing.GlobalEndpointManager.GetDatabaseAccountFromAnyLocationsAsync(Uri defaultEndpoint, IList`1 locations, Func`2 getDatabaseAccountFn)
   at Microsoft.Azure.Documents.Client.GatewayServiceConfigurationReader.InitializeReaderAsync()
   at Microsoft.Azure.Documents.Client.DocumentClient.InitializeGatewayConfigurationReader()
   at Microsoft.Azure.Documents.Client.DocumentClient.GetInitializationTask()
   at Microsoft.Azure.Documents.Client.DocumentClient.EnsureValidClientAsync()
   at Microsoft.Azure.Documents.Client.DocumentClient.ReadDatabasePrivateAsync(String databaseLink, RequestOptions options, IDocumentClientRetryPolicy retryPolicyInstance)
   at Microsoft.Azure.Documents.BackoffRetryUtility`1.<>c__DisplayClass1_0.<<ExecuteAsync>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.Azure.Documents.BackoffRetryUtility`1.ExecuteRetryAsync(Func`1 callbackMethod, Func`3 callShouldRetry, Func`1 inBackoffAlternateCallbackMethod, TimeSpan minBackoffForInBackoffCallback, CancellationToken cancellationToken, Action`1 preRetryCallback)
   at Microsoft.Azure.Documents.ShouldRetryResult.ThrowIfDoneTrying(ExceptionDispatchInfo capturedException)
   at Microsoft.Azure.Documents.BackoffRetryUtility`1.ExecuteRetryAsync(Func`1 callbackMethod, Func`3 callShouldRetry, Func`1 inBackoffAlternateCallbackMethod, TimeSpan minBackoffForInBackoffCallback, CancellationToken cancellationToken, Action`1 preRetryCallback)
   at Microsoft.Azure.Documents.BackoffRetryUtility`1.ExecuteAsync(Func`1 callbackMethod, IRetryPolicy retryPolicy, CancellationToken cancellationToken, Action`1 preRetryCallback)
   at Microsoft.Azure.Documents.Client.DocumentClient.CreateDatabaseIfNotExistsPrivateAsync(Database database, RequestOptions options)
   at Microsoft.Azure.WebJobs.Extensions.CosmosDB.CosmosDBService.CreateDatabaseIfNotExistsAsync(Database database) in C:\azure-webjobs-sdk-extensions\src\WebJobs.Extensions.CosmosDB\Services\CosmosDBService.cs:line 48
   at Microsoft.Azure.WebJobs.Extensions.CosmosDB.CosmosDBUtility.CreateDatabaseAndCollectionIfNotExistAsync(ICosmosDBService service, String databaseName, String collectionName, String partitionKey, Int32 throughput) in C:\azure-webjobs-sdk-extensions\src\WebJobs.Extensions.CosmosDB\CosmosDBUtility.cs:line 41
   at Microsoft.Azure.WebJobs.Extensions.CosmosDB.CosmosDBTriggerAttributeBindingProvider.TryCreateAsync(TriggerBindingProviderContext context) in C:\azure-webjobs-sdk-extensions\src\WebJobs.Extensions.CosmosDB\Trigger\CosmosDBTriggerAttributeBindingProvider.cs:line 141
   --- End of inner exception stack trace ---
   at Microsoft.Azure.WebJobs.Extensions.CosmosDB.CosmosDBTriggerAttributeBindingProvider.TryCreateAsync(TriggerBindingProviderContext context) in C:\azure-webjobs-sdk-extensions\src\WebJobs.Extensions.CosmosDB\Trigger\CosmosDBTriggerAttributeBindingProvider.cs:line 146
   at Microsoft.Azure.WebJobs.Host.Triggers.CompositeTriggerBindingProvider.TryCreateAsync(TriggerBindingProviderContext context) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Triggers\CompositeTriggerBindingProvider.cs:line 22
   at Microsoft.Azure.WebJobs.Host.Indexers.FunctionIndexer.IndexMethodAsyncCore(MethodInfo method, IFunctionIndexCollector index, CancellationToken cancellationToken) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Indexers\FunctionIndexer.cs:line 190
   at Microsoft.Azure.WebJobs.Host.Indexers.FunctionIndexer.IndexMethodAsync(MethodInfo method, IFunctionIndexCollector index, CancellationToken cancellationToken) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Indexers\FunctionIndexer.cs:line 167

Expected behaviour

We use the ServiceBusTriggerAttribute and configure the connection string in an implementation of IPostConfigureOptions<ServiceBusOptions> and this works fine. The CosmosDBTrigger behaves differently and doesn’t appear to support this.

Related information

• TargetFramework: netcoreapp2.1
• Microsoft.Azure.WebJobs 3.0.1
• Microsoft.Azure.WebJobs.Extensions.CosmosDB 3.0.1
• Using generic HostBuilder in Program.Main()
• Cosmos DB Emulator 2.0.0

[tjrobinson]

Another approach, which also doesn’t seem to help:

[assembly: WebJobsStartup(typeof(Startup))]
namespace Empactis.CaseManager.AzureFunctions
{
    internal class Startup : IWebJobsStartup
    {
        public void Configure(IWebJobsBuilder builder)
        {
            builder.AddCosmosDB(
                options => options.ConnectionString = "AccountEndpoint=https://localhost:8081/;AccountKey=C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==");
        }
    }
}

[briandunnington]

Here is how I did it using the PostConfigure options - note that you have to make sure the ConnectionStringSetting is set to "" or else the dynamically set value wont be used: https://briandunnington.github.io/azure_functions_dynamic_connection_string