Error Suppression and Vault Name Mapping

Describe the proposal

We are working on Azure Key Vault in our project and notice that there is no error suppression functionality (or not granular enough for certain use case) when there the Key Vault Secret name doesn’t have match with application configuration name.

For example; We couldn’t define .NET configuration hierarchy in Azure KeyVault due to naming convention limitation for secret names. Key Vault only accepts dash and Alphanumeric values for the names so we can’t use “ConnectionStrings:DatabaseConnStr” or “ConnectionStrings__DatabaseConnStr” as name.

Purposed solution Allocate special metadata prefix like “Dapr.” in metadata keys and use them to control library behavior.

Example Solution in Library

LoadAsync()  method In  DaprSecretStoreConfigurationProvider.cs file
...

    if (secretDescriptors != null)
            {
                foreach (var secretDescriptor in secretDescriptors)
                {
                    var vaultSecretName = String.Empty; 
                        secretDescriptor.Metadata.TryGetValue("Dapr.VaultMapping.Name", out vaultSecretName);

                        if (!String.IsNullOrWhiteSpace(vaultSecretName))
                            secretName = vaultSecretName;

                        Dictionary<string, string> result =  new Dictionary<string, string>();
                        
                        try
                        {
                            result = await client
                                .GetSecretAsync(store, secretName, secretDescriptor.Metadata)
                                .ConfigureAwait(false);
                        }
                        catch (DaprException e)
                        {
                            string? ensure_exist;
                            secretDescriptor.Metadata.TryGetValue("Dapr.Ensure", out ensure_exist);
                            if (String.IsNullOrWhiteSpace(ensure_exist))
                                ensure_exist = "false";
                            
                            if (Convert.ToBoolean(ensure_exist))
                            {
                                throw e;
                            }
                            
                            Console.WriteLine($"'{secretName}' doesn't exists in Key Vault but because it doesn't require existence so all errors suppressed!");
                           result = new Dictionary<string, string>();

                        }

                        foreach (var key in result.Keys)
                        {
                            if (data.ContainsKey(key))
                            {
                                throw new InvalidOperationException(
                                    $"A duplicate key '{key}' was found in the secret store '{store}'. Please remove any duplicates from your secret store.");
                            }
                            
                            data.Add(normalizeKey ? NormalizeKey(secretDescriptor.SecretName) : secretDescriptor.SecretName, result[key]);
                        }

...

Example usage in Program.cs

var daprClientBuilder = new DaprClientBuilder(); var daprClient = daprClientBuilder.Build();

var secretDescriptors = new List<DaprSecretDescriptor>
{

// Mapped with Vault name with old exception behavior. 
// The Secret name should exist on Keyvault otherwise it will throw the exception. 

    new DaprSecretDescriptor("ConnectionStrings:DatabaseConnStr", new Dictionary<string, string>()
    {
        {"Dapr.Ensure","true"},
        {"Dapr.VaultMapping.Name", "Microsservice-DatabaseConnStr"}
    }),

// The exception will be suppressed
    new DaprSecretDescriptor("ConnectionStrings:StorageLocalFolder",new Dictionary<string, string>()
    {
        {"Dapr.Ensure","false"},
        {"Dapr.VaultMapping.Name", "Object-Storage-ContainerName-CfgNotExists"}
    }),

// Old Behavior     
    new DaprSecretDescriptor("Old-Beahvior-StorageAzureConnectionString")
    
};

IConfiguration config = new ConfigurationBuilder()
    .AddJsonFile("appsettings.json")
    .AddDaprSecretStore("azurekeyvault",  secretDescriptors, daprClient, TimeSpan.FromSeconds(10) )
    .AddEnvironmentVariables()
    .Build();

var connectionString = config.GetConnectionString("DatabaseConnStr");
var fileStorageConnectionString = config.GetConnectionString("StorageLocalFolder");
var azureStorageConnectionString = config.GetConfig("Old-Beahvior-StorageAzureConnectionString");

[yash-nisar]

@BCoskun

We do have an option for multiple secret retrievals in dapr (https://docs.dapr.io/reference/api/secrets_api/#get-bulk-secret) and the dotnet-sdk (https://github.com/dapr/dotnet-sdk/blob/master/src/Dapr.Client/DaprClientGrpc.cs#L1253) if that’s what you are looking for.

There can never be a feature that exists in the SDK but not in the Dapr runtime because the SDK makes calls to the runtime APIs itself via HTTP/GPRC.

As for gracefully handling errors, how about including this block in a try catch block itself ?

var result = await client.GetSecretAsync(store, SecretName, Metadata).ConfigureAwait(false);

[halspang]

@BCoskun - Thanks for opening this issue!

Just to clarify for myself, what we’re trying to do here is:

1. Allow secrets which can’t be stored in the .NET hierarchy pattern to still be referenced in the code that way. We’re doing this by providing a secondary key that will actually be used to query the secretstore.
2. Allow certain keys to not exist, also based on a specific key.

I can see the value of this, but I don’t know if I agree with using the reserved metadata values. The metadata here is actually sent to the secretstore so instead I’d rather just add the direct fields to the DaprSecretDescriptor. Perhaps something like this?

/// <summary>
/// Gets or sets the secret name.
/// </summary>
public string SecretName { get; }

/// <summary>
/// A collection of metadata key-value pairs that will be provided to the secret store. The valid metadata keys and values are determined by the type of secret store used.
/// </summary>
public IReadOnlyDictionary<string, string> Metadata { get; }

public bool IsRequired { get; }

public string SecretKey { get; }

/// <summary>
/// Secret Descriptor Construcutor
/// </summary>
public DaprSecretDescriptor(string secretName) : this(secretName, new Dictionary<string, string>(), true, secretName)
{

}

public DaprSecretDescriptor(string secretName, IReadOnlyDictionary<string, string> metadata, bool isRequired, string secretKey)
{
...
}

Thoughts?