AddDaprSecretStore Extension always throws unhandled exception at container start

Expected Behavior

Using AddDaprSecretStore extension will not throw an exception when the side car is not ready yet

Actual Behavior

It throws an unhandled exception crashing my pod. On restart, it goes through fine

Unhandled exception. Dapr.DaprException: Bulk secret operation failed: the Dapr endpoint indicated a failure. See InnerException for details.
 ---> Grpc.Core.RpcException: Status(StatusCode="Internal", Detail="Error starting gRPC call. HttpRequestException: Connection refused (127.0.0.1:50001) SocketException: Connection refused", DebugException="System.Net.Http.HttpRequestException: Connection refused (127.0.0.1:50001)
 ---> System.Net.Sockets.SocketException (111): Connection refused
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
   at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|283_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.DefaultConnectAsync(SocketsHttpConnectionContext context, CancellationToken cancellationToken)
   at System.Net.Http.ConnectHelper.ConnectAsync(Func`3 callback, DnsEndPoint endPoint, HttpRequestMessage requestMessage, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.ConnectHelper.ConnectAsync(Func`3 callback, DnsEndPoint endPoint, HttpRequestMessage requestMessage, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.GetHttp2ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessage request, Nullable`1 timeout)")
   at Dapr.Client.DaprClientGrpc.GetBulkSecretAsync(String storeName, IReadOnlyDictionary`2 metadata, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at Dapr.Client.DaprClientGrpc.GetBulkSecretAsync(String storeName, IReadOnlyDictionary`2 metadata, CancellationToken cancellationToken)
   at Dapr.Extensions.Configuration.DaprSecretStore.DaprSecretStoreConfigurationProvider.LoadAsync()
   at Dapr.Extensions.Configuration.DaprSecretStore.DaprSecretStoreConfigurationProvider.Load()
   at Microsoft.Extensions.Configuration.ConfigurationRoot..ctor(IList`1 providers)
   at Microsoft.Extensions.Configuration.ConfigurationBuilder.Build()
   at Microsoft.AspNetCore.Hosting.WebHostBuilder.BuildCommonServices(AggregateException& hostingStartupErrors)
   at Microsoft.AspNetCore.Hosting.WebHostBuilder.Build()
   at MyApp.Program.Main(String[] args) in /MyApp/Program.cs:line 18

Steps to Reproduce the Problem

Dapr 1.4 running in Kubernetes with a dotnet 5 webapi app.

public static IWebHostBuilder CreateWebHostBuilder(string[] args) => WebHost.CreateDefaultBuilder(args).ConfigureAppConfiguration(config =>
{
  var daprClient = new DaprClientBuilder().Build();
  config.AddDaprSecretStore(KeyvaultStoreName, daprClient);
}).UseStartup<Startup>();

Release Note

RELEASE NOTE:

[harvendra2022]

We have started using Dapr in Microservices written .Net 6 and using DaprSecretStore to get the secretes from Azure Key Vault. It would be great if this issue can be fixed at the earliest? Is there any tentative timeline for the fix?

[halspang]

Actually, healthz/outbound is already marked as ready before it blocks on the app. I’ll just use that since it already exists 😃