[BUG] Service Bus trigger using identity-based connection uses incorrect user identity when run locally

Library name and version

Microsoft.Azure.WebJobs.Extensions.ServiceBus 5.7.0

Describe the bug

I had a lot of trouble getting a managed identity to work correctly locally for a C# Azure Function with a Service Bus Queue-triggered binding on my devbox with VS Code. The problem seems to stem from the fact that the user identity being picked up at runtime is incorrect.

Context – Local Tooling

• Visual Studio Code – code --version == 1.70.2
• Visual Studio Code extensions Azure Account and Azure Functions are installed.
• dotnet --version == 6.0.400
• .csproj Packages: Microsoft.NET.Sdk.Functions 4.1.1, Microsoft.Azure.WebJobs.Extensions.ServiceBus 5.7.0

Context – Cloud

• I have two users which belong to separate tenants: user A who is homed to tenant A, and user B who is homed to tenant B.
• Tenant B contains a Service Bus namespace with a Queue.
• User B is assigned appropriate RBAC roles to read and write from the Queue.

Context – Local Dev Workflow

• I am logged into Windows as user A.
• From Terminal, I perform az account show, which shows me I am logged in as user A (associated with tenant A).
• From Terminal, I do an az login and login as user B.
• From Terminal, I perform az account show, which shows me I am now logged in as user B (associated with tenant B).
• From this Terminal, I launch VS Code (code .) in the folder with my Azure Functions project containing a Service Bus Queue-triggered function.
• Inside VS code, I login to the Azure Account extension with user B. I confirm this login succeeded: I see user B’s email address in the bottom status bar AND see user B’s Azure subscriptions/resources in the Azure extension blade.
• In my local.settings.json file, I specify the ServiceBusConnection__fullyQualifiedNamespace setting as the service bus namespace’s hostname.
• In my local.settings.json file, I specify the ServiceBusConnection__tenantId setting as the id for tenant B.
• I start debugging the project from VS Code.

The Error / Actual Behavior

When I run the code, I see this error spammed in the VS Code terminal:

System.Private.CoreLib: Put token failed. status-code: 401, status-description: InvalidIssuer: Token issuer is invalid. TrackingId:d4148e00-3538-4ddc-b199-7a39f897138f, SystemTracker:NoSystemTracker, Timestamp:2022-09-06T17:11:15.

The error message is indicative of the fact that the auth token presented to the Service Bus in tenant B was issued by tenant A.

In short, the ServiceBusConnection__tenantId setting does not seem to work.

The Fix / Expected Behavior

Instead, what did work, and what I only found thanks to this seemingly incorrectly closed Github issue, was specifying the id for tenant B in AZURE_TENANT_ID in local.settings.json.

After specifying this setting, the error went away, and the function was triggered by messages picked up from the queue.

It’s not clear to me why this setting is needed, because if the user context is already being picked up from the VS Code extension OR the login done with Azure CLI, the proper tenant for that user should already be known.

At the very least, the documentation should be updated to reflect the need to specify the AZURE_TENANT_ID setting. If this approach is taken, please revise these documents:

• Azure Service Bus trigger for Azure Functions | Identity-based connections
• Azure Functions developer guide | Local development with identity-based connections

Expected behavior

No error is seen and the function can be triggered by messages picked up from the queue.

Actual behavior

When I run the project that hosts the function, I see this error spammed in the VS Code terminal:

System.Private.CoreLib: Put token failed. status-code: 401, status-description: InvalidIssuer: Token issuer is invalid. TrackingId:d4148e00-3538-4ddc-b199-7a39f897138f, SystemTracker:NoSystemTracker, Timestamp:2022-09-06T17:11:15.

As best I can tell, this error message is indicative of the fact that the auth token presented to the Service Bus in tenant B was issued by tenant A.

In short, the ServiceBusConnection__tenantId setting does not seem to work.

Reproduction Steps

With a setup as described in the bug description, run the project to start the functions host with a SB queue-triggered function like the one below. Wait until you see the error spam.

public class MyTestClass
    {
        [FunctionName("MyTestFunction")]

        // ServiceBusConnection resolves to ServiceBusConnection__fullyQualifiedNamespace
        // which is referenced within the local.settings.json file to use the managed identity
        public void Run([ServiceBusTrigger("%MyQueueName%", Connection = "ServiceBusConnection")]string command, ILogger log)
        {
            log.LogInformation($"Function triggered with command: {command}");
        }       
    }

Environment

• Visual Studio Code – code --version == 1.70.2
• Visual Studio Code extensions Azure Account and Azure Functions are installed.
• dotnet --version == 6.0.400
• .csproj Packages: Microsoft.NET.Sdk.Functions 4.1.1, Microsoft.Azure.WebJobs.Extensions.ServiceBus 5.7.0

[ashtmMSFT]

I was able to get additional logs by adding the "AzureFunctionsJobHost__logging__logLevel__Default": "Debug" key/value pair to my local.settings.json.

I re-ran all the scenarios described above and was able to determine how the usage of DefaultAzureCredential (and all the underlying chained credentials) failed to match my expectations for its behavior. Rather than share logs and the exact type of credential ultimately used in each scenario above, I’ll just highlight the points of confusion.

The DefaultAzureCredential attempted to use the following credentials in this order on my machine: EnvironmentCredential, ManagedIdentityCredential, VisualStudioCredential, VisualStudioCodeCredential, AzureCliCredential. (Note the SharedTokenCacheCredential was never used despite being mentioned in the documentation.)

EnvironmentCredential The EnvironmentCredential documentation indicates that a multitude of AZURE_* environment variables can be set to to facilitate authentication to Azure AD. This led me to believe that I could specify AZURE_TENANT_ID, AZURE_USERNAME, and AZURE_PASSWORD to have the DefaultAzureCredential login to a particular tenant as the user I specify. This belief was further supported by the documentaton stating that a UsernamePasswordCredential is used under the covers, and that class is constructed by providing a username, password, and tenantId.

Unfortunately, specifying a (username, password, and tenantId) tuple via environment variables is not supported. If you specify a tenantId, the EnvironmentCredential assumes you are not using a username and password and will throw Exception: Azure.Identity.CredentialUnavailableException (0x80131500): EnvironmentCredential authentication unavailable. Environment variables are not fully configured. See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/environmentcredential/troubleshoot.

The table in the troubleshooting guide contains all valid tuples of AZURE_* environment variables that can be used for authentication.

I suggest that we update the currently ambiguous EnvironmentCredential documentation to include this list of valid environment variable combinations.

VisualStudioCredential This credential provided the biggest surprise: even if you aren’t running your project in Visual Studio, VisualStudioCredential will still pick up the credentials you used to login to Visual Studio if you have it installed. VisualStudioCredential documentation does say it “enables authentication to Azure Active Directory using data from Visual Studio”, but I’d bet good money I’m not the only developer that read that with the implied statement “if you’re running your code in Visual Studio.”

This meant that most of the time my identity was unknowingly being pulled from VS and I never got to leverage the credentials further down the chain where I was logged in with the appropriate identity.

Interestingly, if the AZURE_TENANT_ID environment variable is set when your code runs and it does not match the tenantId for the user with which you’re logged into Visual Studio, VisualStudioCredential will try to authenticate your stored user identity against this specified tenant. In my case, I saw this: Exception: Azure.Identity.CredentialUnavailableException (0x80131500): Process "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\IDE\CommonExtensions\Microsoft\Asal\TokenService\Microsoft.Asal.TokenService.exe" has failed with unexpected error: TS003: Error, TS002: The account '<REDACTED EMAIL OF USER A>' was not found in the tenant '<REDACTED ID OF TENANT B>'.

Triggering this failure was significant in my case because it meant a credential further down the chain (Azure CLI) would be used and succeed.

VisualStudioCodeCredential The VisualStudioCodeCredential documentation states that it “enables authentication to Azure Active Directory using data from Visual Studio Code,” but it doesn’t really mention how. It should be updated to clarify that you need to install and login to the Azure Account extension.

Unfortunately, when you figure that out, it flat out doesn’t work: Exception: Azure.Identity.CredentialUnavailableException (0x80131500): Stored credentials not found. Need to authenticate user in VSCode Azure Account. See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/vscodecredential/troubleshoot

The troubleshooting guide mentions that “it’s a known issue that VisualStudioCodeCredential doesn’t work with Azure Account extension versions newer than 0.9.11.”

Ouch. Great to know, but seeing as how this affects literally every fresh install of that extension, could we please publicize this information more broadly? The VisualStudioCodeCredential documentation and Azure Account extension pages are great candidates.

===

ASK Please update the *CredentialClass documentation pages to provide more detail and clarity. The DefaultAzureCredential page lists the other credential class types and suggests one “consult the documentation of these credential types for more information on how they attempt authentication”, even though little can be learned from them.

[christothes]

This is a good one for @scottaddie