`dotnet tool restore --interactive` fails to restore tools from Artifacts unless I UseDotNet@2 Core 3.1.300 via global.json

Note

Issues in this repo are for tracking bugs, feature requests and questions for the tasks in this repo

For a list:
https://github.com/Microsoft/azure-pipelines-tasks/tree/master/Tasks

If you have an issue or request for the Azure Pipelines service, use developer community instead:

https://developercommunity.visualstudio.com/spaces/21/index.html )

Required Information

Entering this information will route you directly to the right team and expedite traction.

Question, Bug, or Feature?
Type: Bug

Enter Task Name: CmdLine or DotNetCoreCLI

list here (V# not needed):
https://github.com/Microsoft/azure-pipelines-tasks/tree/master/Tasks

Environment

• Server - Azure Pipelines?

  • If using Azure Pipelines, provide the account name, team project name, build definition name/build number: https://dev.azure.com/SIEE/IAMM/_build?definitionId=14

• Agent - Hosted :

  • If using Hosted agent, provide agent queue name:

Agent name: ‘Hosted Agent’ Agent machine name: ‘fv-az674’ Current agent version: ‘2.169.1’

Issue Description

Auth to artifact fails when restoring tools (and a normal dotnet restore too) unless I install a newer version of core. Did something get updated in Core 3.1.300 that fixes this?

NuGet.Config:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <clear />
    <add key="IAMM" value="https://pkgs.dev.azure.com/SIEE/IAMM/_packaging/IAMM/nuget/v3/index.json" />
  </packageSources>
</configuration>

Pipeline that fails.

trigger:
- master

pool:
  vmImage: 'windows-latest'

steps:

- task: NuGetAuthenticate@0

- task: CmdLine@2
  displayName: 'Execute Cake Script'
  inputs:
    script: |
     dotnet tool restore --interactive
     dotnet cake

Pipeline that works (with a global.json using 3.1.300)

- task: NuGetAuthenticate@0

- task: UseDotNet@2
  displayName: Install .NET Core
  inputs:
    useGlobalJson: true

- task: CmdLine@2
  displayName: 'Execute Cake Script'
  inputs:
    script: |
      dotnet tool restore --interactive
      dotnet cake

Task logs

2020-06-03T16:45:09.3310611Z ##[section]Starting: Execute Cake Script
2020-06-03T16:45:09.3416578Z ==============================================================================
2020-06-03T16:45:09.3416881Z Task         : Command line
2020-06-03T16:45:09.3417188Z Description  : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows
2020-06-03T16:45:09.3417481Z Version      : 2.164.2
2020-06-03T16:45:09.3417695Z Author       : Microsoft Corporation
2020-06-03T16:45:09.3418016Z Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/command-line
2020-06-03T16:45:09.3418368Z ==============================================================================
2020-06-03T16:45:10.4511412Z Generating script.
2020-06-03T16:45:10.4889845Z ========================== Starting Command Output ===========================
2020-06-03T16:45:10.5146856Z ##[command]"C:\windows\system32\cmd.exe" /D /E:ON /V:OFF /S /C "CALL "d:\a\_temp\6e836ed0-7808-4dcb-bc58-f26f59ee34ca.cmd""
2020-06-03T16:45:16.9848633Z 
2020-06-03T16:45:17.0784976Z Welcome to .NET Core 3.1!
2020-06-03T16:45:17.0785371Z ---------------------
2020-06-03T16:45:17.0785625Z SDK Version: 3.1.202
2020-06-03T16:45:17.0785759Z 
2020-06-03T16:45:17.0788874Z Telemetry
2020-06-03T16:45:17.0789194Z ---------
2020-06-03T16:45:17.0792266Z The .NET Core tools collect usage data in order to help us improve your experience. The data is anonymous. It is collected by Microsoft and shared with the community. You can opt-out of telemetry by setting the DOTNET_CLI_TELEMETRY_OPTOUT environment variable to '1' or 'true' using your favorite shell.
2020-06-03T16:45:17.0793475Z 
2020-06-03T16:45:17.0793884Z Read more about .NET Core CLI Tools telemetry: https://aka.ms/dotnet-cli-telemetry
2020-06-03T16:45:17.0794364Z 
2020-06-03T16:45:17.0794826Z ----------------
2020-06-03T16:45:17.0795058Z Explore documentation: https://aka.ms/dotnet-docs
2020-06-03T16:45:17.0795398Z Report issues and find source on GitHub: https://github.com/dotnet/core
2020-06-03T16:45:17.0796083Z Find out what's new: https://aka.ms/dotnet-whats-new
2020-06-03T16:45:17.0796668Z Learn about the installed HTTPS developer cert: https://aka.ms/aspnet-core-https
2020-06-03T16:45:17.0797093Z Use 'dotnet --help' to see available commands or visit: https://aka.ms/dotnet-cli-docs
2020-06-03T16:45:17.0797450Z Write your first app: https://aka.ms/first-net-core-app
2020-06-03T16:45:17.0797781Z --------------------------------------------------------------------------------------
2020-06-03T16:45:38.6557290Z C:\Program Files\dotnet\sdk\3.1.202\NuGet.targets(124,5): error : Problem starting the plugin 'C:\Users\VssAdministrator\.nuget\plugins\netcore\CredentialProvider.Microsoft\CredentialProvider.Microsoft.dll'. Plugin 'CredentialProvider.Microsoft' failed within 6.361 seconds with exit code -1. [C:\Users\VssAdministrator\AppData\Local\Temp\tckqgfrc.nja\restore.csproj]
2020-06-03T16:45:38.8598216Z C:\Program Files\dotnet\sdk\3.1.202\NuGet.targets(124,5): error : Unable to load the service index for source https://pkgs.dev.azure.com/SIEE/IAMM/_packaging/IAMM/nuget/v3/index.json. [C:\Users\VssAdministrator\AppData\Local\Temp\tckqgfrc.nja\restore.csproj]
2020-06-03T16:45:38.8599346Z C:\Program Files\dotnet\sdk\3.1.202\NuGet.targets(124,5): error :   Response status code does not indicate success: 401 (Unauthorized). [C:\Users\VssAdministrator\AppData\Local\Temp\tckqgfrc.nja\restore.csproj]
2020-06-03T16:45:38.9752582Z 
2020-06-03T16:45:38.9774513Z Package "cake.tool" failed to restore, due to Microsoft.DotNet.ToolPackage.ToolPackageException: The tool package could not be restored.
2020-06-03T16:45:38.9775464Z    at Microsoft.DotNet.Tools.Tool.Install.ProjectRestorer.Restore(FilePath project, PackageLocation packageLocation, String verbosity)
2020-06-03T16:45:38.9776279Z    at Microsoft.DotNet.ToolPackage.ToolPackageInstaller.InstallPackageToExternalManagedLocation(PackageLocation packageLocation, PackageId packageId, VersionRange versionRange, String targetFramework, String verbosity)
2020-06-03T16:45:38.9778882Z    at Microsoft.DotNet.Tools.Tool.Restore.ToolRestoreCommand.InstallPackages(ToolManifestPackage package, Nullable`1 configFile)
2020-06-03T16:45:38.9779283Z 
2020-06-03T16:45:38.9779624Z Restore failed.
2020-06-03T16:45:39.1789940Z Run "dotnet tool restore" to make the "dotnet-cake" command available.
2020-06-03T16:45:39.2953522Z ##[error]Cmd.exe exited with code '1'.
2020-06-03T16:45:39.3333143Z ##[section]Finishing: Execute Cake Script

Troubleshooting

Checkout how to troubleshoot failures and collect debug logs: https://docs.microsoft.com/en-us/vsts/build-release/actions/troubleshooting

Error logs

[Insert error from the logs here for a quick overview]

[esond]

For all those that come after me:

I will be posting this response on a few issues that I frequented when troubleshooting a similar issue. This is not strictly related to this particular issue but someone may find it useful in the end.

In my case, we were having issues with intermittent packages not being restored from a nuget feed with a networking error, for example:

  Failed to download package 'System.Data.OleDb.4.7.0' from 'https://pkgs.dev.azure.com/acme/af808fc2-ae7d-42da-887a-ea7a78a65e52/_packaging/7648731a-3d0b-4888-816b-d229e1e80fd4/nuget/v3/flat2/system.data.oledb/4.7.0/system.data.oledb.4.7.0.nupkg'.
  The SSL connection could not be established, see inner exception.
    Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host..
    An existing connection was forcibly closed by the remote host.

Multiple packages would fail, and the packages that failed would be different every time. We would also periodically see a dotnet tool restore command simply hang and never complete.

We eventually figured out that these issues only seemed to be occurring when multiple builds were happening in parallel on Azure DevOps. This led me to believe that perhaps there was some kind of rate-limiting happening against the Azure Artifacts server, because the solution in question had ~70 projects to be restored, thus making a lot of network calls.

I enabled caching of nuget packages to alleviate this, which seemed to solve the problem. Below is a template YAML that I have added for all build tasks in our pipeline, and this seems to work for us.

steps:
  - task: NuGetAuthenticate@0
    displayName: "Authenticate with NuGet"
    inputs:
      forceReinstallCredentialProvider: true

  - task: UseDotNet@2
    displayName: Install .NET Core
    inputs:
      useGlobalJson: true

  - task: Cache@2
    displayName: Cache NuGet packages
    inputs:
      key: "nuget | $(Agent.OS) | **/global.json, **/*.csproj"
      restoreKeys: "nuget | $(Agent.OS)"
      path: $(NUGET_PACKAGES)

Usage example:

jobs:
  - job: Compile
    displayName: "⚙ Compile"
    dependsOn: []
    steps:
      - template: setup-build-task.azure-pipelines.yml

      - task: CmdLine@2
        displayName: Restore, lint, and build
        inputs:
          script: "./build.cmd RestoreTools Restore Lint Compile --skip"

Godspeed. I had a hell of a few days with this one.

[apluche]

Sometimes dotnet goes to nuget under the hood (I think nuget is shipped with dotnet) so I just wanted to cover all of the possibilities. Glad to hear you were able to delete those lines without having the pipeline fail.

If you need any other help, please let me know or create a new issue. If you do end up seeing another CredentialProvider.Microsoft\CredentialProvider.Microsoft issue, you can create an issue here: https://github.com/Microsoft/artifacts-credprovider/issues