Azure file copy fails to copy to Azure Blob

Note

Issues in this repo are for tracking bugs, feature requests and questions for the tasks in this repo

For a list:
https://github.com/Microsoft/azure-pipelines-tasks/tree/master/Tasks

If you have an issue or request for the Azure Pipelines service, use developer community instead:

https://developercommunity.visualstudio.com/spaces/21/index.html )

Required Information

Entering this information will route you directly to the right team and expedite traction.

Question, Bug, or Feature?
Type: Bug

Enter Task Name: Azure file copy

Environment

• Server - Azure Pipelines

• Account name - dalgs

• Team project name - Digital Workplace

• Release definition - directory-webapp-release/202001.5

• Agent - Hosted windows-2019

Issue Description

Task logs

ReleaseLogs_561.zip

Troubleshooting

Checkout how to troubleshoot failures and collect debug logs: https://docs.microsoft.com/en-us/vsts/build-release/actions/troubleshooting

Error logs

2020-01-15T16:49:40.7822699Z ##[debug][Azure Call]Getting resource details for azure storage account resource: dgsappdirectoryint with resource type: Microsoft.Storage/storageAccounts 2020-01-15T16:49:42.1074825Z ##[debug]System.NullReferenceException: Object reference not set to an instance of an object. 2020-01-15T16:49:42.1096413Z ##[debug] at Microsoft.Azure.Commands.Management.Storage.Models.PSStorageAccount..ctor(StorageAccount storageAccount) in d:\workspace\powershell-sign\src\ResourceManager\Storage\Commands.Management.Storage\Models\PSStorageAccount.cs:line 27 2020-01-15T16:49:42.1111457Z ##[debug] at Microsoft.Azure.Commands.Management.Storage.Models.PSStorageAccount.Create(StorageAccount storageAccount, IStorageManagementClient client) in d:\workspace\powershell-sign\src\ResourceManager\Storage\Commands.Management.Storage\Models\PSStorageAccount.cs:line 87 2020-01-15T16:49:42.1132108Z ##[debug] at Microsoft.Azure.Commands.Management.Storage.StorageAccountBaseCmdlet.<>c__DisplayClass2.<WriteStorageAccountList>b__1(StorageAccount storageAccount) in d:\workspace\powershell-sign\src\ResourceManager\Storage\Commands.Management.Storage\StorageAccount\StorageAccountBaseCmdlet.cs:line 161 2020-01-15T16:49:42.1146318Z ##[debug] at Microsoft.WindowsAzure.Commands.Utilities.Common.GeneralUtilities.ForEach[T](IEnumerable1 sequence, Action1 action) in d:\workspace\powershell-sign\src\Common\Commands.Common\GeneralUtilities.cs:line 147 2020-01-15T16:49:42.1167097Z ##[debug] at Microsoft.Azure.Commands.Management.Storage.StorageAccountBaseCmdlet.WriteStorageAccountList(IEnumerable1 storageAccounts) in d:\workspace\powershell-sign\src\ResourceManager\Storage\Commands.Management.Storage\StorageAccount\StorageAccountBaseCmdlet.cs:line 161 2020-01-15T16:49:42.1180004Z ##[debug] at Microsoft.WindowsAzure.Commands.Utilities.Common.AzurePSCmdlet.ProcessRecord() in d:\workspace\powershell-sign\src\Common\Commands.Common\AzurePSCmdlet.cs:line 610 2020-01-15T16:49:42.3306724Z ##[debug]Processed: ##vso[task.logissue type=error;code={“Task_Internal_Error”:“TemporaryCopyingToBlobContainerFailed”};] 2020-01-15T16:49:42.3351803Z ##[debug]Trying to disconnect from Azure and clear context at process scope 2020-01-15T16:49:42.5623196Z ##[debug]Caught exception from task script. 2020-01-15T16:49:42.5655643Z ##[debug]Error record: 2020-01-15T16:49:42.6254818Z ##[debug]Get-AzureRmStorageAccount : Object reference not set to an instance of an object. 2020-01-15T16:49:42.6267065Z ##[debug]At D:\a_tasks\AzureFileCopy_eb72cb01-a7e5-427b-a8a1-1b31ccac8a43\2.154.16\AzureUtilityGTE1.0.ps1:27 char:47 2020-01-15T16:49:42.6279224Z ##[debug]+ … ResourceDetails = Get-AzureRmStorageAccount -ErrorAction Stop | Where … 2020-01-15T16:49:42.6290268Z ##[debug]+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2020-01-15T16:49:42.6303078Z ##[debug] + CategoryInfo : CloseError: (😃 [Get-AzureRmStorageAccount], NullReferenceException 2020-01-15T16:49:42.6314916Z ##[debug] + FullyQualifiedErrorId : Microsoft.Azure.Commands.Management.Storage.GetAzureStorageAccountCommand 2020-01-15T16:49:42.6326337Z ##[debug] 2020-01-15T16:49:42.6345394Z ##[debug]Script stack trace: 2020-01-15T16:49:42.6379888Z ##[debug]at Get-AzureStorageAccountResourceGroupName, D:\a_tasks\AzureFileCopy_eb72cb01-a7e5-427b-a8a1-1b31ccac8a43\2.154.16\AzureUtilityGTE1.0.ps1: line 27 2020-01-15T16:49:42.6392058Z ##[debug]at Get-AzureStorageKeyFromARM, D:\a_tasks\AzureFileCopy_eb72cb01-a7e5-427b-a8a1-1b31ccac8a43\2.154.16\AzureUtilityRest.ps1: line 28 2020-01-15T16:49:42.6406408Z ##[debug]at Get-StorageKey, D:\a_tasks\AzureFileCopy_eb72cb01-a7e5-427b-a8a1-1b31ccac8a43\2.154.16\Utility.ps1: line 149 2020-01-15T16:49:42.6422458Z ##[debug]at <ScriptBlock>, D:\a_tasks\AzureFileCopy_eb72cb01-a7e5-427b-a8a1-1b31ccac8a43\2.154.16\AzureFileCopy.ps1: line 101 2020-01-15T16:49:42.6434089Z ##[debug]at <ScriptBlock>, <No file>: line 1 2020-01-15T16:49:42.6446172Z ##[debug]at <ScriptBlock>, <No file>: line 22 2020-01-15T16:49:42.6459269Z ##[debug]at <ScriptBlock>, <No file>: line 18 2020-01-15T16:49:42.6470994Z ##[debug]at <ScriptBlock>, <No file>: line 1 2020-01-15T16:49:42.6489785Z ##[debug]Exception: 2020-01-15T16:49:42.6511296Z ##[debug]System.NullReferenceException: Object reference not set to an instance of an object. 2020-01-15T16:49:42.6523029Z ##[debug] at Microsoft.Azure.Commands.Management.Storage.Models.PSStorageAccount…ctor(StorageAccount storageAccount) in d:\workspace\powershell-sign\src\ResourceManager\Storage\Commands.Management.Storage\Models\PSStorageAccount.cs:line 27 2020-01-15T16:49:42.6535296Z ##[debug] at Microsoft.Azure.Commands.Management.Storage.Models.PSStorageAccount.Create(StorageAccount storageAccount, IStorageManagementClient client) in d:\workspace\powershell-sign\src\ResourceManager\Storage\Commands.Management.Storage\Models\PSStorageAccount.cs:line 87 2020-01-15T16:49:42.6547081Z ##[debug] at Microsoft.Azure.Commands.Management.Storage.StorageAccountBaseCmdlet.<>c__DisplayClass2.<WriteStorageAccountList>b__1(StorageAccount storageAccount) in d:\workspace\powershell-sign\src\ResourceManager\Storage\Commands.Management.Storage\StorageAccount\StorageAccountBaseCmdlet.cs:line 161 2020-01-15T16:49:42.6558457Z ##[debug] at Microsoft.WindowsAzure.Commands.Utilities.Common.GeneralUtilities.ForEach[T](IEnumerable1 sequence, Action1 action) in d:\workspace\powershell-sign\src\Common\Commands.Common\GeneralUtilities.cs:line 147 2020-01-15T16:49:42.6569978Z ##[debug] at Microsoft.Azure.Commands.Management.Storage.StorageAccountBaseCmdlet.WriteStorageAccountList(IEnumerable1 storageAccounts) in d:\workspace\powershell-sign\src\ResourceManager\Storage\Commands.Management.Storage\StorageAccount\StorageAccountBaseCmdlet.cs:line 161 2020-01-15T16:49:42.6581642Z ##[debug] at Microsoft.WindowsAzure.Commands.Utilities.Common.AzurePSCmdlet.ProcessRecord() in d:\workspace\powershell-sign\src\Common\Commands.Common\AzurePSCmdlet.cs:line 610 2020-01-15T16:49:42.6641627Z ##[error]Object reference not set to an instance of an object. 2020-01-15T16:49:42.6642015Z ##[debug]Processed: ##vso[task.logissue type=error]Object reference not set to an instance of an object.

[asranja]

Yes, this was a breaking change with Azure File Copy V4 (hence a new version).

The Service Principal needs to also have Storage Blob Data Contributor permissions on the Storage account. This is because of AzCopy.exe logging in using the Service Principal Credentials in V4 task. Can you assign the permission and re-queue the pipeline. More details on Azure File Copy V4 task and roles.

AzureFileCopyV4 task is based on new Az modules as opposed to older AzureRm modules. V4 is the way going forward. That being said, we are working on finding the root cause in V2 and V3, and will come with a fix asap.

[20shivangi]

@trevonmckay We have fixed the issue in latest version of AzureFileCopy. It will take at least 5-6 weeks to get deployed.