Automated UI Tests Using Visual Studio Online

Follow on from this issue: https://github.com/microsoft/appcenter/issues/25

FAO: @hassanuz

I am trying to Automate my UWP Ui tests using these instructions:

https://github.com/microsoft/WinAppDriver/wiki/WinAppDriver-in-CI-with-Azure-Pipelines.

I am using a Local Build Server

Step 1: Start WinApp Driver Step 2: Install app using this script:

$currentDirectory = Get-Location

Write-Host "Removing FieldStrike if present"
try
{
    Get-AppxPackage MyApp | Remove-AppxPackage #uninstall App if its present
}
catch
{
    Write-Host "failed to remove FieldStrike"
    Write-Host $Error[0]
    exit

}
Write-Host "Installing MyApp"
try
{
    $installScriptLocation = Get-ChildItem -Path $currentDirectory.Path -Recurse -Filter Add-AppDevPackage.ps1

    powershell.exe -File $installScriptLocation.FullName -Force 
}
catch
{
    Write-Host "Failed to install Fieldstrike"
    Write-Host $Error[0]
    exit
}

This fails with

Error: Could not install the app. To ensure security, please consider uninstalling the signing certificate until you can install the app. Instructions for doing so can be found here:

If I run this script locally it works, so my guess is something to do with the user?

Step 3: after installing app locally try to run UI tests using this script:

$currentDirectory = Get-Location

Write-Host "Running Tests"
try
{

    $uiTests= $currentDirectory.Path + "\nunit3-console.exe"
    $testdll = $currentDirectory.Path + "\MyApp.UI.Tests.UWP.dll"
    $logFile = $currentDirectory.Path + "\TestResults.txt"
    Write-Host $uiTests
    Write-Host $testdll
    Write-Host $logFile
	Start-Process $uiTests $testdll -Wait  -RedirectStandardOutput $logFile -WindowStyle Normal
    
}
catch
{
    Write-Host "Error whilst running tests"
    Write-Host $Error[0]
    Stop-Process -Name  "WinAppDriver"
    exit
}


Write-Host "Tests Completed Results in TestResults.txt"
Get-Content -Path $logFile

Now this runs the tests but they all fail with this error

Unexpected error occured: Creating an instance of the COM component with CLSID {45BA127D-10A8-46EA-8AB7-56EA9078943C} from the IClassFactory failed due to the following error: 8000401a The server process could not be started because the configured identity is incorrect. Check the username and password. (Exception from HRESULT: 0x8000401A).

Code in the setup method is like so:

            OpenQA.Selenium.Appium.AppiumOptions desktopOptions = new AppiumOptions();
            desktopOptions.AddAdditionalCapability("app", "Root");
            var desktopSession = new WindowsDriver<WindowsElement>(new Uri(WindowsApplicationDriverUrl), desktopOptions);
            app = new WinDriverApp(desktopSession);

It may be the test initialisation code that is wrong, but there are two issues here, cant install the app automatically and can’t run the tests remotely

[JKennedy24]

@hassanuz I never did manage to get this automated UWP UI tests up and running in the end.

I had 2 blockers.

1. I couldn’t configure interactive mode on my local build machine because the machine is on the domain.

2. When I ran in the cloud the machine didn’t have enough memory to deal with the size of my project and would eventually throw an error which was essentially linked to “out of memory”. Something to do with .Net native tool chain

Conclusion I think the disjointedness of Xamarin and UWP is mad.

The ideal solution would be for Automated UWP UI tests to be part of the TestCloud

[JKennedy24]

I am currently at a bit of a stalemate with this issue being blocked by other bugs/issues.

I am going to close this issue for now and hope that Microsoft provide a better solution in future