Async JS execution on iOS always timeouts

The problem

Trying to uppgrade a test suite from iOS 10.3 to iOS 12.x, my team need the bugfixes from Appium 1.12.1, but newest Appium versions consistently produce timeouts on async JS executions. 😥

When running Appium 1.13.0 (iOS 12.4 / iPhone 7 Simulator) locally with a small repro case, I get the error: “Encountered internal error running command: Error: Did not get any response after 300s”

NB: This is very similar to #11925 but I get different logs and Selenium seems to never receive any response from the Appium server, making tests consistently fail after the command timeout.

Environment

• Appium version (or git revision) that exhibits the issue: 1.13.0
• Last Appium version that did not exhibit the issue (if applicable): 1.9.1
• Desktop OS/version used to run Appium: OS X Mojave
• Node.js version (unless using Appium.app|exe): (10.16.1)
• Npm or Yarn package manager: -
• Mobile platform/version under test: iOS 12.4
• Real device or emulator/simulator: iPhone 7 Simulator
• Appium CLI or Appium.app|exe: Appium.app
• C# Selenium.WebDriver: 3.141.0
• C# Appium.WebDriver: 4.0.0.6-beta

Details

Please note that the exact version of iOS / OS X may not matter that much. We were testing iOS 12.2 on Saucelabs with their latest Appium version (1.13.0), and the details above are just from the repro case done on a fresh OS X install. They do not necessarily represent SauceLabs setup.

The same code, run on iOS 10.3 / Appium 1.9.1, via SauceLabs, was working perfectly fine.

(NB: When running via SauceLabs, we use the “execute_async” route instead of “execute/async” with local Appium, but the command still seems to end up with a timeout.)

From what I could see in the logs (the part related to the execute/async command is provided below), this seems to be a failure within Appium itself.

Appium logs

[info] [HTTP] --> POST /wd/hub/session/a6a1d0c2-0cdb-4803-8081-56f9d9f6148a/execute/async
[info] [HTTP] {"script":"var done = arguments[arguments.length - 1];\nvar duration = arguments[0];\n\nwindow.setTimeout(() => done(true), duration);\n","args":[2000]}
[debug] [W3C (a6a1d0c2)] Calling AppiumDriver.executeAsync() with args: ["var done = arguments[arguments.length - 1];\nvar duration = arguments[0];\n\nwindow.setTimeout(() => done(true), duration);\n",[2000],"a6a1d0c2-0cdb-4803-8081-56f9d9f6148a"]
[debug] [XCUITest] Executing command 'executeAsync'
[debug] [RemoteDebugger] Sending javascript command window.location.href
[debug] [RemoteDebugger] Sending '_rpc_forwardSocketData:' message to remote debugger (id: 10)
[debug] [RemoteDebugger] Received receipt for message '10'
[debug] [RemoteDebugger] Found data handler for response
[debug] [RemoteDebugger] Received data response from socket send: 'about:blank'
[debug] [RemoteDebugger] Original command: sendJSCommand
[debug] [iOS] Response url for executeAsync: http://0.0.0.0:4723/wd/hub/session/a6a1d0c2-0cdb-4803-8081-56f9d9f6148a/receive_async_response
[debug] [RemoteDebugger] Executing 'execute_async_script' atom in default context
[debug] [RemoteDebugger] Sending javascript command (function(){return function(){var e=this;
[debug] [RemoteDebugger] funct...
[debug] [RemoteDebugger] Sending '_rpc_forwardSocketData:' message to remote debugger (id: 11)
[debug] [RemoteDebugger] Received receipt for message '11'
[debug] [RemoteDebugger] Found data handler for response
[debug] [RemoteDebugger] Received data response from socket send: '{"result":{"type":"undefined"}}'
[debug] [RemoteDebugger] Original command: sendJSCommand[debug] [RemoteDebugger] Received page change notice for app 'PID:7364' but the listing has not changed. Ignoring.
[debug] [RemoteDebugger] Received page change notice for app 'PID:7364' but the listing has not changed. Ignoring.[info] [HTTP] --> OPTIONS /wd/hub/session/a6a1d0c2-0cdb-4803-8081-56f9d9f6148a/receive_async_response
[info] [HTTP] {}
[debug] [HTTP] No route found. Setting content type to 'text/plain'
[info] [HTTP] <-- OPTIONS /wd/hub/session/a6a1d0c2-0cdb-4803-8081-56f9d9f6148a/receive_async_response 404 1 ms - 117
[info] [HTTP] [debug] [W3C (9e3de4f6)] Encountered internal error running command: Error: Did not get any response after 300s
[debug] [W3C (9e3de4f6)]     at XCUITestDriver.waitForAtom (/Applications/Appium.app/Contents/Resources/app/node_modules/appium/node_modules/appium-xcuitest-driver/lib/commands/web.js:285:13)[info] [HTTP] <-- POST /wd/hub/session/a6a1d0c2-0cdb-4803-8081-56f9d9f6148a/execute/async - - ms - -

Code To Reproduce Issue

The following .NET Core 2.2 / C# program can be used to produce the issue quite consistently. ⚠️ On our test machine, it did require a few manual runs for Appium to properly warmup, and getting to the point where the async command was actually executed.

AppiumAsyncScriptTest.csproj

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp2.2</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Appium.WebDriver" Version="4.0.0.6-beta" />
    <PackageReference Include="Selenium.WebDriver" Version="3.141.0" />
    <PackageReference Include="Serilog" Version="2.8.0" />
    <PackageReference Include="Serilog.Sinks.Console" Version="3.1.1" />
  </ItemGroup>

</Project>

Program.cs

using System;
using OpenQA.Selenium.Appium;
using OpenQA.Selenium.Appium.Enums;
using OpenQA.Selenium.Appium.iOS;
using OpenQA.Selenium.Remote;
using Serilog;

namespace AppiumAsyncScriptTest
{
    internal static class Program
    {
        private static void Main()
        {
            const string appiumUrl = "http://127.0.0.1:4723/wd/hub";
            const string asyncWaitScript = @"var done = arguments[arguments.length - 1];
var duration = arguments[0];

window.setTimeout(() => done(true), duration);
";

            var logger = new LoggerConfiguration()
                .WriteTo.Console()
                .CreateLogger();

            var options = new AppiumOptions();

            options.AddAdditionalCapability(MobileCapabilityType.BrowserName, "safari");
            options.AddAdditionalCapability(MobileCapabilityType.PlatformName, "iOS");
            options.AddAdditionalCapability(MobileCapabilityType.PlatformVersion, "12.4");

            options.AddAdditionalCapability(MobileCapabilityType.DeviceName, "iPhone 7");
            options.AddAdditionalCapability(IOSMobileCapabilityType.ScreenshotWaitTimeout, 30);
            options.AddAdditionalCapability("screenshotQuality", 0);
            options.AddAdditionalCapability("showIOSLog", false);

            try
            {
                using (var executor = new HttpCommandExecutor(new Uri(appiumUrl, UriKind.Absolute), new TimeSpan(30 * TimeSpan.TicksPerSecond)))
                {
                    executor.SendingRemoteHttpRequest += (_, e) =>
                    {
                        string url = e.Request.RequestUri.AbsoluteUri;

                        // Use a longer timeout for session creation.
                        if (url.EndsWith("/session", StringComparison.Ordinal))
                        {
                            e.Request.Timeout = 180_000;
                        }

                        if (!string.Equals(e.Request.Method, "POST", StringComparison.OrdinalIgnoreCase))
                        {
                            logger.Information("Sending {Method} request to {Url}.", e.Request.Method, e.Request.RequestUri.AbsoluteUri);
                        }
                        else
                        {
                            logger.Information("Sending {Method} request to {Url} with payload:\n{Payload}", e.Request.Method, e.Request.RequestUri.AbsoluteUri, e.RequestBody);
                        }
                    };

                    using (var driver = new IOSDriver<AppiumWebElement>(executor, options))
                    {
                        logger.Information("IOSDriver was successfully created.");

                        var timeouts = driver.Manage().Timeouts();

                        timeouts.PageLoad = new TimeSpan(30 * TimeSpan.TicksPerSecond);
                        timeouts.AsynchronousJavaScript = new TimeSpan(30 * TimeSpan.TicksPerSecond);

                        driver.Navigate().GoToUrl("about:blank");

                        driver.ExecuteAsyncScript(asyncWaitScript, 2000);

                        driver.GetScreenshot();

                        logger.Information("All commands executed sucessfully.");
                    }
                }
            }
            catch (Exception ex)
            {
                logger.Error(ex, "An exception occured.");
            }
        }
    }
}

[imurchie]

https://github.com/appium/appium-base-driver/pull/342

[mykola-mokhnach]

The fix has been published to appium@beta