ActionChains in W3C works only 'mouse', want to change it to other types

The problem

Hi, so I’m currently trying to use selenium “ActionChains” with the “appium-python-client” on Windows 10 with the WinAppDriver.

• On appium versions v1.19.1 (+ any prior to v1.20.0), any action chain event seems to work
• However, on v1.20.0+ I keep receiving the following exception message

Note the the same:

• code and version of “appium-python-client” was used
• each version of appium tested also used the same WinAppDriver v1.2-RC (as printed by npm install -g appium@<version>)

Therefore, it seems that the issue stems from the “appium” library and not “WinAppDriver” nor the “appium-python-client”.

Traceback (most recent call last):
  File "C:/p4/perforce/depot/qa/tests/licensing/Fuse/dev/tests/failure.py", line 19, in <module>
    actions.double_click(two).perform()
  File "C:\p4\perforce\depot\qa\tests\licensing\Fuse\dev\venv-379\lib\site-packages\selenium\webdriver\common\action_chains.py", line 80, in perform
    self.w3c_actions.perform()
  File "C:\p4\perforce\depot\qa\tests\licensing\Fuse\dev\venv-379\lib\site-packages\selenium\webdriver\common\actions\action_builder.py", line 76, in perform
    self.driver.execute(Command.W3C_ACTIONS, enc)
  File "C:\p4\perforce\depot\qa\tests\licensing\Fuse\dev\venv-379\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "C:\p4\perforce\depot\qa\tests\licensing\Fuse\dev\venv-379\lib\site-packages\appium\webdriver\errorhandler.py", line 31, in check_response
    raise wde
  File "C:\p4\perforce\depot\qa\tests\licensing\Fuse\dev\venv-379\lib\site-packages\appium\webdriver\errorhandler.py", line 26, in check_response
    super().check_response(response)
  File "C:\p4\perforce\depot\qa\tests\licensing\Fuse\dev\venv-379\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: Currently only pen and touch pointer input source types are supported

Environment

• Python version: Python 3.7.9 (but reproducible on any Python3 version)
• Appium version (or git revision) that exhibits the issue: v.1.20.0+
• Last Appium version that did not exhibit the issue (if applicable): v1.19.1 (any version before v.1.20.0)
• Desktop OS/version used to run Appium: Windows 10 (any version)
• Node.js version: 14.15.4 (should not matter though)
• Npm version: 6.14.10 (should not matter though)

Link to Appium logs

https://gist.github.com/vong0/727ef226073466324ae5631e029fcd13 (logs of both success and fail version)

Code To Reproduce Issue [ Good To Have ]

1. Create a virtual environment and only install Appium-Python-Client==1.0.2
2. Install appium failing version npm install -g appium@1.20.0
3. Run the following code segment and check that it fails
4. Rerun the code segment for a successful version npm install -g appium@1.19.1

from appium import webdriver
from selenium.webdriver.common.action_chains import ActionChains

"Microsoft.WindowsCalculator_8wekyb3d8bbwe!App"

desired_caps = {"app": "Microsoft.WindowsCalculator_8wekyb3d8bbwe!App",
                "platformName": "Windows",
                "deviceName": "WindowsPC",
                "newCommandTimeout": 10000}
app = webdriver.Remote(command_executor='http://127.0.0.1:4723/wd/hub',
                       desired_capabilities=desired_caps)

# Normal actions work
two = app.find_element_by_name("Two")
two.click()

# Any ActionChain event fails
actions = ActionChains(app)
actions.double_click(two).perform()
# actions.move_to_element(two).perform()

[vong0]

@mykola-mokhnach thanks for your help.

For now, I’ve managed to get it working using the {"forceMjsonwp": True} hack @KazuCocoa mentioned.

• This seems to work even on the latest appium@1.20.2

I guess I’ll keep using this until appium@2.0 finally comes out:

• Hopefully by then there is some alternative (either by selenium or who else) for complex mouse actions.

from appium import webdriver
from selenium.webdriver.common.action_chains import ActionChains

desired_caps = {"app": "Microsoft.WindowsCalculator_8wekyb3d8bbwe!App",
                "platformName": "Windows",
                "deviceName": "WindowsPC",
                "newCommandTimeout": 10000,
                "forceMjsonwp": True}   # this allows the action chains written by selenium to work

app = webdriver.Remote(command_executor='http://127.0.0.1:4723/wd/hub',
                       desired_capabilities=desired_caps)

# Normal actions work
two = app.find_element_by_name("Two")
actions = ActionChains(app)
actions.double_click(two).perform()

[mykola-mokhnach]

[debug] [WinAppDriver] {“actions”:[{“type”:“pointer”,“parameters”:{“pointerType”:“mouse”},“id”:“mouse”,“actions”:[{“type”:“pointerMove”,“duration”:250,“x”:0,“y”:0,“origin”:{“ELEMENT”:“42.787924.4.59”,“element-6066-11e4-a52e-4f735466cecf”:“42.787924.4.59”}},{“type”:“pointerDown”,“duration”:0,“button”:0},{“type”:“pointerUp”,“duration”:0,“button”:0},{“type”:“pointerDown”,“duration”:0,“button”:0},{“type”:“pointerUp”,“duration”:0,“button”:0}]},{“type”:“key”,“id”:“key”,“actions”:[{“type”:“pause”,“duration”:0},{“type”:“pause”,“duration”:0},{“type”:“pause”,“duration”:0},{“type”:“pause”,“duration”:0},{“type”:“pause”,“duration”:0}]}]} [WD Proxy] Got response with status 500: {“status”:104,“value”:{“error”:“unsupported operation”,“message”:“Currently only pen and touch pointer input source types are supported”}}

The issue has nothing to do with Appium python lib. The selenium’s Actions implementation sets pointer type to mouse ("parameters":{"pointerType":"mouse"}) by default and that is not supported in WinAppDriver. A possible hack to this:

from selenium.webdriver.common.actions.pointer_input import PointerInput
from selenium.webdriver.common.actions import interaction

actions = ActionChains(app)
actions.w3c_actions.mouse = PointerInput(interaction.POINTER_TOUCH, "touch")
actions.double_click(two).perform()