[🐛 Bug]: find('canvas').click(x: 100, y: 100) not working

What happened?

After upgrading selenium-webdriver gem 4.2.1 -> 4.3.0 the following stopped working:

find('canvas').click(x: 100, y: 100)

i.e. it no longer seems to “draw” on the canvas, so the test cannot continue due to validation error.

We’re using it to draw a signature on a signature_pad canvas.

Any ideas what change I need to make to fix this?

How can we reproduce the issue?

Run that line with 4.3.0, it doesn't "click" the `<canvas>`.

Relevant log output

When I run this line on its own, there's no error but it no longer seems to work.

Operating System

macOS, Ubuntu

Selenium version

4.3.0

What are the browser(s) and version(s) where you see this issue?

Chromium 103

What are the browser driver(s) and version(s) where you see this issue?

selenium-webdriver Ruby with minitest

Are you using Selenium Grid?

No response

[titusfortner]

Yes, there was a breaking change as a result of buggy behavior we couldn’t make consistent. We threw a warning about the change in 4.2 and changed it in 4.3

The History

The old JSON Wire Protocol element origin is from the “upper left corner” of the element. The new W3C specification element origin is from the “in-view center point” of the element.

For moving to an element with an offset, Java just swapped over to the new approach for 4.0 without any warnings or deprecations.

Ruby, Python, .NET all attempted to maintain backward compatibility with 3.x by attempting to calculate the upper left corner of the element and adjusting for it in the implementation.

The problem is that the Selenium implementation can’t parse the “in-view” part of the “in-view center point,” so if the element is partially scrolled out of view, then the origin point is either not actually inside the bounds of the element, or results in an “out of bounds” error.

Since we can’t reliably provide “upper left corner” behavior, it was agreed for all languages to match Java’s implementation making the origin the “in-view center point.”

Code Changes

If the element is always completely inside the viewport, or you use the new scrolling functionality to bring it inside the viewport, you can use the sizing methods to determine the upper left corner using the code we just removed:

Code that was previously being used:

• Python — https://github.com/SeleniumHQ/selenium/commit/64a04f8bf7cd2268e6150688635acbb1ba9044cd
• Ruby — https://github.com/SeleniumHQ/selenium/commit/c6cc197df70b7ec4af3e5881ad68e82d3744b2b7
• .NET — https://github.com/SeleniumHQ/selenium/commit/fbfb491b9f2209a74bea2d0d9301330fc4575ed4

[mihkelal]

Hi, we stumbled on the same problem with our Rails CI failing. After some investigation and comparison between 4.2.1 and 4.3, we discovered this warning when using Selenium 4.2.1:

2022-06-27 xx:xx:xx WARN Selenium moving to an element with offset currently tries to use
the top left corner of the element as the origin; in Selenium 4.3 it will use the in-view
center point of the element as the origin.

So in 4.2.1, in cases with offset the origin is top-left corner, while in 4.3 the origin is in the center. Once we redid the click coordinate calculation to take this into account, specs and CI started working again.