question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. ItĀ collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

move_to_element_with_offset sometimes moves to wrong location

See original GitHub issue

šŸ› Bug Report

It’s possible for a call like chain.move_to_element_with_offset(driver.find_element(By.TAG_NAME, "canvas"), 0, 0) to miss the canvas entirely and move to a different element.

To Reproduce

HTML:

<!DOCTYPE html>
<html>
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width" />
    <title>Repro</title>
  </head>
  <style type="text/css">
    html * {
      box-sizing: border-box;
    }
  </style>
  <body style="padding: 0; margin: 0;">
    <div style="display: grid; grid-template-rows: 50px; position: absolute; left: 0; top: 0; right: 0; bottom: 0;">
      <div>
        Header
      </div>

      <div style="position: relative;">
        <canvas style="padding: 0px; margin: 0px; border: 0px; position: absolute; top: 0px; left: 0px; width: 500px; height: 500px;">
        </canvas>
      </div>
    </div>
  </body>
</html>

Python script (using selenium==4.0.0a7):

from time import sleep
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver import ActionChains

driver = webdriver.Chrome() # Or Firefox

driver.get("http://localhost:8002/repro.html") # The HTML file above

driver.set_window_size(780, 600) # Broken
#driver.set_window_size(780, 670) # Works

# Log the locations of all the clicks so we can see if things are behaving
driver.execute_script("""
    window.clicks = [];
    document.addEventListener("click", (event) => { 
        window.clicks.push([event.clientX, event.clientY]);
    });
""")

# Almost certainly not necessary, just to demonstrate it's not a race condition
sleep(2)

WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.TAG_NAME, "canvas")))

# The canvas rect is always correct
rect = driver.find_element(By.TAG_NAME, "canvas").rect

chain = ActionChains(driver)
chain.move_to_element_with_offset(driver.find_element(By.TAG_NAME, "canvas"), 0, 0)
chain.click()
chain.move_by_offset(200, 200)
chain.click()
chain.perform()

clicks = driver.execute_script("return window.clicks")
print("Rect", rect)
print("Clicks", clicks)

# Depending on the window size, this is correct or not
assert clicks[0] == [rect['x'], rect['y']]
assert clicks[1] == [clicks[0][0] + 200, clicks[0][1] + 200]

Expected behavior

The script should click the top left of the canvas and then the point (200, 200) from the top left of the canvas.

Instead, the actual points clicked vary.

With a large window size, the behaviour is correct. With smaller window sizes (it seems that the existence of a vertical scrollbar on the page can trigger the issue), different coordinates are clicked. The x-coordinate is always correct, the y-coordinate varies but is always smaller than the real y-coordinate of the canvas, so that the header is clicked and not the canvas.

This can be verified using document.addEventListener("click") as above, and also by observing that the text ā€œHeaderā€ on the page gets selected if a drag operation is used.

In all cases the actual position of the canvas is identical, and Selenium reports the rect of the canvas correctly.

Reproduced with Chrome and Firefox, though they report different y-coordinates.

Environment

OS: Ubuntu 20.04 Browser: Reproduced on Firefox and Chrome Browser version: Firefox 82.0.3, Chrome 86.0.4240.198 Browser Driver version: ChromeDriver 86.0.4240.22, geckodriver 0.28.0 Language Bindings version: Python: selenium==4.0.0a7

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:7 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
Dharin-shahcommented, Mar 15, 2021

Hi @amagee ,

I am able to reproduce the same behaviour with ruby bindings as well. This could very well be a chromedriver issue and not a selenium one, will verify certain combinations (w3c on/off) and try it out

` require ā€œselenium-webdriverā€ require ā€˜test/unit’ extend Test::Unit::Assertions capabilities = { ā€œchromeOptionsā€ => {ā€˜w3c’ => false} }

opts = Selenium::WebDriver::Chrome::Options.new(args: options) driver = Selenium::WebDriver.for(:chrome)

driver.get ā€œhttp://localhost:8000/repro.htmlā€ sleep 1 driver.manage.window.resize_to(780, 600)

driver.execute_script(’ window.clicks = []; document.addEventListener(ā€œclickā€, (event) => { window.clicks.push([event.clientX, event.clientY]); }); ') sleep 1

rect = driver.find_element(:tag_name, ā€œcanvasā€).rect puts rect sleep 1

driver.action.move_to(driver.find_element(:tag_name, ā€œcanvasā€), 0, 0).click.move_by(200, 200).click.perform

clicks = driver.execute_script(ā€œreturn window.clicksā€) puts rect puts clicks[0] assert_equal(clicks[0], [rect[ā€˜x’], rect[ā€˜y’]]) assert_equal(clicks[1], [clicks[0][0] + 200, clicks[0][1] + 200])

sleep 2 driver.quit `

0reactions
github-actions[bot]commented, Jan 26, 2022

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

Read more comments on GitHub >

github_iconTop Results From Across the Web

click at at an arbitrary position in web browser ... - Stack Overflow
You should use move_to_element_with_offset if you want the mouse position to be relative to an element. Otherwise, move_by_offset moves theĀ ...
Read more >
move_to_element_with_offset method - Action Chains in ...
move_to_element_with_offset method is used to move the mouse by an offset of the specified element. Offsets are relative to the top-left cornerĀ ...
Read more >
Re: Issue 2766 in selenium: Chrome - Element is not clickable ...
the element moves when ChromeDriver attempts to click it. This could ... change. This sometimes can be difficult, which is why you might...
Read more >
move by offset selenium | The Search Engine You Control
move_to_element_with_offset method – Action Chains in Selenium Python. PREVIOUS ... Moves the mouse from its current position (or 0,0) by the given offset....
Read more >
How can I tell where the mouse is moving to with selenium?
It can move the cursor and the position change can trigger an event (as with mouseOver), but that's about it. For that reason...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found