Champion Level 3You're correct — interacting with Canvas UI in web apps can be challenging since the elements inside the <canvas> are not part of the DOM and can't be accessed via standard selectors.
One approach is to use coordinate-based clicking, where you locate the canvas element and simulate clicks at specific (x, y) positions. If your application exposes any JavaScript APIs or metadata about canvas object positions, you can execute JavaScript from your Python script to retrieve those coordinates.
Here's a suggested approach and conceptual Python sample code. You'll need to adjust the logic based on your application's implementation and available JavaScript hooks.
def click_canvas_at_dynamic_coords():
page = Sys.Browser("*").Page("*")
js_code = """
// Example: return coordinates from app-specific logic
return typeof getButtonPosition === 'function'
? getButtonPosition('SaveButton')
: { x: 150, y: 100 };
"""
coords = page.Evaluate(js_code)
if coords and 'x' in coords and 'y' in coords:
canvas = page.NativeWebObject.Find("tagName", "canvas", 10)
if canvas.Exists:
canvas.Click(coords['x'], coords['y'])
else:
Log.Error("Canvas element not found")
else:
Log.Error("Could not retrieve canvas coordinates")
🤖 AI-assisted response
👍 Found it helpful? Click Like
✅ Issue resolved? Click Mark as Solution
Hi,
Thanks for your response.
Yes, as you mentioned, the application provides APIs that return the coordinates of elements inside the Canvas. We are able to retrieve these coordinates using the API and use them to perform click actions.
However, before performing the click, we would like to ensure that we are interacting with the correct element. Could you please suggest how we can validate or confirm that the coordinates correspond to the intended element before clicking?
Your guidance on this would be very helpful.
Champion Level 3That’s a thoughtful question — and you're right to be cautious about clicking within the canvas without confirming the element underneath.
Since Canvas content is not part of the DOM, TestComplete can’t directly inspect it. However, here are two approaches you can consider to validate that you're targeting the correct element:
If your application gives a visual indication (like a highlight or color change) when hovering over a Canvas element, you can simulate a hover on the coordinates and visually compare that region to a known reference.
This uses TestComplete’s built-in image comparison features, such as:
📘 Relevant Documentation:
This option is useful when you cannot programmatically confirm what's at the coordinates but you can detect visual changes.
If your app includes a JavaScript method such as getElementAt(x, y), you can call that directly from your TestComplete script using the Evaluate method.
This allows you to validate what element is at the given coordinates before clicking, using your application’s own internal logic.
📘 Relevant Documentation:
Example (conceptual):
isCorrect = page.Evaluate("return getElementAt(150, 100) === 'SaveButton';")
if isCorrect:
canvas.Click(150, 100)
This approach depends on your application exposing such APIs.
🤖 AI-assisted response
👍 Found it helpful? Click Like
✅ Issue resolved? Click Mark as Solution
Champion Level 3Another avenue you might consider is using TestComplete’s VisualTest integration.
This feature is designed for scenarios like yours where traditional object recognition isn’t possible — such as Canvas-based UIs. Visual Testing allows you to:
🔹 Note: Visual Testing is for validation only — it can serve as a reliable pre-check to confirm the correct element is displayed before interacting through other means.
For more details:
📘 VisualTest Integration – Official Docs
🤖 AI-assisted response
👍 Found it helpful? Click Like
✅ Issue resolved? Click Mark as Solution