Forum Discussion

thiyagu's avatar
thiyagu
Occasional Contributor
3 months ago
Solved

How to Click the Canvas object using test complete web module

Hi Team, I’m starting a new web test automation initiative, and our web application is primarily built using Canvas UI. I’m facing challenges in accessing and interacting with objects inside the Can...
  • rraghvani's avatar
    rraghvani
    3 months ago

    Hi Thiyagarajan. I suggest you check Supported Technologies and Applications to see if Canvas UI is actually supported by TestComplete. By the looks of it, it's not supported. So you will have an extremely difficult time interact with the UI objects, even the browsers' DevTools is not able to identify any of the UI objects in Canvas UI Demo!

Hassan_Ballan's avatar
Hassan_Ballan
Icon for Champion Level 3 rankChampion Level 3
3 months ago

You're correct — interacting with Canvas UI in web apps can be challenging since the elements inside the <canvas> are not part of the DOM and can't be accessed via standard selectors.

One approach is to use coordinate-based clicking, where you locate the canvas element and simulate clicks at specific (x, y) positions. If your application exposes any JavaScript APIs or metadata about canvas object positions, you can execute JavaScript from your Python script to retrieve those coordinates.

Here's a suggested approach and conceptual Python sample code. You'll need to adjust the logic based on your application's implementation and available JavaScript hooks.

def click_canvas_at_dynamic_coords():
    page = Sys.Browser("*").Page("*")
    
    js_code = """
        // Example: return coordinates from app-specific logic
        return typeof getButtonPosition === 'function' 
            ? getButtonPosition('SaveButton') 
            : { x: 150, y: 100 };
    """
    coords = page.Evaluate(js_code)
    
    if coords and 'x' in coords and 'y' in coords:
        canvas = page.NativeWebObject.Find("tagName", "canvas", 10)
        if canvas.Exists:
            canvas.Click(coords['x'], coords['y'])
        else:
            Log.Error("Canvas element not found")
    else:
        Log.Error("Could not retrieve canvas coordinates")

🤖 AI-assisted response
👍 Found it helpful? Click Like
✅ Issue resolved? Click Mark as Solution

thiyagu's avatar
thiyagu
Occasional Contributor
3 months ago

Hi,

Thanks for your response.

Yes, as you mentioned, the application provides APIs that return the coordinates of elements inside the Canvas. We are able to retrieve these coordinates using the API and use them to perform click actions.

However, before performing the click, we would like to ensure that we are interacting with the correct element. Could you please suggest how we can validate or confirm that the coordinates correspond to the intended element before clicking?

Your guidance on this would be very helpful.



 

 

  • Hassan_Ballan's avatar
    Hassan_Ballan
    Icon for Champion Level 3 rankChampion Level 3
    3 months ago

    That’s a thoughtful question — and you're right to be cautious about clicking within the canvas without confirming the element underneath.

    Since Canvas content is not part of the DOM, TestComplete can’t directly inspect it. However, here are two approaches you can consider to validate that you're targeting the correct element:

    🔹 Option 1: Visual Validation Using Image Comparison

    If your application gives a visual indication (like a highlight or color change) when hovering over a Canvas element, you can simulate a hover on the coordinates and visually compare that region to a known reference.

    This uses TestComplete’s built-in image comparison features, such as:

    • Picture – to capture a snapshot of a region on the canvas.
    • Region.Check – to compare it against a stored baseline image.

     

    📘 Relevant Documentation:

    This option is useful when you cannot programmatically confirm what's at the coordinates but you can detect visual changes.

    🔹 Option 2: JavaScript Validation (If Your App Provides API)

    If your app includes a JavaScript method such as getElementAt(x, y), you can call that directly from your TestComplete script using the Evaluate method.

    This allows you to validate what element is at the given coordinates before clicking, using your application’s own internal logic.

    📘 Relevant Documentation:

    Example (conceptual):

    isCorrect = page.Evaluate("return getElementAt(150, 100) === 'SaveButton';")
if isCorrect:
    canvas.Click(150, 100)

    This approach depends on your application exposing such APIs.

    🤖 AI-assisted response
    👍 Found it helpful? Click Like
    ✅ Issue resolved? Click Mark as Solution

    • Hassan_Ballan's avatar
      Hassan_Ballan
      Icon for Champion Level 3 rankChampion Level 3
      3 months ago

      Another avenue you might consider is using TestComplete’s VisualTest integration.

      This feature is designed for scenarios like yours where traditional object recognition isn’t possible — such as Canvas-based UIs. Visual Testing allows you to:

      • Validate that specific areas of the application look exactly as expected.
      • Detect any unexpected visual changes in the rendered canvas content.
      • Use full-screen or region-level comparisons (including AI-assisted visual diffs).

      🔹 Note: Visual Testing is for validation only — it can serve as a reliable pre-check to confirm the correct element is displayed before interacting through other means.

      For more details:
      📘 VisualTest Integration – Official Docs

      🤖 AI-assisted response
      👍 Found it helpful? Click Like
      ✅ Issue resolved? Click Mark as Solution