Forum Discussion

thiyagu's avatar
thiyagu
Occasional Contributor
2 months ago

How to Click the Canvas object using test complete web module

Hi Team,
I’m starting a new web test automation initiative, and our web application is primarily built using Canvas UI. I’m facing challenges in accessing and interacting with objects inside the Canvas, as they are not standard HTML elements.


Additionally, the Canvas objects are dynamically generated and frequently change, which makes it difficult to consistently identify and click the correct object.
Could you please suggest a reliable solution for automating interactions with Canvas elements? Also, it would be helpful if you could share a sample Python script demonstrating how to handle such scenarios.


Thanks & Regards,
Thiyagarajan

 

5 Replies

  • Hassan_Ballan's avatar
    Hassan_Ballan
    Icon for Champion Level 3 rankChampion Level 3
    2 months ago

    You're correct — interacting with Canvas UI in web apps can be challenging since the elements inside the <canvas> are not part of the DOM and can't be accessed via standard selectors.

    One approach is to use coordinate-based clicking, where you locate the canvas element and simulate clicks at specific (x, y) positions. If your application exposes any JavaScript APIs or metadata about canvas object positions, you can execute JavaScript from your Python script to retrieve those coordinates.

    Here's a suggested approach and conceptual Python sample code. You'll need to adjust the logic based on your application's implementation and available JavaScript hooks.

    def click_canvas_at_dynamic_coords():
    page = Sys.Browser("*").Page("*")
    
    js_code = """
        // Example: return coordinates from app-specific logic
        return typeof getButtonPosition === 'function' 
            ? getButtonPosition('SaveButton') 
            : { x: 150, y: 100 };
    """
    coords = page.Evaluate(js_code)
    
    if coords and 'x' in coords and 'y' in coords:
        canvas = page.NativeWebObject.Find("tagName", "canvas", 10)
        if canvas.Exists:
            canvas.Click(coords['x'], coords['y'])
        else:
            Log.Error("Canvas element not found")
    else:
        Log.Error("Could not retrieve canvas coordinates")

    🤖 AI-assisted response
    👍 Found it helpful? Click Like
    ✅ Issue resolved? Click Mark as Solution

    • thiyagu's avatar
      thiyagu
      Occasional Contributor
      2 months ago

      Hi,

      Thanks for your response.

      Yes, as you mentioned, the application provides APIs that return the coordinates of elements inside the Canvas. We are able to retrieve these coordinates using the API and use them to perform click actions.

      However, before performing the click, we would like to ensure that we are interacting with the correct element. Could you please suggest how we can validate or confirm that the coordinates correspond to the intended element before clicking?

      Your guidance on this would be very helpful.



       

       

      • Hassan_Ballan's avatar
        Hassan_Ballan
        Icon for Champion Level 3 rankChampion Level 3
        2 months ago

        That’s a thoughtful question — and you're right to be cautious about clicking within the canvas without confirming the element underneath.

        Since Canvas content is not part of the DOM, TestComplete can’t directly inspect it. However, here are two approaches you can consider to validate that you're targeting the correct element:

        🔹 Option 1: Visual Validation Using Image Comparison

        If your application gives a visual indication (like a highlight or color change) when hovering over a Canvas element, you can simulate a hover on the coordinates and visually compare that region to a known reference.

        This uses TestComplete’s built-in image comparison features, such as:

        • Picture – to capture a snapshot of a region on the canvas.
        • Region.Check – to compare it against a stored baseline image.

         

        📘 Relevant Documentation:

        This option is useful when you cannot programmatically confirm what's at the coordinates but you can detect visual changes.

        🔹 Option 2: JavaScript Validation (If Your App Provides API)

        If your app includes a JavaScript method such as getElementAt(x, y), you can call that directly from your TestComplete script using the Evaluate method.

        This allows you to validate what element is at the given coordinates before clicking, using your application’s own internal logic.

        📘 Relevant Documentation:

        Example (conceptual):

        isCorrect = page.Evaluate("return getElementAt(150, 100) === 'SaveButton';")
if isCorrect:
    canvas.Click(150, 100)

        This approach depends on your application exposing such APIs.

        🤖 AI-assisted response
        👍 Found it helpful? Click Like
        ✅ Issue resolved? Click Mark as Solution

  • rraghvani's avatar
    rraghvani
    Icon for Champion Level 3 rankChampion Level 3
    2 months ago

    Hi Thiyagarajan. I suggest you check Supported Technologies and Applications to see if Canvas UI is actually supported by TestComplete. By the looks of it, it's not supported. So you will have an extremely difficult time interact with the UI objects, even the browsers' DevTools is not able to identify any of the UI objects in Canvas UI Demo!