Finding text within a bitmap

[rlent]

In the application I am testing, we add objects to a template. I can click on them to interact with the objects, but TestComplete can't see the object. I access the object by clicking on X-Y coordinates within the template. This normally works, but it will often miss and click away from the object, if, for example, the application isn't sized the way it was when the script was captured. The template the object is on is an XtraScrollableControl object.

I don't know if this is possible, but what would be useful is if I could take a bitmap of the template, and return the X-Y location of the desired text. If it returned the coordinates of the center of the "Text Label" string, that would be ideal. That way, I'd always be sure that my clicks would be on target.

I expect that TestComplete itself can't do this; I haven't seen anything in the Region compare which lets you search for text. Are there third party tools that could allow you to search a bitmap for text and return a coordinate? Even if such a thing existed, it would slow down my script, but that still might be worthwhile.

[Colin_McCrae]

Yep.

Can be done.

I use exactly this to find little padlock images (which indicate locks) inside a large single image.

https://support.smartbear.com/testcomplete/docs/testing-with/checkpoints/regions/finding-image-withi...

Another option to look at might be OCR. Which I also use on the same images.

https://support.smartbear.com/testcomplete/docs/testing-with/advanced/ocr/index.html

[tristaanogre · Accepted solution]

Actually, probably easier (and more accurate) than region comparison is the Text Recognition feature of TestComplete.  This is a feature specifically designed to find components, etc., within an application for which there is no easy recognition. It's similar to the OCR item that @Colin_McCrae but uses a bit of a different technology so might work better if OCR does not. One caveat... it only works for desktop applications so this would not apply to web apps or mobile apps.

[Colin_McCrae]

@tristaanogre .... if it contains text.

My canvas objects certainly don't. I'd be surprised if a bitmap image did ....

**EDIT**

I just realised. You're not actually working on a bitmap. You want to take a screenshot of it and search in that.

Right. In that case, yes. Have a look at the text recognition method @tristaanogre suggested. Definitely easier than mucking about with bitmaps and OCR if it works!

[rlent]

Thanks! I use an older version of TC (I'd love it if my company upgraded), but I was able to find the OCR sample. I didn't realize you could not only extract text but it would tell you where it was. This will be very helpful in keeping my scripts working.

[tristaanogre]

Yeah, I'm using it currently to automate a console application. Now, I'm not "clicking" on objects within the console app, but I'm using the text recognition to help me determine if, after I send keystrokes, is what I end up with on screen correct and, if I'm navigating via the keystrokes and menus, do I end up where I should be? It's pretty cool and I'm enjoying playing with it.

[rlent]

I do have one more question. The sample code uses the default Courier font, but I'm not sure how to specify a different font. The application I am testing uses Tahoma. I looked up the OCROptions object, but I couldn't figure out how to specify the font. Do you know what change I need to make to specify the font?

Thanks.

' Call the FindRectByText function with its default options.
' In the list box, the Courier New font is used (it is applied by the OCR
' object by default), so there is no need to specify recognition options.
FindRes = OCRObjList.FindRectByText("E-mail")', OCROptionsList)

[Colin_McCrae]

@rlent - https://support.smartbear.com/testcomplete/docs/reference/program-objects/ocroptions/index.html

@tristaanogre - Yep. I use it in much the same way. It's a touchscreen panel driven by windows CE which uses a map of virtual hotspots behind the canvas to determine whats a button and what isn't. Much like you, I use it for verification of actions on the back end application. OCR isn't reliable enough for heavy duty testing.