There are different approaches to finding text in a drop-down list. This article focused on using the Find Text in Image action to find and click on a selection in a drop-down list.
In this article we will focus on using the Find Text in Image action. This action uses optical character recognition (OCR) to scan an image and extract text, allowing it to match on text you’ve asked it to find. Here’s how to use the Find Text in Image feature to select from a drop-down list.
First, if not already open, create a step to open the drop-down list. Then as a child of that step, drag over Find Text in Image under the Images actions.
For the Text to Find property you can enter the text you want it to find; or you can enter a variable name surrounded by % (i.e. %Timerange%). The next property to set is the Text area. You need to tell it to search in the drop-down list for the text and nowhere else on the screen.
Helpful Hint – if the drop-down isn’t open when you click the ellipse for Text area, then before clicking it grab a sticky note, open the drop-down and place the sticky edge along the bottom edge of the drop-down. Now when you need to define the search area, you know how far down to go.
Try to run those steps to see if it can find the text correctly. If it found the text successfully, then set up all the possible items in the drop-down in a CSV file and run the script multiple times to look for each item in the drop-down list. It is possible it will not be able to find all of them successfully without adjusting some of the properties.
If it is having issues finding the right text, check to see what text it is finding in the drop-down. To do this, click the Open OCR output text button that appears on the step.
The Debug OCR window appears.
You can tell how successful your settings are by the correctness of the text in this list to the actual drop-down text.
Another way to see this is via the OCR Configurator. To open the OCR Configurator window, right-click on the step and select Smart Configurator from the menu.
At the bottom of the window you see the image it took of the text area. On the right is the text that it found. In addition to viewing the OCR output, you can make configuration changes and try them out.
Here are the properties that can be changed in the OCR Configurator:
- OCR type – Auto, Single block, Single Line, Single word. The default is Auto and that should work in most cases for a drop-down list with multiple values.
- Language – a list of various languages currently available for use with the OCR technology. If you need to use a language other than English, it may require an additional download.
- Scale Factor – the system will automatically calculate a scale factor to use for reading the text in the image. If it isn't finding text correctly, then you can override the calculated setting with a scale factor between 1 and 10. This defaults to 0 but can be set anywhere from 0-10. Where possible, the value should be between 1 and 6. If the text is small, use a scale factor of 1 to start.
- Match Exact Text – selecting this property indicates the search text must match all of the text found in the image. This is not a good option for locating text in a drop-down list.
- Character Threshold – this is the number of characters that can be off and still make a match. By default, it is set to 0. This means the text to find and the OCR text found must match exactly. If the Find Text in Image action is having issues with a character or two in a phrase you can adjust the number of characters that can be different and still make a match. For example, if you are looking for the text "ID 108764917" and it was having issues correctly recognizing the 9, you could set the Finding Threshold property to 1. This means the text it is looking for and the text it finds can be off by 1 character and still make a successful match. The higher the number, the more characters that can be different and still make a match. Care must be taken when using this property as setting the value too high could result in false positives.
- Include Characters – Small Letters, Capital Letters, Numbers, Special Characters, Other. You can select the characters you want it to find. If you select Other, then enter those characters in the Other text box.
- Exclude Characters – Small Letters, Capital Letters, Numbers, Special Characters, Other. These are the same choices as Include Characters. However, selecting items here will cause the OCR technology to ignore those characters instead of specifically look for them.
- OCR Engine – by default, this is set to 3 which should work best in most cases. Other possible values are 0 (legacy engine only), 1 (neural nets LSTM engine only) and 2 (legacy and LSTM engines).
- Text Area – this is the search area where it searches for the text. If the image in the lower left isn’t the drop-down, click the ellipse button and select the area of the drop-down.
You can adjust the properties and then click the Play button to see the impact of the changes on recognizing the text.
If you altered the settings and it is reading the text correctly, click the Apply configurations to action button to save the changes and close the OCR Properties window.
If your drop-down has a scroll bar, then you would need to combine the Find Text in Image step set to continue on failure with an action that scrolls down the list so it can search through the entire drop-down for the needed text.