How to fix image failures

Scripts will fail if the robot cannot find the image it is supposed to find. Automai's Smart Image Configurator will help you make adjustments to prevent this from happening.

There are several ways to access the Smart Image Configurator tool:

  • When the Execution Report appears noting the failure, there is a button -
  • When an image action has a failure, there is an icon on the step - .
  • If the icon doesn't appear, right-click on the action and choose Smart Configurator from the pop-up menu.

Once you've accessed Smart Image Configurator, here is how to use it to try and improve the robot's ability to successfully find the image.

NOTE - the screenshots that appear in Smart Image Configurator are captured automatically by executing scenario(s) in ScenarioBuilder or Automai's testing tools.

Get Suggestions from the Application

The first action is to test out suggestions from the application. To do this:

  1. Click the second icon from the left in the Image toolbar (). This will adjust the Tolerance and Search Area and test the settings. If the failure screenshots in the Screenshots Gallery have green boxes around them, then these settings helped ScenarioBuilder successfully find the image in the failure screenshots.
  2. To save the changes, click the Apply changes icon in the Image toolbar ().

If you have images in the Screenshots Gallery panel with red boxes around them, it couldn’t find the image in those. If they are not valid screenshots (pop-up in the way, on the wrong screen, etc.) then you can right-click on the image and select Remove. If the image should have been found in the screenshot and it wasn’t, then you’ll need to continue to adjust settings.

Captured Image Don't Match Onscreen Image

Look at the images in the Images panel on the left and then look at the screenshots. Are there differences in how the image appears? For example, hovering over a button can change the color of the button, etc. If that is the issue, then capturing a new image is the next change to try. To do this:

  1. Select the failure screenshot from the Screenshots Gallery that has the image you want to capture.
  2. Click the Capture image from screenshot icon in the Image toolbar (). It will open a new window.
  3. If the image you want to capture is already inside of the purple box, then press Enter to recapture the image. If the purple box does NOT include the image you want, then use the mouse to click and drag over the image you want to capture. You’ll see the newly captured image in the Images panel.
  4. Click the Start Searching icon in the Images toolbar (). If all the images in the Screenshots Gallery panel have green boxes, then it was able to successfully find the image in all of them.
  5. To save the changes, click the Apply changes icon in the Image toolbar ().

If it still did not successfully find the image in all the valid screenshots in the Screenshots Gallery panel, then you’ll need to continue to adjust settings and do a search to see if it can find them.

Adjusting Tolerance and/or Contrast

Try adjusting the Tolerance and Contrast. Under Properties in the panel on the left, the first box is Tolerance and the second is Contrast. Tolerance determine how closely the pixels must match. A Tolerance of 0 indicates it must be a pixel for pixel match. A Tolerance of 10 indicates 90% of the pixels must match. Avoid setting Tolerance too high as doing so can cause false positives. Setting Contrast will convert the image to grayscale as well as the screen before looking for a match.

Remember to perform a search after each property change to see the results of the search. Once it can find all the images successfully, click Apply changes before closing Smart Image Configurator.

For further assistance, there is a video covering the use of Smart Image Configurator that demonstrates the above changes.