The challenge of testing in today’s world
Better techniques for emulating human interaction with apps promises to simplify testing across real and virtual computing environments. One of the biggest challenges in any large-scale test can be the backend work involved in making the test work with the database and application infrastructure. The dawn of screen capture testing tools promises to vastly simplify the testing process by allowing QA personnel to create tests by specifying features on the screen to look for. Furthermore, screen capture tools can also work on native Windows applications, which is beyond the reach of most web based tools.
In human computer interaction, vision plays a dominant role. We look for cues on the screen and interact with these cues to perform tasks. When we go to a website, we look for links and features like text entry boxes for entering or searching data.
History of testing using front end images
Interest in screen reading software agents dates back to the mid-1990s, when Richard Potter at the University of Maryland created a system called Triggers that could interpret basic pixel patterns to control a macro. But these early systems suffered performance delays, as it took several seconds to interpret a screen capture. More recently researchers at MIT have developed the open-source Sikuli for automating and testing applications using a graphical scripting language.
Automai has developed a platform for simulating virtual users’ visual interaction with applications. These simulated users are called “rUsers.” Anything seen on the screen can be acted upon, regardless of which application or system call is involved.
One of the big challenges with traditional testing applications is the need to configure and interact with back end processes. Quality assurance personnel have to be familiar with the various back end interfaces, which can be complicated. In contrast, this new robotic visual testing approach mimics the way people are used to interacting with applications.
Robotic Visual Technology used across industries in variety of use cases:
Testing Oracle Applications running on Citrix
“Robotic tools reduce the need for the DBA because they are image based and intelligent,” said Steve Sutphin, an Oracle EBS consultant. “With traditional testing tools, I have a plugin and I have to enable the middle layer for forms so the testing tool can see the fields.”
Sutphin has used Automai’s AppLoader to help test out a major software deployment across a chain running applications using a virtual desktop application on Citrix terminals. He said that the tool significantly reduced the time and complexity of testing out the new application.
In order to simulate visual interaction, the testing application needs to quickly recognize images on the screen so that they can be interpreted as text, icons, buttons, or data entry fields. To address these needs, AppLoader uses a fast image search component which minimizes the load on the system generating the test. Organizations can use terminal servers or PCs to quickly generate a large number of simulated users running on physical or virtual desktops. A web based controller allows a manager to easily configure and deploy hundreds or even thousands of users remotely.
Robotic tools allow a user with minimal experience to create scripts which interact with applications using screen shots of the graphical interface. The software uses machine vision software techniques to recognize common features like buttons, data fields, and icons. This approach allows a programmer to create scripts that touch multiple applications without any understanding of the underlying APIs and database calls.
Such technology is being used to test and automated a variety of processes in medical, banking, and fast food applications. Sutphin said, “Automai’s robotic engine does not care what client you are running it on. If the image is the same, you could run it on a Mac. You don’t have to worry about the client combinations, or all of the things happening behind the user interface.”
In one deployment, Sutphin was involved in both a hardware and software upgrade at the same time. There were big questions about whether the configuration was right and whether it would load balance correctly.
This created significant challenges for the upgrade. Sutphin said, “Everything gets scrunched at the end, because it takes too long to get everything ready before testing can begin. I needed a tool that was flexible enough to create the scenarios on a test platform and move them to a production platform. The window of opportunity to do these tests is a few weeks where we can use the production hardware, and it is very time sensitive.”
Sutphin had looked at using HP’s LoadRunner, but was concerned about the need for back end DBA support and the need for plugins. With Automai’s AppLoader, they could work with screen shots to highlight important section of the screen to create the test. He noted, “I was trying to have a tool that would eliminate the whole DBA layers of getting the system ready. This time around, we were successful in not having to engage the infrastructure team too much in the testing part.”
Use in Automating Banking Processes
In the financial industry, robotic technology is helping to automate various banking processes. Roger Smith, Chief Technical Officer at Bank of Commerce / Mortgage, began using front end robotic testing tool Scenario Builder to help automate some of their testing needs. But he quickly found that it also helped to automate many of the tedious account lookups required by new financial reporting requirements.
Smith explained, “I am currently working on an audit where we have about 5000 loans on record, and I have a complex series of conditions that the loan has to meet, and if an account does, it falls under the auspices of Regulation B. In order to enter these 5000 lines, it would take 30 key clicks through the app to make the correct entry in the correct field in the database. This would have otherwise taken one person a week.”
The bank had tried to automate this process using SQL programming, but was constrained by the limits of a commercial bank application from Ellie Mae, which made it impossible to programmatically access the data. Since the application helps control critical financial data, it is heavily firewalled off, which makes it difficult to test. Smith noted, “Other testing applications, with a few exceptions, have to be Web based. It seems the world is full of browsers, so everyone has forgotten about client applications. This tool works on any application, unlike other products that are restricted to browsers.”
Smith believes that robotic tools like Automai’s could prove valuable for banks and other institutions that periodically have to check the status of their accounts. He said, “You would think there would be an automated tool for this, but there is not. Maintaining a mortgage database is only going to get more complicated with new rules coming down the pipe. Even now, sometimes we have to make changes every day and other times only on Monday and Friday. Complying for most people means you must do it manually. If you are BOA with 100,000 loans, this is not a three day job.”
Improving testing process from Development to Production environment
The medical industry is also using robotic automation tools to help improve their testing process. For example, the University of California Irvine Medical Center has been using this technology to help improve EMR applications for doctors and staff. The ease of developing new tests allows them to use these tool sets for data conditioning, load and stress testing, and as a monitoring solution for its enterprise applications.
Omer Cheema, Manager, Quality Assurance & Testing at the medical center said, “Our needs are ever-changing and timelines always becoming shorter. Tools like AppLoader require low-overhead and the user-friendly interface reduces overhead and development time by making the scripting process intuitive and straightforward.”
Conclusion
In the long run, this screen scripting technology promises to change the way people think about testing. As Sam Benihya, Technical Director at Automai, explained, “The traditional wisdom is that if you want to test something, you have to understand how it is written and how the data is transported through all the application layers. Robotic testing tools don’t care how an application is written or how the components talk to each other. Emulation of human interaction with apps is the way to approach testing in today’s complex IT environments.”