Many of our clients have enterprise applications that are accessed 24 hours a day, seven days a week. Their applications may be accessed remotely from all over the world. In such cases, total users could number in the thousands.
When configuring a test plan for our clients we want to establish a realistic number of concurrent users, in other words, the number of users actually in the system, actively using the application at a given time.
Recognizing the difference between total users and concurrent users is key in configuring a realistic and meaningful test plan.
Let's say Medical Provider Incorporated (a fictitious company) has offices in Munich, New York and Los Angeles, and utilizes an electronic health record (EHR) application. Each office has 2,000 employees. First, determine the number of employees that actually access the EHR application under test. For our example, we'll say 1,000 in each office. Of the 1,000, let's say 750 clerical users login to the application at the beginning of their day, add and update records, then log out at the end of the day. The other 250 query users login only when they need to do a query, then logout.
We're beginning to get a range for the number of concurrent users for a load test. But there's one more consideration. Because of the time differences between the three offices, there are times when users from one office will be active, while users from other offices will be home sleeping!
At 9:00 AM when the Los Angeles office opens, it's noon in New York, but it's already 6:00 PM in Munich. If Munich closes at 6:00, the peak load for our application need only include two offices accessing the application at the same time. Let's do some quick math:
750 clerical logged in users x 2 offices = 1,500
250 query logged in users x 2 offices = 500
Do we need to run a 2,000 user load test? Not necessarily. That would assume that 100% of all potential users are logged in at the same time. Anyone who works in an office knows that 100% of the employees are rarely in the office at the same time, let alone sitting at their desks hammering away at an application.
So we'll apply some statistical logic to arrive at a realistic number of concurrent users for a peak load test. To arrive at the number of logged in users, let's say 95% of clerical users and 33% of query users are logged in at once, therefore: (1,500 x .95) = 1425; (500 x .33) = 165; Total logged in users = 1,590.
So our load test will feature 1,590 logged in users. However, we're not quite through. We still need to account for the actual concurrent user activity. Not all of our users will be active in the application simultaneously. So, again, applying some statistical logic, let's say 33% of the clerical users are concurrently adding records and 33% are updating records, and 75% of the query users are performing queries simultaneously. Here's the math:
1425 x .33 = 470 (updating); 1425 x .33 = 470 (adding); 165 x .75 = 124 (queries)
And now, based on the above example, here's the recommended Test Case:
Test Plan 1 – Login 1,590 users. Logged in users, even if dormant, still require system resources so it's important to test with an adequate number of logged in users. Completely login all users before advancing to Test Plan 2. Although not all 1,590 users will be active in the application, the fact that they're logged into the application means that their imposing a load on the resources.
Test Plan 2 – Perform the following workflows simultaneously:
- Workflow 1 - 470 users updating existing records in the application;
- Workflow 2 - 470 users adding new records in the application;
- Workflow 3 – 124 users querying records in the application.
So, in summary, on the surface we might have assumed that a 6,000 user load test would be required for Medical Provider Incorporated to adequately load test their EHR application. However, by differentiating between total users and concurrent users, we find that a realistic and meaningful load test would login 1590 users, of which 1,064 would execute actions within the application. Be sure to analyze your environment carefully to determine your meaningful load test user requirements.