How to create a Test plan in AppVerify Manager?

To create a new Test plan, simply select “Test Plans” option from navigation and click on “+Test plan” button.

Screenshot_48-1

Once you click on it, “New test plan“ form will populate starting with the basic information screen.

Screenshot_46-1

The above image shows how the Test plan form looks upon opening.

Test plan form has 4 steps: 1. Basic information 2. Process variables 3. Test cases 4. Expected outcome

The steps that have not been visited yet will be disabled since each step is dependent on the previous steps. For instance, if you have reached to step 3, steps 1 and 2 will be available but step 4 will still be disabled.

Tip: During the creation process if any change is made in the Test plan or Test case (add, delete, edit, etc), all the updates are saved automatically.

(*) indicates this field is required in order to proceed.

Below we will walk through each section of the form and provide an image with sample data to demonstrate how to create a Test plan:

1. Basic information

Test plan name *: A unique name used to identify the Test plan.

Project *: Select the project that contains the processes you want to include in the Test plan.

Description: Enter an optional description of the contents of the Test plan.

Stop on error: During a Test run, you may need to interrupt the entire Test run after an error occurs. In order to do this, use the Test plan’s “Stop on error” option. By default, this option is disabled so that whenever an error is encountered, the Manager simply continues running the test until it gets another command. When the test is over, you will be able to view all the errors that occurred during the Test run in the Test run report.

To enable this option, check the "Stop on error" option.

When the option is enabled and an error occurs, the Manager will continue the Test run until the current process is over and then it will stop the Test run.

* Do not confuse this option with the “Stop on error” option for the processes. For more information, see Test cases section.

Add initializing or finalizing process: If checked , optional fields will be added to the form to select initializing and/or finalizing processes.

Initialize test plan with: Select a process that will run before any processes in the Test run.

Finalize test plan with: Select a process that will run after the rTester finishes executing the processes in the Test run.

* Initialization and finalization processes are similar to other processes. The purpose of these processes is to perform one-time initialization routines upon Test run starts (such as launching Web Browser, loading website, login to application, etc.) and finalization routines (properly close application, cleanup, and release memory or other resources) upon Test run finishes.

Click "Add process" to open “Add process” form.

Select group: Select a group from the groups' list. To learn more about groups, see this article

Select process: Select a process that you want to execute. You can choose multiple processes.

Screenshot_50-2

Select robot group: Select a robot/robot group to be used to execute the process in your Test plan. There are 3 options:

  • All Available Robots: This is the default selection. All available robots will run the Test cases of the selected process in parallel. 
  • A Robot Group: All available robots associated with the selected robot group will run the Test cases of the selected process in parallel. To learn how to manage robot groups, see this article.
  • A specific Robot: Only the selected robot will run the Test cases of the selected process.

Note: This option is only visible for administrators. To learn how to manage users, see this article.

If there are multiple processes in a Test plan, the processes are executed sequentially. The selected robots will play the first process. Once the execution of all Test cases in that process is done, the selected robots will play the Test cases of the next process, and so on. This continues until all processes of the Test plan are executed.

Note: If you select a different robot or robot group for subsequent processes, the execution will not start for that process until the preceding process is done.

To learn how to manage robot groups, see this article.

 

Once you are done, click “Submit”. The form will close.

 

* If you need to close the Add process form prior to submitting, click the X in the top right-hand corner. The selected processes will automatically appear here.

Screenshot_49-1

* The Manager gives you the option to change the order of processes by clicking Up/Down icons.

* The processes can be deleted by clickingScreenshot_85button.

* If you need to change the robot/robot groups that will be used to execute the process, you can open the Robot group drop-down for the process and make the desired selection.

Screenshot_79

Global variables: If any of the processes you added contain global variables, they appear below the description. Global variables can be accessible from any process. Unlike script variables, global variables retain their value during executing process in different iterations. Therefore, the values of global variables will be the same for all the test cases within the process.

To modify the global variables values, click Manage process variables icon.

There are two options for the source of the global variables:

  • Static – the variables are set in this form
    Click on each variable value you want to modify, make the needed changes, and then press Enter to save the updated data. 
  • Dynamic – the variables are set in an external CSV or Excel file. The values from the external file are read each time a Test run is generated (either manually or via a schedule); allowing the data to be updated outside of the AppVerify Manager as needed. To learn more about dynamic variable files, see this article

* If you are using dynamic variables, the variables that appear in the form are read from the associated file and are read-only.

Screenshot_68-1

 

Note: The “Generate all test cases” button will generate all Test cases for selected processes automatically.

After adding all the necessary data, click “Next” button to go to the desired process to continue.

2. Process variables

Here you can see the processes default variables along with their values.

Screenshot_70

Variables are attributes (scenario parameters) of the tested application used to generate Test cases. They can be used to create multiple Test cases from a process to support various configurations. Create a Test case to define the set of conditions, actions, expected results, and other criteria used to determine if an application works properly and meets its specified requirements.

Use “Selected process” drop down list to select a process and view the corresponding variables.

By default, process variables are static. Click Manage process variables icon to change variables type. To learn more about dynamic variable files, see this article

Tip: If you are using dynamic variables for the selected process, the “Process variables” step will be populated from the associated variable file. If it contains multiple rows, they will show up here. The data that appears is read-only. If changes are needed, they must be made in the dynamic file.

In the “Process variables” step, there is only one state (Default) for each process. The default variable data is populated from the defined variables or the first row of the CSV file attached to the scenario in ScenarioBuilder.

* You can edit the default variable but you cannot delete it.

 

* To add new variable state to the Test plan, click “+New” button.

Screenshot_72

Specify the state of the variable. The variable state is part of the Test case name. Each variable in your process may have multiple data states.

Click green "+" icon to set your desired value for the variable. Once done, press Enter.

Tip: For date and time variable values, the default values are always based on current date and time. However, you can add or subtract values from the individual month, date, year, hour, minute or second. For instance, [MM+1/dd/yy], [MM/dd-1/yyyy], [hh-1:mm:ss].

❖ Do not remove the beginning and ending brackets from the pre-defined values; multiple modifications (addition, subtraction) are allowed per variable value.

* If there is no value ( + ), the Manager will use the default values when generating Test cases.

* The variables can have empty values. Once you put empty value, it will be displayed as “empty value”.

av_new_test_plan_18

 

* To edit a variable state, click on it, enter your desired value and press Enter to update.

Screenshot_71

 

* The variables can be deleted by clickingScreenshot_85button.

After adding all the necessary variables and data, click “Generate test cases” button. This will generate all Test cases based on the process variables and the Test plan will move to the next step.

If you’ve already generated Test case, once you click “Generate test cases”, a message pop up will be displayed;

av_new_test_plan_20

To generate new test cases, click on “Yes, proceed” button.

* Click “View test cases” button to see the existing Test cases.

3. Test cases

Here you can see the list of all Test cases that have been created in the previous step. The image below is for a Test plan that is using static process variables. If using dynamic process variables, the “+New test case” button and the icon to delete Test cases do not appear. Adding and removing Test cases is done in the associated variables file.

From this list, you can go through each Test case and edit or delete them.

Screenshot_73

* To create a new Test case and add it to the Test plan, click “+New test case” button.

av_new_test_plan_22

Next, enter a name for the Test case by clicking “Click to edit”.

Then, click “Select” by hovering over each Test case.

Here you can modify the variables values.

av_new_test_plan_23

* To edit a Test case, click on each variable value you want to modify and make the changes.

av_new_test_plan_24

Note: Any changes in this section have no effect on the previous section (Process Variables).

* The variables values cannot be deleted on this section. You may delete them on “Process Variables” section.

* To delete a Test case, clickScreenshot_85 button.

Stop on error: If the option in this step is enabled and an error occurs, the Manager will stop running the process but it will continue playing the other processes of the Test run.

When all the Test cases are ready, the Test plan can move to its final step.

Click on one of the expected outcomes to go to “Expected outcome” section.

Screenshot_75

4. Expected outcome

In this section, you can see the processes steps (actions) which will be performed in an application in sequential order.

Here you can define the validation points and expected results for each Test case which describe the outcome of performing that Test case and criteria to determine if the Test case passes or fails.

av_new_test_plan_26

Before continuing, let’s first take a look at the meaning of the following terms:

Action: An action can be thought of as the act of doing after a specific Test case occurred. The action cares about the results and it will act based on its value which is defined by one of the following: Find Image, Desktop Screenshot or Component Cleanup. In other words, an action specifies what happened after a Test case is conducted and it is used to set the Validation point.

Validation point: A validation point verifies the outcome of the Test case. Usually, a Test case has several possible outcomes. For example, login might behave differently depending on whether the username and password are correct, incorrect, too long, too short or just non-existent. The action is the act of responding and the validation point verifies the outcome when using valid, invalid, too long, too short or just non-existent usernames and passwords.

Think of the manual checkpoints that a human would look for and make sure that each validation point is positioned in the script where the behavior is expected.

* It is important to provide complete steps and expected results to avoid ambiguity during testing.

IMPORTANT: To be clear, Test cases do NOT require validation points. It is possible to craft a scenario in ScenarioBuilder that can validate several Test cases from one process without the need to add validation points to them in the Manager.

For example, you have a time tracking application and you created a scenario in ScenarioBuilder to enter time for the week. After entry, the time tracking system automatically calculates overtime, double-time, etc. You have steps at the end of the scenario that validates that the system calculated the time correctly. In the Manager, all you need to do is have multiple Test cases that enter a variety of times for the employee along with the expected calculated values. When the Test plan is executed, each Test case will enter the time data and then validate that the system calculated data is correct. In this example, you would not need to add any validation points to determine if the Test case passed or failed.

However, here is an example of when you would need to add validation points to Test cases. You have a patient manager application and have a scenario that adds a patient. You need additional Test cases to verify that the system correctly flags empty required fields. The notification for a missing required field appears as soon as you exit a required field without entering data, therefore that is the point where you need to validate the system is providing proper notification. There is no need to continue with adding a patient at that point, so you want to exit from the scenario.

Tip: When a Test case with one or more validation points is executed, once the last validation point is reached, the remainder of the steps following the validation point are not executed. If there are steps that need to be executed after the validation point to prepare the application for the next Test case, you will need to add a component to the scenario in ScenarioBuilder that performs the needed steps. Then add a “Component Cleanup” validation point that executes the component.

If the selected Test case needs one or more validation points, follow these steps to define an expected outcome:

Test case: Select the Test case you want to define an expected result for it.

Outcome name: Enter a unique name to identify the Test case result.

Select validation points: Click to add a new validation point or select an existing validation point.

Validation point name: Enter a name for the validation point.

Actions: There are 3 options that can be selected as an action:

• Find Image

• Desktop Screenshot

• Component Cleanup

av_new_test_plan_27

1. When you choose “Find Image” action, a window will be popped up so that you can choose images.

av_new_test_plan_28

In order to upload a new image, click on the “Upload new image” button. The “File Upload” dialog box will appear over the active window. Select the desired image file, then click “Open”.

av_new_test_plan_29

The image will be added to the “Created actions” section.

* Remember only images with “BMP” format and less than 2 MB can be uploaded.

If the image is already present under the process images, you can choose your desired image and click “Select”.

The image will be automatically added to the “Created actions” section.

av_new_test_plan_30

2. When you choose “Desktop Screenshot” action, the value (Desktop Screenshot) will be automatically added to the “Created actions” section.

av_new_test_plan_31

3. You can choose “Component Cleanup” action if an available component does exist in the process. Select your component from the drop down list.

av_new_test_plan_32

Once a component is selected, it will automatically be added to the “Created actions” section

av_new_test_plan_35

* We recommend you to choose the “Component Cleanup” action as the last action.

* You can add many actions and multiple validation points for each Test case.

* The validation points can also be added inside the sub-scenarios or components.

* To reset actions, click “Reset action” button.

After adding the actions, click “Add Validation Point” button.

Once you click on it, “Insert below” button will activate.

av_new_test_plan_34

Validation points can be inserted anywhere in the process, simply click to insert the validation point to the desired position.

av_new_test_plan_37

* To edit the Validation point, just select your desired Test case and then validation point.

Once it is selected, you can edit the fields of the validation point in the left side of the section.

av_new_test_plan_36

When hovering over a validation point, a “Reposition” button appears next to X button. By clicking on it, you will activate “Insert below” button again to select after which action to insert your validation point.

av_new_test_plan_37

* To delete a Validation point, click “remove” X button.

Publish

When you are done with defining expected results and adding validation points to the Test plan, click “Publish” to complete the Test plan.

You will be automatically moved to the Test plans page.

av_new_test_plan_39

Now the Test plan is ready to be executed.

 

Note: If you have not published a Test plan, the Test plan is in draft mode and it cannot be executed. You can use theScreenshot_83-1icon next to Test plan to publish the Test plan.

Tip: Any changes to the process, sub-processes and/or variables and resending the scenario to the Manager, along with other changes to the test plan in the Manager requires you to republish the test plan.