Mobile Test Run Type
Overview
Test Run Type defines how test scenarios are assigned to multiple test environments during execution. A test environment refers to a mobile device or a browser configuration selected in the test plan.
This feature enables teams to balance environment coverage, execution predictability, and operational simplicity based on their testing strategy.
Two run types are supported:
- Cross
- Random
Each run type serves a different testing purpose and should be selected based on the intended outcome.
At least two test environments must be added to the plan to use run types.
Run types are not supported for API tests because they do not use test environments.
Cross Run Type
Description
Cross Run Type executes each test scenario in every test environment included in the plan. It ensures full coverage of the selected environments and produces one result per scenario per environment.
This run type is best suited for compatibility, regression, and consistency validation across mobile devices or browser configurations.
Behavior
- Each scenario runs in every selected environment.
- Coverage of all selected environments is guaranteed within the plan execution.
- For example, a plan containing 10 scenarios and 2 environments generates 20 results, because each scenario runs in both environments.
When to Use
- Regression testing before a release
- Compatibility testing across selected mobile devices or browsers
- UI and behavior consistency validation
- Detection of defects specific to a device, operating system, or browser configuration
Random Run Type
Description
Random Run Type executes each test scenario once in a randomly selected test environment. An environment is selected independently for each scenario, so the next scenario may run in a different environment.
This run type produces one result per scenario. It is suited for continuous testing, stability validation, and broadening environment coverage over repeated executions.
Behavior
- A test environment is selected randomly for each scenario.
- Each scenario is treated independently.
- Previously selected environments remain eligible for selection.
- The same environment may be selected more than once.
- For example, a plan containing 10 scenarios and 2 environments generates 10 results, because each scenario runs once in one randomly selected environment.
Random Run Type does not guarantee that every selected environment will be used within a single plan execution.
When to Use
- CI/CD pipelines and continuous testing
- Long-running or recurring test plans
- Stability and resilience testing
- Testing where varied environment selection is preferred over predictable assignment
Important Notes
- Use Cross Run Type for controlled and repeatable validation across all selected environments.
- Use Random Run Type to vary environment selection automatically across scenarios and repeated executions.
- Random selection does not guarantee full environment coverage within a single plan execution or across repeated executions.
- Selecting the same environment multiple times is expected behavior with Random Run Type.
- To guarantee that every scenario runs in every selected environment within a single plan execution, use Cross Run Type.