Plans
Overview
Creating Plans allows you to define how your automated test scenarios will be executed. A Plan acts as the execution configuration that determines which scenarios will run, under which conditions, and how the execution behaves.
By creating a Plan, you prepare your test automation for execution without modifying the underlying test code. The "Plans" page is where the plans for the projects are created to execute tests.
What is a Plan?
A Plan is a configuration layer on top of your test scenarios.
It defines:
- Which scenarios will be executed
- Which environment the tests will run on
- How the execution will behave (parallelism, retries, execution mode, etc.)
A Plan does not contain test logic. Instead, it controls how and when your existing test scenarios are executed.
Creating a New Plan
To create a new Plan:
- Navigate to the Plans page
- Click Create New Plan
- Fill in the required configuration fields
- Save the Plan
Once created, the Plan becomes available for execution.
Plan Configuration
Basic Information
Define the core identity of your Plan:
- Plan Name: A unique and descriptive name
- Project: The project this Plan belongs to
- Group: The group this Plan belongs to (a group must exist before it can be selected).
Scenario Selection
Select one or more scenarios to include in the Plan.
- Only selected scenarios will be executed
- Scenarios can be updated later if needed
Platform Selection
Choose the environment where the tests will run:
- Web (browser-based execution)
- Mobile (iOS / Android)
- API / Service does not require an environment
The available options depend on your project setup.
Multiple environments can be selected.
Execution Configuration
Dispatch Method & Execution Mode
- You can decide whether the scenarios are run independently in parallel (one by one) or as a continuous stream where each scenario is dependent on the previous one (all together).
- One-by-One: Scenarios run independently and can be parallelized
- All-in-One: Scenarios run sequentially (dependent on each other) in a single environment
- Use All-in-One when execution order or shared state is important.
Run Type (Mobile Only)
- When executing mobile tests, you can define how scenarios will be distributed across available devices using the Run Type option:
- Cross: Each selected scenario is executed on all selected devices. This is useful for validating the same test across multiple devices, OS versions, or configurations.
- Random: Scenarios are distributed across selected devices, and each scenario runs on one available device only. This helps reduce execution time when full device coverage is not required.
- The Run Type option allows you to balance between coverage and execution speed depending on your testing needs.
Advanced Settings
Advanced Settings tab is visible during "Edit Plan" action.
Failed Test Retry Count
The "Failed test retry count" option determines how many times a failed test in your plan will be automatically rerun before being reported as an error. The goal is to reduce false failures caused by "flaky" tests, such as network latency or temporary resource issues, and focus on real failures.
- Only failed scenarios are retried.
- The configured value represents the number of retry attempts, excluding the initial execution.
- Setting the value to 0 disables retries.
For example, if Failed Test Retry Count is set to 2, a failed scenario can be executed up to 3 times in total: 1 initial execution and 2 retry attempts.
Note: Failed Test Retry Count does not apply to All-in-One executions because all scenarios run within a single execution session.
Parallel Test Limit
The Parallel Test Limit setting defines the maximum number of scenarios that can run simultaneously within a plan.
This limit is based on the number of parallel execution licenses available for your account.
- Scenarios run in parallel up to the selected limit.
- Additional scenarios wait until an execution slot becomes available.
- The selected value cannot exceed your licensed parallel capacity.
- Setting the value to 0 means unlimited, allowing the plan to use all available licensed parallel capacity.
For example, if your account has 4 parallel licenses and the Parallel Test Limit is set to 0, the plan can run up to 4 scenarios in parallel.
Note: Parallel Test Limit does not apply to All-in-One executions.
Max Execution Time per Scenario
This setting defines the maximum amount of time a single scenario is allowed to run during execution. If a scenario exceeds this limit:
- The execution of that scenario is automatically terminated
- The scenario is marked as "Timed Out" status
This setting helps prevent:
- Scenarios getting stuck indefinitely
- Long-running tests are blocking execution resources
- Unpredictable execution durations
- Use this option to ensure that all scenarios complete within a controlled and expected time frame, especially in large or parallel test executions.
Screenshots
The Screenshots setting controls whether screenshots are captured during scenario execution.
A screenshot is taken for each executed command and associated with that command in the Command tab of the Execution Report.
Available options:
- Yes: Screenshots are captured during execution.
- No: Screenshot capture is disabled.
- Only Failure: Screenshots are enabled only if the scenario status is Failed.
Captured screenshots can be viewed under the Command tab in the Execution Report.
Record Video
The Record Video option enables video recording of the entire scenario execution. When enabled:
- A full video of the test run is recorded
- The recording is attached to the execution results
Captured video can be viewed under the Video tab in the Execution Report.
Clear App Data
The Clear App Data option resets the application state before each scenario execution (Android only).
When enabled:
- All app data is cleared before the scenario starts
- The app behaves as if it is launched for the first time
This helps ensure:
- Scenario consistency
- No leftover data affects execution
For iOS devices, if a clean application state is required, the Uninstall App option can be used instead to remove and reinstall the application before execution.
Note: Clear App Data does not apply to All-in-One executions because all scenarios run within a single execution session.
Uninstall App
The Uninstall App option removes the application from the device before each scenario execution.
When enabled:
- On iOS devices, the application is uninstalled during scenario initialization.
- On Android devices, the application is uninstalled after the scenario execution is completed.
- This helps prevent application data or configuration from previous executions from affecting the next run.
Note: Uninstall App does not apply to All-in-One executions because all scenarios run within a single execution session.
Public Devices: The application is uninstalled automatically, so this setting does not apply.
Capture Performance Data
Available in Selenium projects only.
Performance data capture/recording is the process of recording and analyzing measurable values such as the operating speed, efficiency, and resource usage (CPU, RAM, disk, network) of a system, application, or network over a specific period. This process provides critical data for finding bottlenecks, determining the cause of system slowdowns, and optimizing overall performance.
System Parameters
The System Parameters section allows you to define key-value pairs that are injected into your test code at runtime.
Each parameter consists of:
- Name: The identifier used in your test code
- Value: The value assigned to the parameter
- Type: Defines how the parameter is handled (e.g., text, secret)
- Description: Optional explanation for documentation purposes
When a Plan is executed:
- These parameters are passed to the execution environment
- Your test code can dynamically read and use them
This enables:
- Environment-specific configurations (e.g., URLs, credentials)
- Reusable test scenarios across different contexts
- Separation of test logic and runtime data
Note: System Parameters do not modify your test code directly. They are provided at runtime and must be consumed within your test implementation.
Scheduling & Notifications
"Schedule & Notifications" tab is visible during "Edit Plan" action
Scheduling
You can configure the Plan to run automatically:
- At specific times
- On a recurring basis
This is useful for:
- Regression testing
- Continuous validation
Notifications
The Notifications section allows you to receive updates about your test executions based on defined conditions.
When notifications are enabled:
- You can configure alerts to be sent via email
- Notifications are triggered based on execution timing and test results
Each notification rule includes:
- Notification Type: The delivery channel (e.g., email)
- E-mail: Recipient address
- Execution Status: When the notification should be sent
- Beginning of each execution
- End of each execution
- Test Result Status: Only available for "End of each execution" mode, allows control for which results should trigger the notification (e.g., success, failure)
This enables you to:
- Stay informed about test execution progress
- Get immediate alerts on failures
- Monitor critical test outcomes without manual tracking
Multiple notification rules can be defined to support different recipients and conditions.
Note: Notifications are sent per execution based on the configured criteria and do not include full reports, but may include summary information and links to detailed results.