Introduction
This article introduces the concept of running scenarios in random order. This makes use of the new functionality introduced in Cucumber-JVM version 4.4.0. This option is available with the cucumber.api.cli.Main class.This can help in determining any existing dependency between the scenarios.
Another related functionality that has also been introduced is the ability to limit the number of scenarios to be executed.
Source Setup
Create a parent folder ‘cukerandom’ and a folder ‘code’ inside it.
To compile and run the command from CLI, cucumber jars version 4.4.0 are used here. Refer to the ‘Cucumber Jars’ section in this article for details about downloading these jars. Keep the jars in the ‘cukerandom’ folder.
Copy the stepdefinition class and feature files as described in the same article into the ‘code‘ folder.
Compile the step definition classes from the ‘cukerandom’ folder at the command line.
javac -cp “.;<path to cukerandom folder>/*” code\*.java
Run the following command from the ‘cukerandom’ folder to execute the feature files.
java -cp “.;<path to cukerandom folder>/*” cucumber.api.cli.Main -g code code
An output similar to below should be visible on the console window.
Given Scenario Outline Row 1 from scenariooutlines feature file. Given Scenario Outline Row 2 from scenariooutlines feature file. Given Scenario 1 from scenarios feature file. Given Scenario 2 from scenarios feature file.
Scenario Ordering
The scenarios can be run in random order using the –order option of the cucumber.api.cli.Main class. There are three order options available –
- –order random
- –order random:seed
- –order reverse
Random order – The scenarios are executed with a random seed. The seed is displayed at top of the output. This option is useful for determining if there is any dependency between the scenarios.
java -cp “.;<path to cukerandom folder>/*” cucumber.api.cli.Main --order random -g code code
An output something similar to below should be visible on the console window.
Using random scenario order. Seed: 9155175292024747033 Given Scenario Outline Row 2 from scenariooutlines feature file. Given Scenario 1 from scenarios feature file. Given Scenario Outline Row 1 from scenariooutlines feature file. Given Scenario 2 from scenarios feature file.
Random order with seed – The scenarios are executed with a pre-defined seed. This is useful in repeating the scenario execution order in case there is a failure.
java -cp “.;<path to cukerandom folder>/*” cucumber.api.cli.Main --order random:9876543210 -g code code
An output in the console window should be same as below.
Given Scenario 2 from scenarios feature file. Given Scenario 1 from scenarios feature file. Given Scenario Outline Row 2 from scenariooutlines feature file. Given Scenario Outline Row 1 from scenariooutlines feature file.
Reverse order – The scenario is run in opposite order of the natural execution.
java -cp “.;<path to cukerandom folder>/*” cucumber.api.cli.Main --order reverse -g code code
An output in the console window should be same as below.
Given Scenario 2 from scenarios feature file. Given Scenario 1 from scenarios feature file. Given Scenario Outline Row 2 from scenariooutlines feature file. Given Scenario Outline Row 1 from scenariooutlines feature file.
Refer to this for the official usage details.
Scenario Count
The number of scenarios that are executed can be limited by using the –count option of the cucumber.api.cli.Main class. Refer to this for the official usage details.
The limit is applied only after the filter criterion of tags, names and lines have been applied to all the available scenarios. If the count option is greater than the filtered scenarios, the count option will be not considered.
java -cp “.;<path to cukerandom folder>/*” cucumber.api.cli.Main --count 2 -g code code
An output in the console window should be same as below.
Given Scenario Outline Row 1 from scenariooutlines feature file. Given Scenario Outline Row 2 from scenariooutlines feature file.
The count option is independent of the order option and can be used separately or together with it.