Cucumber-JVM Access data stored in Excel, Json and Text file formats

What happens if one has existing data, possibly a large amount, in an excel sheet and wants to use cucumber for testing? What about same set of data which is required across multiple scenarios or feature files? What if the same test data is used for multiple testing tools? The solution is to store multiple copies of the same data in requisite locations which introduces the problem of maintainability.

Overview

This mechanism involves executing a program using the Maven exec plugin at a specific life cycle phase to parse a feature ‘template’ file which contains location of the data store and create the feature file with the necessary data. This can be used to store data externally for DataTables and Examples table for ScenarioOutlines in an Excel, Json or Text file formats. Out of the box it supports simple access, along with ability to write custom logic to manipulate data. One can also keep and access data in Java classes instead of file formats.

The main components of this framework are Feature ‘Template’ File, Data File Format and POM Exec Plugin Details which are sufficient for basic use. The advanced usage of this will be creating custom parsing logic for the three formats and using data stored in Java classes.

The source code can be found here. And a sample project using this is located here.

Basic Usage

This makes use of the in-built data retriever classes and data storage formats. Below are the steps required for this.

Maven pom dependency – Add the following dependency to the pom file.

<dependency>
    <groupId>tech.grasshopper</groupId>
    <artifactId>cukefeatureprocessor</artifactId>
    <version>1.1.1</version>
</dependency>

Maven exec plugin configuration – The entry class for the framework is Main class located in the processor package. This is executed by the Maven exec plugin in the generate-test-resources phase. The first argument to this class is the feature ‘template ‘ file folder path. The second is the actual feature file folder path.

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>exec-maven-plugin</artifactId>
    <version>1.6.0</version>
    <executions>
        <execution>
            <id>generate-test-resources-execution</id> 
            <phase>generate-test-resources</phase>
            <goals>
                <goal>java</goal>
            </goals>
            <configuration>
                <mainClass>processor.Main</mainClass>
                <arguments>
                    <argument>src/test/resources/template/</argument>
                    <argument>src/test/resources/feature/</argument>
                </arguments>
            </configuration>
        </execution>
    </executions>
</plugin>

Feature ‘template’ file – This is similar to the actual feature except that instead of datatable and examples data there is an indicator for the actual data source. To indicate a data store the line must begin with “| Data: |”, leading and trailing spaces are not relevant. This file can also contain normal scenarios and also those with datatable and examples data present.

Refer here for a scenario outline feature ‘template’ file is located here and for a datatable can be found here.

The sample syntax for defining the data store details in the feature ‘template’ file for an excel format. The second parameter indicates the path to the excel data file. The third is the worksheet name. The fourth is the data area.

And Process given datatable
   | Data: | src/test/resources/data/excel/exceldata.xlsx | Sheet1 | D3:F9 |

The sample syntax for a json format. The third parameter is the name of the json array or json object that contains the data.

And Process given datatable
| Data: | src/test/resources/data/json/jsondata.json | data |

The sample syntax for a text format. The third parameter is the delimiter, if empty then the delimiter in text file should be “|”.

And Process given datatable
| Data: | src/test/resources/data/text/textdata.txt | , |

Data storage and retrieval – The examples in the above step are handled by default classes for the specific data store format. Below are the details.

The default class for Excel is ExcelSimpleDataRetriever and data store format is here. The data file format is self explanatory. This is the most convenient to use especially for larger amount of data in table format. A single file can contain multiple data in different sheets or different cell areas. There is another excel retriever class ExcelTransposeDataRetriever which switches the row and column data. To use this the value of the class needs to be mentioned as below.

And Process given datatable
| Data: | src/test/resources/data/excel/excelone.xlsx | Sheet1 | A1:C2 | retriever.excel.ExcelTransposeDataRetriever |

The default class for Json is JsonSimpleDataRetriever and data store format is here. The data file format will contain json arrays or objects for each relevant data. Json would be suitable for small amount of data or data generated by a program. There is another json data retriever class JsonExplodedDataRetriever for this format in which the headers are repeated for each row.

 And Process given datatable
| Data: | src/test/resources/data/json/jsonone.json | data1 | retriever.json.JsonExplodedDataRetriever |

The default class for Text is TextSimpleDataRetriever and data store format is here. For a text file the data for each cell is delimited by “|” by default, otherwise it is mentioned in the template file. Text file can only have a single data. Ideal for existing data in delimited form or small amount of data as readability could be an issue.

Advanced Usage

What happens if one needs to manipulate the data with some custom logic? How about a column needs to be removed from the data in the excel sheet? What if one needs to add a prefix or suffix to certain data values? Also what about passing arguments to the parser class?

One can use custom code in addition to the in-built parsing logic simultaneously.

This is where creating custom parsing classes for the existing formats is useful. Also in this section steps to using classes to store data will be looked at. The maven dependency is the same for both cases.

Below is the changes needed to use a custom parser.

Maven pom dependency – Same as before.

Maven exec plugin configuration – The entry class remains the same though the phase of the exec plugin is changed to process-test-classes phase. The two arguments remain the same. There is now an additional classpath argument. The complete configuration is as below.

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>exec-maven-plugin</artifactId>
    <version>1.6.0</version>
    <executions>
        <execution>
            <id>process-test-classes-execution</id>
            <phase>process-test-classes</phase>
            <goals>
                <goal>java</goal>
            </goals>
            <configuration>
                <mainClass>processor.Main</mainClass>
                <arguments>
                    <argument>src/test/resources/template/</argument>
                    <argument>src/test/resources/feature/</argument>
                </arguments>
                <additionalClasspathElements>
                    <additionalClasspathElement>
                        ${project.basedir}\target\test-classes
                    </additionalClasspathElement>
                </additionalClasspathElements>
            </configuration>
        </execution>
    </executions>
</plugin>

Feature ‘template’ file – The syntax is similar to that explained above. The only change is to mention the name of the custom class and any arguments that may be needed. Below is one for excel, though it is similar for json and text formats.

And Based on pass message changes
| Data: | src/test/resources/data/excel/excelone.xlsx | Sheet1 | A1:C2 | retriever.CustomExcelDataRetriever | arg1 | arg2 |

The custom classes will need to be placed in the src/test/java folder structure of the project.

Data storage and retrieval – The custom parser for excel needs to extend the ExcelDataRetriever class. For json it will be the JsonDataRetriever and TextDataRetriever for text. As mentioned above the class needs to be placed in the src/test/java folder structure of the project. One can override the getData method or one can override other methods instead, which are called by the parent getData method.

Java Class Data Store or Generator

Below is the changes needed to use a class as data store or generator.

Maven pom dependency & exec plugin configuration – Same as before.

Feature ‘template’ file – The second parameter is the name of class. The third parameter is the name of the method to be executed. This is the syntax of the first example. Any further parameters are passed on to the method of the class. This is the syntax of the second example.

And Process given datatable
| Data: | progdata.ProgramSimpleDataRetriever.java | getTableData |

And Process given datatable
| Data: | progdata.ProgramArgumentsDataRetriever.java | getChangedTableData | Before | After |

Data storage and retrieval – The class does not need to extend any class. It just needs a method which takes a string array and returns a string list. The class needs to be placed in the src/test/java folder structure of the project.