As a software tester, at one point or another you definitely will run into repetitive and time-consuming test scenarios. A simple example is testing the Login page, where you need to test more than dozens of credential pairs to account for all of the scenarios, especially the invalid credential scenarios. In such cases, you need data-driven testing. To do data-driven testing, you need a data-driven testing framework.
You can build a data-driven testing framework in Selenium. The idea is to store test data in external sources like spreadsheets or databases and dynamically feed this data into Selenium scripts during test execution.
Why is this significant? Imagine effortlessly testing your application across various user inputs, browsers, and platforms, all while reducing maintenance efforts. Data-driven frameworks make data-driven testing a breeze, saving you time and resources, and ultimately enhancing the quality of your software. Say goodbye to repetitive testing and hello to streamlined, data-powered success!
In this article, we’ll explain the concept of a data-driven framework in-depth and then provide you with the code for such a framework in Selenium.
Data-driven testing (DDT) is a testing method where you execute the same test scenario across a wide range of data input. Instead of creating separate test scripts for each input data, DDT allows you to reuse your test code. Its primary goal is to validate application behavior under a wide range of input conditions, focusing on executing the same test logic using multiple data sets to ensure the application behaves accurately in different scenarios.
At its core, DDT is a process with the following four steps:
Imagine testing a login functionality where you want to check various scenarios: valid login, invalid login, login with different user roles, and so on. Instead of writing separate scripts for each scenario, DDT lets you input different usernames and passwords from your data source, automatically running the tests for all combinations.
For software testers, DDT brings a variety of benefits:
As a whole, DDT provides better flexibility, maintainability, and scalability for test automation efforts. Indeed, it helps QA teams streamline the testing process and improve efficiency by separating test data from test script logic.
Selenium can be used to create a data-driven testing framework. Leveraging Selenium, you can create the backbone for efficient and scalable data-driven testing, enabling your team to seamlessly manage test data, thereby enhancing the reliability and maintainability of automated tests.
The key components of this framework include:
Overall, a data-driven framework enhances test automation. Whether you're testing web applications, APIs, or mobile apps, mastering the data-driven framework will undoubtedly elevate your Selenium test automation game.
Read More: 6 Common Types of Software Test Automation Frameworks
Imagine that you are the tester of an established e-commerce company. Your goal is to ensure that the checkout process is working as expected, so you want to run a test to see if users can authenticate their account seamlessly on your website. We can immediately see that this test scenario poses several challenges:
Those challenges can easily be addressed with data-driven testing. Instead of manually typing in specific values for each scenario (which is essentially a hard-coding approach), we can simply prepare the test data for all of the possible scenarios in one database, with each row of data representing one scenario, then write a script to read test data and execute the test steps.
There are two options when it comes to data-driven testing:
Let’s compare the two approaches using the example above.
In our example, you can prepare an Excel sheet with four columns: “Username”, “Password”, “Address”, and “Payment Method”. You can add as many rows as you like for different sets of user data.
Username | Password | Address | Payment Method |
username1 | password1 | Street 1 City 1 | Visa credit card |
username2 | password2 | Street 2 City 2 | Visa debit card |
username3 | password3 | Street 3 City 3 | PayPal |
After that, you need to create a new Java project in your IDE and add the Selenium WebDriver and TestNG libraries to your project.
Finally, write your test script to incorporate logic to read data. In our case, the script should iterate through each username and password, use them to login to the user account, and then initiate the checkout process where it fills in the address, before initiating the payment method.
If you are not familiar with coding, check out the top automation testing tools on the market that can help you automate this process without having to write a lot of code. This is the second approach. Many automated testing tools already have a DDT framework for you to use.
If you decide to go with this approach, simply create the test cases leveraging their built-in features. Top testing tools should come with helpful test authoring features such as Built-in Keywords or Record-and-Playback, which can drastically reduce your time spent writing code. They even come with Smart Reporting features which automatically generate out-of-the-box reports with rich insights on the quality of the system under test. It’s really simple!
In other words, to do data-driven testing, follow the following steps:
Let’s find out how we can do data-driven testing with Selenium and with Katalon – a comprehensive automation testing tool.
Katalon Platform is a comprehensive test automation platform that allows you to easily create, manage, execute, and generate reports for test cases across web, desktop, mobile, and APIs. Katalon comes with a data-driven testing framework already built in, so all you have to do is create an account, and download Katalon Studio, which is a productive IDE to generate automated tests without having to write a single line of code.
Download Katalon & Start Testing Within Minutes
Once you’ve downloaded Katalon Studio, launch it. You should see the Katalon Studio interface as below. You can then click the “Create new Test Case” button.
You can craft test cases from the wide variety of keywords we offer (which are essentially code snippets to perform specific actions). Here are just a few examples of them: Navigate to URL, Open Browser, Refresh, Right Click, Scroll, Send Keys, etc.
Read more: Create test cases with Katalon Studio
Here we have created a test case called “Find a place”, which includes the following:
This test case is put under the DDT with MS Excel Files test suite.
Let’s prepare a data file in Excel with two columns: “City” and “Expected Result”. The values Atlanta, New York, and Tokyo will be dynamically added to the script as we execute.
After that, in the Data File section let’s create new test data called excelfile_100cities. Once done, you should arrive at the test data interface, and you can click Browse to import the Excel file we’ve just created into Katalon Studio.
Once imported, we go back to the DDT With MS Excel Files test suite. Click Add and choose the newly imported Excel to bring it to your test suite. After that, click Map All, and Katalon automatically maps all of the columns in your Excel file to the corresponding variable in your test script.
Finally, click Run, and you’re good to go!
As the test gets executed, you should see that it automatically opens the browser, navigates to Airbnb, searches for the exact location we specify in our Excel file, and checks for the available spaces there.
What’s even better is that once the execution is done, the test results are automatically imported into Katalon TestOps for management and reporting purposes. For more information, you can check out Data-driven testing with Katalon Studio.
You can even watch a detailed guide on how to do data-driven testing at Katalon Academy.
You can see how the process of doing data-driven testing is much more straightforward and simpler than going with the traditional scripted approach. A good advantage that tools have over libraries is that they come with self-healing features, which automatically fix any broken test scripts during regression test runs. If you go with Selenium, you must manually fix each individual broken test script, and that is quite time-consuming. Read more for a Katalon vs. Selenium detailed comparison.
Apache POI is an open-source Java library designed to help developers work with Microsoft Office file formats such as Excel, Word, and even PowerPoint documents. It enables you to interact with data sets, and when paired with Selenium, an automation testing framework, you can easily create data-driven test cases that can both read data, automate web interactions, and generate test reports.
First, you need to set up your development environment for creating and running Selenium automation tests. You can use popular Java IDEs like Eclipse and IntelliJ, and then, in their starting interface, you can navigate to File > New > Project to set up a new project.
Make sure to also download the JAR files for Selenium WebDriver here and Apache POI libraries here.
In Eclipse:
In IntelliJ IDEA:
You are now ready to use Selenium WebDriver to automate web tests and Apache POI to work with Excel files in your project.
Simply prepare your test data in a spreadsheet. Each row represents a different test case, and the data in each column corresponds to the parameters you want to pass to your test cases. To know what type of data you should populate your sheet with, it is recommended to write out your test plan and test strategy document first. They give you a comprehensive view over your testing objectives.
Once it’s already, you can write your Selenium test script to automate your test cases, but instead of hard-coding the test data, you can read it from the Excel file you prepared. To do it, you need to initiate a Workbook and Sheet object to read data.
Here’s an example of a test script written in Java using Selenium and Apache POI to test the login feature. Let’s assume that you have a ‘username’ and ‘password’ field in your XLSX file called “data.xlsx”, and the site you want to test the login feature is called “https://testwebsite.com”:
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.apache.poi.ss.usermodel.*;
import org.apache.poi.xssf.usermodel.XSSFWorkbook;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
public class DataDrivenTest {
public static void main(String[] args) throws IOException {
// Initialize WebDriver
System.setProperty("webdriver.chrome.driver", "path/to/chromedriver");
WebDriver driver = new ChromeDriver();
// Initialize Excel file
FileInputStream file = new FileInputStream(new File("path/to/your/excel/data.xlsx"));
Workbook workbook = new XSSFWorkbook(file);
Sheet sheet = workbook.getSheet("Sheet1");
// Iterate through the rows and columns to read the data
for (int rowNum = 1; rowNum <= sheet.getLastRowNum(); rowNum++) {
Row row = sheet.getRow(rowNum);
String username = row.getCell(0).getStringCellValue();
String password = row.getCell(1).getStringCellValue();
// Execute the test with the current set of data
driver.get("https://testwebsite.com");
WebElement usernameField = driver.findElement(By.id("username"));
WebElement passwordField = driver.findElement(By.id("password"));
WebElement loginButton = driver.findElement(By.id("login-button"));
usernameField.sendKeys(username);
passwordField.sendKeys(password);
loginButton.click();
// Add verification/assertion steps here
// Close the browser or perform any necessary cleanup
}
file.close();
driver.quit();
}
}
However, please note that the previous script did not include a test reporting mechanism. To enable test logging, you need to import the java.util.logging package. Here’s the updated version for test logging mechanism:
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.apache.poi.ss.usermodel.*;
import org.apache.poi.xssf.usermodel.XSSFWorkbook;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.util.logging.FileHandler;
import java.util.logging.Level;
import java.util.logging.Logger;
public class DataDrivenTest {
public static void main(String[] args) throws IOException {
// Initialize WebDriver
System.setProperty("webdriver.chrome.driver", "path/to/chromedriver");
WebDriver driver = new ChromeDriver();
// Initialize logging
Logger logger = Logger.getLogger(DataDrivenTest.class.getName());
FileHandler fileHandler = new FileHandler("test_report.log");
logger.addHandler(fileHandler);
// Initialize Excel file
FileInputStream file = new FileInputStream(new File("path/to/your/excel/data.xlsx"));
Workbook workbook = new XSSFWorkbook(file);
Sheet sheet = workbook.getSheet("Sheet1");
// Iterate through the rows and columns to read the data
for (int rowNum = 1; rowNum <= sheet.getLastRowNum(); rowNum++) {
Row row = sheet.getRow(rowNum);
String username = row.getCell(0).getStringCellValue();
String password = row.getCell(1).getStringCellValue();
// Execute the test with the current set of data
driver.get("https://testwebsite.com");
WebElement usernameField = driver.findElement(By.id("username"));
WebElement passwordField = driver.findElement(By.id("password"));
WebElement loginButton = driver.findElement(By.id("login-button"));
usernameField.sendKeys(username);
passwordField.sendKeys(password);
loginButton.click();
// Add verification/assertion steps here
// Example: You can check the URL, page title, or specific elements for expected conditions.
if (driver.getCurrentUrl().equals("https://testwebsite.com/dashboard")) {
logger.log(Level.INFO, "Test case PASSED: Username=" + username + ", Password=" + password);
} else {
logger.log(Level.SEVERE, "Test case FAILED: Username=" + username + ", Password=" + password);
}
// Close the browser or perform any necessary cleanup
}
file.close();
driver.quit();
}
}
Data-driven testing is an amazing approach to improve your efficiency in highly repetitive test scenarios. You can either build a data-driven testing framework from scratch using Selenium and libraries, which is highly customizable for specific scenarios, but once those scripts are broken, maintaining them is quite a daunting task. Or, you can go with an automated testing tool that comes with a prebuilt data-driven testing framework that you can use right away.