The wider adoption of Agile and DevOps practices in recent years have led to a significant shift in the way software is developed and tested. 2 emerging concepts are shift left testing (moving testing to the left on the project timeline) and shift right testing (performing testing in production). Both carry their own advantages, and in this article we’ll explore primarily the concept of shift right testing and explore the differences between shift left vs. shift right.
Shift right testing, also known as testing in production, focuses on conducting testing activities after the development process. It involves gathering insights from real user feedback and interactions after the software has been deployed. Developers then use those insights to improve software quality and come up with new feature ideas.
Shift right testing aims to bring testing closer to the end users, hence the term “shift right”. There are 3 major criteria for any shift right testing activity:
Shift right testing is also where experiments happen, with QA teams trying out new features or improvements (usually with a carefully selected set of users). The only catch is that the software, application, or website needs to receive a considerable number of user visits in the first place for the experiment results to be statistically significant.
The table below lists out several notable activities you can try during your shift right testing sessions:
Activity | What To Do |
Production Monitoring | Continuously monitor the performance of the software in the live production environment. Important items to measure include:
|
Log Analysis | Analyzing logs generated by the production system to detect any abnormal patterns and troubleshoot issues early on |
User Feedback Collection | Actively engaging with real users to gather feedback on their experience. The team can build an in-app survey form to collect their opinions when a certain event is triggered (such as installing the app, or during registration on the website) |
A/B Testing | Create 2 versions of your application/webpage and split the incoming user visits between those 2 versions. Run them for a sufficient duration to collect data on any metrics you are measuring. |
Feature Toggling | Use feature toggles to selectively enable or disable specific features or configurations in the live environment, allowing for more controlled rollouts and quick adjustments based on user feedback. |
Continuous Improvement | Using insights gained from monitoring, feedback, testing, and analysis to drive continuous enhancements, prioritize feature refinements, optimize performance, and address usability issues. |
Shift Left testing is an approach that focuses on conducting testing activities earlier in the development process. The idea is to shift testing to earlier stages rather than waiting until the final stages. This proactive approach aims to identify and resolve defects early, preventing them from spreading throughout the entire application, reducing the cost and effort required for fixing issues.
Many QA teams in the past often delayed testing until the final stage, believing that testing should be separate from development and performed only when the product is fully built. However, this also means that there is not enough time for testing, causing delays before the app can be released.
Shift left testing is an innovative approach where testing and development happens almost at the same time, with the testing giving continuous feedback to the development team as they build the software.
Read More: What is Shift Left Testing?
So what is the difference between Shift Left and Shift Right Testing? The answer is that they are only 2 sides of the same coin. They don’t cancel each other, but rather complement each other, and using both approaches can actually bring a more comprehensive testing experience to the team.
In this table, we compare shift left vs shift right testing based on 7 criteria:
Shift Left Testing | Shift Right Testing | |
When It Happens | Early stages of software development life cycle | Post-production, in the live environment |
Objective | Early bug detection and prevention | Real-time monitoring and user feedback for improvements |
Activities | Static code analysis, unit testing, code reviews | User acceptance testing, monitoring, feedback analysis |
Emphasis | Prevention-oriented | Improvement-oriented |
Test Environments | Development and testing environments | |
Collaboration | Developers, testers, QA teams | Developers, testers, operations, customer support |
Iterative Approach | Yes | Yes |
Katalon is a modern, AI-augmented test automation solution and quality management platform for web, mobile, API, and desktop applications. It provides a unified platform for teams to plan, design, execute, and manage automated testing efforts. Most importantly, Katalon has everything you need to shift left all testing activities of your organization.