.svg)
The wider adoption of Agile and DevOps practices in recent years have led to a significant shift in the way software is developed and tested. 2 emerging concepts are shift left testing (moving testing to the left on the project timeline) and shift right testing (performing testing in production). Both carry their own advantages, and in this article we’ll explore primarily the concept of shift right testing and explore the differences between shift left vs. shift right.
What Is Shift Right Testing?
Shift right testing, also known as testing in production, focuses on conducting testing activities after the development process. It involves gathering insights from real user feedback and interactions after the software has been deployed. Developers then use those insights to improve software quality and come up with new feature ideas.
Shift Right Testing Activity
Shift right testing aims to bring testing closer to the end users, hence the term “shift right”. There are 3 major criteria for any shift right testing activity:
- Stability
- Performance
- Usability
Shift right testing is also where experiments happen, with QA teams trying out new features or improvements (usually with a carefully selected set of users). The only catch is that the software, application, or website needs to receive a considerable number of user visits in the first place for the experiment results to be statistically significant.
The table below lists out several notable activities you can try during your shift right testing sessions:
Activity | What To Do |
Production Monitoring | Continuously monitor the performance of the software in the live production environment. Important items to measure include:
|
Log Analysis | Analyzing logs generated by the production system to detect any abnormal patterns and troubleshoot issues early on |
User Feedback Collection | Actively engaging with real users to gather feedback on their experience. The team can build an in-app survey form to collect their opinions when a certain event is triggered (such as installing the app, or during registration on the website) |
A/B Testing | Create 2 versions of your application/webpage and split the incoming user visits between those 2 versions. Run them for a sufficient duration to collect data on any metrics you are measuring. |
Feature Toggling | Use feature toggles to selectively enable or disable specific features or configurations in the live environment, allowing for more controlled rollouts and quick adjustments based on user feedback. |
Continuous Improvement | Using insights gained from monitoring, feedback, testing, and analysis to drive continuous enhancements, prioritize feature refinements, optimize performance, and address usability issues. |
Benefits Of Shift Right Testing
- Real-time Feedback: the sole purpose of shift right testing is to test with real data for real results instead of using simulated data. This brings back authentic feedback and even helps to uncover valuable insights on user preferences.
- Improved User Experience: the insights brought back from shift right testing sessions is translated into technical improvements for the software and boosts its business value. Going beyond just fixing “bugs”, now the dev team can also optimize certain features to bring users a better experience.
- Continuous Improvement: the shift right approach encourages continuous testing and learning about the software, knowing that there is always room for optimization for the software as more and more features are added. Over time, these insights can even be “consolidated” into principles about the software and the business itself, serving as reference for strategy development.
Tools For Shift Right Testing
- Application performance monitoring (APM) tools: to monitor the performance and behavior of live applications in real-time to identify performance issues. Examples: New Relic, Dynatrace, AppDynamics
- Log analysis tools: to extract meaningful data from logs to pinpoint the root cause of any system error. Examples: ELK Stack (Elasticsearch, Logstash, Kibana), Splunk, Graylog
- User analytics tools: to help you collect user behavior data and gain insights into their preferences, guiding UX optimization activities. Examples: Google Analytics, Mixpanel, Amplitude
- Feedback collection tools: to gather feedback from end-users to understand their experiences and make necessary improvements. Examples: UserVoice, Zendesk, Intercom
What Is Shift Left Testing?
Shift Left testing is an approach that focuses on conducting testing activities earlier in the development process. The idea is to shift testing to earlier stages rather than waiting until the final stages. This proactive approach aims to identify and resolve defects early, preventing them from spreading throughout the entire application, reducing the cost and effort required for fixing issues.
Many QA teams in the past often delayed testing until the final stage, believing that testing should be separate from development and performed only when the product is fully built. However, this also means that there is not enough time for testing, causing delays before the app can be released.
Shift left testing is an innovative approach where testing and development happens almost at the same time, with the testing giving continuous feedback to the development team as they build the software.
Read More: What is Shift Left Testing?
Benefits Of Shift Left Testing
- Early Bug Detection: at earlier stages of development, bugs are less expensive and easier to fix since there is less code and fewer dependencies between components. By minimizing the number of interactions and connections between software features, bugs are less entrenched in the software design. Testers can troubleshoot without making drastic changes to the existing code.
- More Flexibility For Both Teams: with shift left testing, the development team can build the app and quickly get feedback from testing. Similarly, the QA team can test smaller chunks of code one by one instead of a full build with a complex network of bugs. Both teams benefit from this approach.
- Improved collaboration: shift left testing leverages the expertise of both dev and testing teams to increase project efficiency. It fosters innovation through the diversity of skills and perspectives involved. However, they must have a shared understanding of expectations, so using a testing tool with collaboration capabilities is a recommended starting point.
Tools For Shift Left Testing
- Static code analysis tools: to help scan code and identify coding errors issues early in the development process, allowing developers to quickly review their code. Popular tools: SonarQube, Checkstyle, ESLint
- Unit testing frameworks: to automate unit testing, so that developers can run tests themselves before committing code. Example frameworks: JUnit, NUnit, pytest
- Test automation tools: to automate test creation and test execution, helping QA teams quickly catch bugs and find root cause. Examples: Katalon, Selenium
- Continuous integration tools: to automate the build and testing processes, ensuring that code changes integrate smoothly and identify issues promptly. Examples: Jenkins, Travis CI, CircleCI
Shift Left vs. Shift Right Testing
So what is the difference between Shift Left and Shift Right Testing? The answer is that they are only 2 sides of the same coin. They don’t cancel each other, but rather complement each other, and using both approaches can actually bring a more comprehensive testing experience to the team.
In this table, we compare shift left vs shift right testing based on 7 criteria:
Shift Left Testing | Shift Right Testing | |
When It Happens | Early stages of software development life cycle | Post-production, in the live environment |
Objective | Early bug detection and prevention | Real-time monitoring and user feedback for improvements |
Activities | Static code analysis, unit testing, code reviews | User acceptance testing, monitoring, feedback analysis |
Emphasis | Prevention-oriented | Improvement-oriented |
Test Environments | Development and testing environments | |
Collaboration | Developers, testers, QA teams | Developers, testers, operations, customer support |
Iterative Approach | Yes | Yes |
Level Up Your Testing With Katalon
Katalon is a modern, AI-augmented test automation solution and quality management platform for web, mobile, API, and desktop applications. It provides a unified platform for teams to plan, design, execute, and manage automated testing efforts. Most importantly, Katalon has everything you need to shift left all testing activities of your organization.
- Early automated testing with Katalon Studio: Katalon has an extensive library of Built-in Keywords, which are essentially code snippets to perform any interactions you want with the system, such as Open Web Browser, Click, SetText, and many more. Just drag-and-drop them in order and you’ll have a full test script ready to be executed across a wide variety of environments, OS, browsers, and devices.
- CI/CD integrations: Katalon seamlessly integrates with popular CI/CD tools as well as collaboration tools such as Slack, Microsoft Teams, Git, improving transparency and enhanced communication between team members.
- Test Management Capabilities: Testers can store application elements in the Object Repository, a centralized location for test artifacts. This allows for sharing and reusing test cases and data among team members and projects, enhancing test reusability.
