Smartphones literally dominate the world. There are 6.92 billion smartphone users in the world today, which is 85.88% of the world population. That figure is only 49.40% in 2016. Such ubiquity of smartphones demands mobile app developers to deliver high-quality applications at all times to remain competitive and relevant through effective mobile application testing.
No matter what type of applications they work on, there are typically 2 primary approaches to mobile testing: manual mobile testing and automated mobile testing. The rising popularity of AI-powered testing technologies and automated QA tools make automated mobile testing seem like a more sensible approach. However, manual mobile testing carries its own potential that QA teams must know how to unlock.
In this article, we will take a deep-dive into the process of manual mobile application testing and see how teams should embrace a hybrid approach to receive the best of both worlds.
Mobile testing, or mobile app testing, is the process of checking and validating all aspects of the mobile app to ensure that they work as expected. Any issues and defects found during the process are reported to the dev team to address immediately so that they will not slip into production.
In today's digital age, mobile app testing plays an indispensable role in:
Bugs cost money.
Modern applications are built from highly interconnected components that must work together seamlessly to deliver the final expected feature. One single bug impacting a crucial component may create a negative ripple effect to other components and break the entire app.
For applications in sensitive fields (medicine, law, finance, insurance, etc.), such bugs can have irreversible consequences. Good mobile app testing catches those bugs and prevents them from slipping into production.
The journey of developing a mobile application pose never-ending questions like:
Testing answers those questions, so we can confidently and continuously improve our app thanks to insights from test results.
When doing mobile app testing manually, human testers interact with the application manually just as an end-user would to check if the application delivers the expected functionality and performance. They don’t rely on help from any automated scripts or tools, but solely use their human capabilities to find bugs.
Of course, we are humans, and humans make mistakes. We are not perfect creatures. Certain bugs can only occur due to repetitive and abnormal series of interactions with the system, but we are not built to do things repetitively - it is boring and, most importantly, time-consuming. That’s why automated testing is so much preferred for repetitive scenarios. However, manual testing really shines through for non-repetitive scenarios.
When a developer manually runs their application and tries out the features they coded, that is manual mobile testing. It would be an overkill to build an entire automation test script for a simple, personal project. Even when working on a large-scale project (say, a mobile app for a global bank with highly secured and feature-rich banking experience), testers may still have to do some manual testing first to identify the areas worth automating.
But manual testing is more than just a precursor to automation testing. By nature, testing mobile apps manually come with unique advantages:
Learn How To Do Mobile Testing Without Code
Emulators and simulators both replicate a mobile device, but emulators replicate both the software and hardware, while simulators replicate only the software part. They have several advantages and disadvantages compared to testing on real mobile devices:
Advantages:
Disadvantages:
To test on emulators/simulators, you can:
Read More: Mobile Emulator vs Simulator vs Real Device: Key Differences
A “real device cloud” is a cloud-based platform providing access to a wide range of physical mobile devices for testing and development purposes. You can choose between a lot of makes, models, OS, screen sizes, and even network configurations on these clouds. Even better, users can access and control those physical devices remotely through the web.
Generally speaking, real device cloud gives testers a much more comprehensive testing experience. For mobile applications with a diverse user base, or having specific hardware requirements, this is a great option compared to emulators/simulators.
For example, with Katalon TestCloud, you can gain access to mobile test environments to execute your tests without having to buy any physical machines or set up local emulators/simulators. Katalon TestCloud allows you to run tests on mobile browsers as well as for mobile native applications.
To get started, create your free Katalon account. You should get access to the Katalon TestCloud and Katalon Studio.
Katalon Studio allows you to create mobile test cases by drag-and-dropping uilt-in Keywords (which are essentially code snippets) or record your on-screen actions and turn them into a test script. In other words, you don’t need to have coding experience to create tests for mobile apps in Katalon Studio.
You can then click on the Run icon to select where you want to run your mobile tests. Choose TestCloud to execute tests on common browsers and operating systems on the cloud.
Here you can choose the specific device, OS, version, and browser you want to run your test on. After everything is fully configured, simply click “Run”.
Real device cloud is invaluable for ensuring the quality, compatibility, and reliability of mobile applications across a wide range of real-world devices and conditions.
Start Mobile Testing With Katalon
Testing on real devices simply means executing your tests on actual physical devices rather than using emulators, simulators, or virtual environments. The obvious advantage of this approach is its level of realism. Real devices provide the most accurate representation of how an application will perform in real-world scenarios.
However, access to a wide range of real devices is challenging. Purchasing the devices is only the first stage. Organizations also need to maintain and manage them, and over time this can compound into enormous cost, which is not at all ideal for smaller teams.
Real device testing is also constrained by environmental factors, such as physical device availability, testing space, and the need for manual intervention.
Automated mobile testing involves creating test scripts that can be stored in the version control system, and its results are logged with detailed information about each step performed. For manual mobile testing, it is not that simple. Human testers have to manually test then manually document the results in a spreadsheet or a test case management system. This is why having a clear test strategy and test plan is especially important when testing mobile apps manually.
In a test plan, determine testing objectives, which allow you to better identify test scenarios and establish the potential test cases. Have a test case template to help you note down your test steps in a structured fashion. This should also help to standardize the testing process and improve test result consistency throughout many iterations.
Many manual mobile testers develop a click path to help them achieve that goal.
A detailed click path provides step-by-step instructions for navigating through the mobile app's user interface. This ensures that testers can reproduce the same sequence of actions consistently during testing, which is essential for reliable results.
Knowing what you are clicking on and where it is leading you to should also improve your test coverage. Compare your click path and the app architecture to identify areas that have not been adequately tested. The click path also makes it much easier to reproduce the bug, since you can just reverse the path from the breakpoint to the starting point. Developers can use that insight to find the root cause and fix the bug more easily.
Read More: Bug Life Cycle in Software Testing
It is actually an ambitious endeavor to test all devices and environments there are, so QA teams usually deep-dive into analytics of their user base structure to identify which techstack app users tend to use to identify the devices and environments worth testing.
When talking about environments, we are also talking about network conditions. Certain applications provide both online and offline modes. Testing under different network conditions also provides insights into how the app consumes network data, and if any features cause slower-than-average loading time.
Manual testing is the ultimate playground for testers. They don’t really have to follow a path as rigid as automated testing, so they can totally take advantage of this liberty to explore the more complex scenarios. Interestingly, a recent research shows that manual mobile testing catches bugs requiring 3+ interactions from users i.e. complicated bugs, while automated testing can only catches bugs requiring fewer than 3 interactions.
The power of testing manually does not stop there. This is an opportunity to check for UX-related issues, such as unintuitive workflows, confusing interfaces, or unexpected user interactions.
Ad-hoc testing and exploratory testing are both popular manual testing techniques allowing testers to go beyond the “happy path” and discover bugs that they have never thought of before. Testers rely solely on their intuition, experience, and existing knowledge of the application to freely “explore” the system, trying combinations of interactions that can potentially lead to a bug. This degree of liberty is much harder to achieve in automation testing, so make sure to utilize it when doing manual mobile testing.
Finally, as we are doing everything manually, results must also be carefully documented in a spreadsheet for future analysis. For aspects of the bug that can be quantified, make sure to establish a structured framework for better bug understanding, future analysis, and troubleshooting. This is also known as the bug taxonomy.
Essentially, in a bug taxonomy, bugs sharing common attributes are organized into predefined categories. Below is a compilation of fundamental bug categories for your consideration:
There are several types of mobile app testing, including: