Manual Mobile Testing: A Comprehensive Guide

Smartphones literally dominate the world. There are 6.92 billion smartphone users in the world today, which is 85.88% of the world population. That figure is only 49.40% in 2016. Such ubiquity of smartphones demands mobile app developers to deliver high-quality applications at all times to remain competitive and relevant through effective mobile application testing.
 

No matter what type of applications they work on, there are typically 2 primary approaches to mobile testing: manual mobile testing and automated mobile testing. The rising popularity of AI-powered testing technologies and automated QA tools make automated mobile testing seem like a more sensible approach. However, manual mobile testing carries its own potential that QA teams must know how to unlock.
 

In this article, we will take a deep-dive into the process of manual mobile application testing and see how teams should embrace a hybrid approach to receive the best of both worlds.

What is Mobile App Testing?

Mobile testing, or mobile app testing, is the process of checking and validating all aspects of the mobile app to ensure that they work as expected. Any issues and defects found during the process are reported to the dev team to address immediately so that they will not slip into production.
 

In today's digital age, mobile app testing plays an indispensable role in:

1. Ensuring that app defects are timely addressed

Bugs cost money.
 

Modern applications are built from highly interconnected components that must work together seamlessly to deliver the final expected feature. One single bug impacting a crucial component may create a negative ripple effect to other components and break the entire app. 
 

For applications in sensitive fields (medicine, law, finance, insurance, etc.), such bugs can have irreversible consequences. Good mobile app testing catches those bugs and prevents them from slipping into production.

2. Maintain and enhance app quality

The journey of developing a mobile application pose never-ending questions like:

• How can we improve our user experience? 
• How do we know which areas to test? 
• In which area should we optimize and allocate our resources? 
• Is the user feedback on our app valid?

Testing answers those questions, so we can confidently and continuously improve our app thanks to insights from test results. 

What is Manual Mobile App Testing?

When doing mobile app testing manually, human testers interact with the application manually just as an end-user would to check if the application delivers the expected functionality and performance. They don’t rely on help from any automated scripts or tools, but solely use their human capabilities to find bugs.
 

Of course, we are humans, and humans make mistakes. We are not perfect creatures. Certain bugs can only occur due to repetitive and abnormal series of interactions with the system, but we are not built to do things repetitively - it is boring and, most importantly, time-consuming. That’s why automated testing is so much preferred for repetitive scenarios. However, manual testing really shines through for non-repetitive scenarios.

Why and When To Do Mobile Testing Manually?

When a developer manually runs their application and tries out the features they coded, that is manual mobile testing. It would be an overkill to build an entire automation test script for a simple, personal project. Even when working on a large-scale project (say, a mobile app for a global bank with highly secured and feature-rich banking experience), testers may still have to do some manual testing first to identify the areas worth automating.
 

But manual testing is more than just a precursor to automation testing. By nature, testing mobile apps manually come with unique advantages:
 

• Allow for more human inputs: to be human is to be creative. When manually testing a mobile game, being able to creatively explore the game grants testers the necessary liberty to act “like an end-user” and experiment with new ways of finding bugs that automation testing just can’t deliver. Similarly, when testing more abstract aspects of the mobile app that can’t be accurately quantified such as the aesthetics or the ease of navigation, we also need a real human.
• Truly “test” for bugs and not “check” for bugs: after all, automated mobile testing was born to help us perform “repetitive tests”, and those tests follow a predefined set of test steps. In other words, it only “checks” if a certain feature has a certain type of bugs, not “find” new bugs. When a real human manually interacts with the app, they can try out new ways to interact with the system and find bugs that we could have never found out in the first place.
• Lower learning curve: because it is manual, you don’t have to possess a high level of technical expertise and coding skills to start testing. Basically, you only have to think of the most creative ways to “break” the app. This is why teams can get started more quickly with the manual approach.
• Lower maintenance cost: automation test scripts have 1 major weakness: they may have to be revised if the feature being tested undergoes some major change. In the ever-changing Agile environment of modern mobile app development, teams have to make those little tweaks quite frequently, and together they eat up a lot of the team’s bandwidth. On the other hand, when doing manual testing, testers can adapt immediately when a code update occurs. The more unstable the system, the more important manual testing becomes.

Learn How To Do Mobile Testing Without Code

Manual Mobile Testing in Different Test Environments

1. Emulators and Simulators

Emulators and simulators both replicate a mobile device, but emulators replicate both the software and hardware, while simulators replicate only the software part. They have several advantages and disadvantages compared to testing on real mobile devices:
 

Advantages:

1. Cost-Efficiency: Emulators and simulators are considerably more cost-effective compared to purchasing and maintaining a range of physical devices, knowing that there are thousands of possible device - OS - browser combinations to be tested. This approach is especially beneficial for smaller development teams with limited budgets.
2. Availability: These virtual environments are readily available and can be quickly set up, making them a convenient choice for testing during the development cycle.
3. Ease of Setup: Setting up emulators and simulators is often straightforward and integrated into popular development environments like Xcode and Android Studio. This also means easier test result tracking for future data analysis to gain insights into the testing efficiency.
4. Parallel Testing: You can run multiple instances of emulators or simulators simultaneously, allowing for parallel testing across different device configurations and operating system versions.
5. Repeatable and Consistent: Emulators and simulators provide a controlled testing environment, ensuring consistent testing conditions for reproducible results.
    

Disadvantages:

1. Limited Realism: After all, emulators and simulators are just virtual environments, and therefore do not perfectly replicate real devices' hardware/software behavior. There are so many other factors at play when a user interacts with an application in a production environment, leading to potential discrepancies in testing results.
2. Hardware-Specific Issues: Certain features of a mobile application must occur in a real-world context, such as testing for the GPS, camera, sensors, battery life, or network signal strength. These domains may produce less accurate results when performed on emulators or simulators.
3. UI and UX Testing Limitations: Emulators/simulators can indeed be used to perform basic UI/UX tests, but it would be a stretch if we try to capture the feel and responsiveness of real devices. For example, it’s difficult to measure touch sensitivity or the quality of physical buttons without a real device.
    

To test on emulators/simulators, you can:

1. Set up and configure the emulator/simulator for the target platform (e.g., Android or iOS) on your development machine. You can use Android Virtual Device (AVD) Manager for Android or Xcode for iOS emulators. Emulators are more commonly used in Android while simulators are usually specific to iOS.
2. Start the emulator/simulator, which will run a virtualized version of the mobile operating system
3. Install the mobile app you want to test on the emulator/simulator. You can often drag and drop the app's installation file (APK for Android or IPA for iOS) onto the emulator.
4. After that, you can start testing. Use the emulator's interface to interact with the app just like you would on a physical device. You can click, swipe, enter text, and perform any actions you need to test the app's functionality.

Read More: Mobile Emulator vs Simulator vs Real Device: Key Differences

2. Real device cloud

A “real device cloud” is a cloud-based platform providing access to a wide range of physical mobile devices for testing and development purposes. You can choose between a lot of makes, models, OS, screen sizes, and even network configurations on these clouds. Even better, users can access and control those physical devices remotely through the web.
 

Generally speaking, real device cloud gives testers a much more comprehensive testing experience. For mobile applications with a diverse user base, or having specific hardware requirements, this is a great option compared to emulators/simulators.
 

For example, with Katalon TestCloud, you can gain access to mobile test environments to execute your tests without having to buy any physical machines or set up local emulators/simulators. Katalon TestCloud allows you to run tests on mobile browsers as well as for mobile native applications. 
 

To get started, create your free Katalon account. You should get access to the Katalon TestCloud and Katalon Studio.

Katalon Studio allows you to create mobile test cases by drag-and-dropping uilt-in Keywords (which are essentially code snippets) or record your on-screen actions and turn them into a test script. In other words, you don’t need to have coding experience to create tests for mobile apps in Katalon Studio.
 

You can then click on the Run icon to select where you want to run your mobile tests. Choose TestCloud to execute tests on common browsers and operating systems on the cloud.

Here you can choose the specific device, OS, version, and browser you want to run your test on. After everything is fully configured, simply click “Run”.

Real device cloud is invaluable for ensuring the quality, compatibility, and reliability of mobile applications across a wide range of real-world devices and conditions.

Start Mobile Testing With Katalon

3. Test on real devices

Testing on real devices simply means executing your tests on actual physical devices rather than using emulators, simulators, or virtual environments. The obvious advantage of this approach is its level of realism. Real devices provide the most accurate representation of how an application will perform in real-world scenarios.
 

However, access to a wide range of real devices is challenging. Purchasing the devices is only the first stage. Organizations also need to maintain and manage them, and over time this can compound into enormous cost, which is not at all ideal for smaller teams. 
 

Real device testing is also constrained by environmental factors, such as physical device availability, testing space, and the need for manual intervention.

How To Perform Effective Manual Mobile Testing?

1. Set A Clear Plan

Automated mobile testing involves creating test scripts that can be stored in the version control system, and its results are logged with detailed information about each step performed. For manual mobile testing, it is not that simple. Human testers have to manually test then manually document the results in a spreadsheet or a test case management system. This is why having a clear test strategy and test plan is especially important when testing mobile apps manually.
 

In a test plan, determine testing objectives, which allow you to better identify test scenarios and establish the potential test cases. Have a test case template to help you note down your test steps in a structured fashion. This should also help to standardize the testing process and improve test result consistency throughout many iterations. 
 

Many manual mobile testers develop a click path to help them achieve that goal.

2. Have A Detailed Click Path

A detailed click path provides step-by-step instructions for navigating through the mobile app's user interface. This ensures that testers can reproduce the same sequence of actions consistently during testing, which is essential for reliable results.
 

Knowing what you are clicking on and where it is leading you to should also improve your test coverage. Compare your click path and the app architecture to identify areas that have not been adequately tested. The click path also makes it much easier to reproduce the bug, since you can just reverse the path from the breakpoint to the starting point. Developers can use that insight to find the root cause and fix the bug more easily.
 

Read More: Bug Life Cycle in Software Testing

3. Test On Multiple Devices and Environments

It is actually an ambitious endeavor to test all devices and environments there are, so QA teams usually deep-dive into analytics of their user base structure to identify which techstack app users tend to use to identify the devices and environments worth testing.
 

When talking about environments, we are also talking about network conditions. Certain applications provide both online and offline modes. Testing under different network conditions also provides insights into how the app consumes network data, and if any features cause slower-than-average loading time.

4. Choose Complicated Use Cases

Manual testing is the ultimate playground for testers. They don’t really have to follow a path as rigid as automated testing, so they can totally take advantage of this liberty to explore the more complex scenarios. Interestingly, a recent research shows that manual mobile testing catches bugs requiring 3+ interactions from users i.e. complicated bugs, while automated testing can only catches bugs requiring fewer than 3 interactions. 
 

The power of testing manually does not stop there. This is an opportunity to check for UX-related issues, such as unintuitive workflows, confusing interfaces, or unexpected user interactions.

5. Leverage Ad-hoc and Exploratory Testing

Ad-hoc testing and exploratory testing are both popular manual testing techniques allowing testers to go beyond the “happy path” and discover bugs that they have never thought of before. Testers rely solely on their intuition, experience, and existing knowledge of the application to freely “explore” the system, trying combinations of interactions that can potentially lead to a bug. This degree of liberty is much harder to achieve in automation testing, so make sure to utilize it when doing manual mobile testing.

6. Document Testing Results

Finally, as we are doing everything manually, results must also be carefully documented in a spreadsheet for future analysis. For aspects of the bug that can be quantified, make sure to establish a structured framework for better bug understanding, future analysis, and troubleshooting. This is also known as the bug taxonomy.
 

Essentially, in a bug taxonomy, bugs sharing common attributes are organized into predefined categories. Below is a compilation of fundamental bug categories for your consideration:

• Severity (High - Medium - Low impact to system performance/security)
• Priority (High - Medium - Low urgency)
• Reproducibility (Reproducible, Intermittent, Non-Reproducible, or Cannot Reproduce)
• Root Cause (Coding Error, Design Flaw, Configuration Issue, or User Error, etc.)
• Bug Type (Functional Bugs, Performance Issues, Usability Problems, Security Vulnerabilities, Compatibility Errors, etc.)
• Areas of Impact
• Frequency of Occurrence

Best Practices For Testing Mobile Apps Manually

• Choose the right platform: Depending on your target audience and business goals, you may need to develop for Android, iOS, Windows, or web browsers. Each platform has its own development requirements, user base, and market share, so choosing the right one is crucial for reaching your intended audience.
• Understand your target end users: A huge part of testing is checking the technical side of the app, but it is also essential to have a deep understanding of your target audience, including their demographics, preferences, needs, and pain points. Knowing your users grants you knowledge of the features and user interfaces that cater to their specific requirements, resulting in more user-centric testing.
• Prioritize functionality: Not all features of a mobile app are equally important. Functionality is the core of any app, and testing this aspect first confirms that your application is at least working as expected. As manual testing can be more time-consuming than automation testing, allocating resources to this low-hanging fruit is always a smart move. However, make sure to identify areas that can be automated and leverage low-code automation testing tools to help you write better tests faster without having to build a team with strong technical skills. If needed, leverage AI.
• Keep in mind the User Experience (UX): once we have the functionality under check, it is time to move to the UX. UX testing is a highly manual process, so take advantage of the “free-style” nature of manual testing to check for that aspect. Recruit participants who match your user personas. Aim for a diverse group of participants to capture a variety of perspectives. Common methods include usability testing, user interviews, surveys, card sorting, and eye-tracking. Consider both qualitative and quantitative approaches. 
• Don’t forget performance testing: finally, nail your performance testing. Integrate performance monitoring and profiling tools into your app to collect data on resource usage. For example, Android Profiler is a commonly used tool for Android apps. Measure and record the baseline performance of your app under normal conditions, then check various performance scenarios such as typical and peak usage situations.

Mobile Testing FAQs

1. When should testers choose manual testing or automation testing for mobile testing projects?

• Choose manual testing when you want a human tester to interact with the mobile app, explore its user interface, and perform ad-hoc or exploratory testing. Manual testing is best for scenarios where frequent changes occur or when testing new features.
• Choose automation testing when you want to run repetitive tests quickly and consistently. Automation is ideal for regression testing, where you need to ensure that existing functionality isn't affected by code changes. It's also helpful for load testing and running tests on various devices simultaneously. Read More: How To Build A Regression Test Suite?

2. What are types of mobile app testing? 

There are several types of mobile app testing, including:

• Functional Testing: Ensures that the app's features work as intended.
• Usability Testing: Focuses on the user experience, navigation, and ease of use.
• Performance Testing: Evaluates app speed, responsiveness, and resource usage.
• Compatibility Testing: Checks how the app behaves on different devices and operating systems.
• Security Testing: Identifies vulnerabilities and ensures data protection.
• Automation Testing: Involves using scripts to automate test cases.
• Load Testing: Measures app performance under heavy user loads.
• Accessibility Testing: Ensures the app is usable by people with disabilities.

3. How to create an effective mobile testing strategy? 

1. Define clear testing objectives and goals.
2. Identify the target audience and devices.
3. Prioritize testing based on critical features and user scenarios.
4. Develop detailed test plans and test cases.
5. Choose appropriate testing tools and frameworks.
6. Conduct manual and automated testing as needed.
7. Perform regression testing after updates.
8. Continuously gather user feedback and adapt testing accordingly.

4. What are some best mobile testing tools/frameworks?

• Appium: An open-source automation tool for mobile apps.
• Katalon: going beyond just a mobile testing tool, Katalon also allows you to create, manage, execute, and report for tests on web, desktop, and API. It is a comprehensive solution for testing teams of any size. See how Katalon helps your team grow.
• XCTest and XCTestUI: Native testing frameworks for iOS.
• Espresso: A testing framework for Android apps.
• Calabash: An open-source tool for mobile automation.
• Selendroid: A framework for testing Android apps.
• TestComplete: A comprehensive testing tool for mobile and web.