What device should be used to run the tests? What operating system version, manufacturer and model to use? How to get these devices?
These are some of the questions we ask ourselves when planning a mobile application testing strategy. If we consider automating these tests, other questions will surely come to mind.
Designing and executing automated tests for mobile devices, both for native applications and responsive web apps, pose technical and non-technical challenges, many of them related to the devices and platforms on which the tests will be run.
In automation processes, it is essential to have integrated platforms that cover the entire test cycle, like Katalon, which provide capabilities for automating tests, simplifying test execution setup, and providing solutions for multi-device operation.
Every day, new models of mobile devices are launched on the market, featuring new operating system versions, new manufacturers and technologies, wider resolutions and screen sizes, and more. End customers use them to download different apps, so they are ideal devices to carry out the tests.
So, how to ensure apps will run smoothly on these devices? What options are available to access such a range of devices? How much time will it take to make them available, configure and run tests on them? What will happen to those devices when the testing phase ends? What costs will be associated with their purchase and maintenance?
These are some questions that arise, and each one in itself represents a challenge to be addressed.
As an alternative to some of the previous questions, tests could be performed by emulating the devices required for the automated tests, configuring the necessary emulators for execution and, once the tests are completed, proceeding to discard them. At first glance, it looks like a good choice to avoid purchasing real devices and minimize automated test setup and execution efforts.
But emulators have an extremely important disadvantage that affects the quality of our work. An emulator is not a real device, but rather a piece of software that runs the app as if it were on a mobile device. Therefore, there is always a risk that the applications under test behave differently on an emulated device than on a real device.
Read more: Mobile Emulator vs Simulator vs Real Device Testing Differences
The answer to the questions and challenges mentioned before might be up in the cloud. With the advantages of PaaS (Platforms as a Service), the cloud provides the ability to access a large and diverse collection of devices from anywhere in the world.
These real mobile device platforms in the cloud are the solution to the above challenges:
Katalon TestCloud is one of the most recent products developed by Katalon for the execution of cloud-based automated tests. TestCloud allows you to run tests of a Katalon project on a wide range of browsers, operating systems and mobile devices.
Katalon TestCloud allows you to:
TestCloud is integrated into the Katalon platform. From this platform, test executions (Test Runs) are managed and the devices where the tests will be executed (Environments) are assigned.
To start, you must schedule a Test Run, select the project and the test suite to be executed, and then access the “Environment ”section.
From here it is possible to select the “TestCloud” option and then choose among the devices available.
As shown in the previous images, the Katalon platform provides details of all the devices available to run the tests. By selecting multiple devices, the same tests can be run on all of them, either sequentially or simultaneously.
When confirming the execution, TestCloud will run the test suite on each selected device. You can view the execution log for each device in real time and also access test information and results, and even export them to popular formats such as CSV, HTML and PDF.
Although we have only mentioned TestCloud's features so far, Katalon has always stood out for its high integration capabilities with other tools and platforms. It provides the ability to integrate and run automated tests on other mobile device providers in the cloud, such as BrowserStack, SauceLabs and Kobiton. This feature provides Katalon users with the possibility of having several alternatives when running mobile tests through the cloud, which makes Katalon a versatile and highly extensible tool.
Since TestCloud is a solution developed and maintained by Katalon, it has certain advantages when dealing with Katalon projects.
Mobile test automation poses several challenges with different levels of technical complexity, such as the need to configure, maintain and manage the devices where tests are to be run. To get an idea of this, consider how many different manufacturers, models, screen resolutions, and operating systems there are today.
The use of emulators is a risky solution, as the behavior of the system under test may vary depending on whether it is a real device or an emulator.
The cloud -a better alternative- allows us to access a wide range of real devices, avoiding acquisition, maintenance and/or management costs.
Through the Katalon platform, Katalon TestCloud allows access to more than 300 different real devices, both Android and iOS. It provides a high-level interface for the search and selection of devices, supports simultaneous runs, and offers information and reports showing the results for each device.