Modern applications are expected to be up 24/7, and that puts certain pressure on the performance testing team. One of their goals is to continuously monitor real users’ interactions with the site and develop a performance benchmark based on that information.        
 

However, there’s a catch: sometimes real user monitoring doesn't allow you to observe the system's behavior in extreme scenarios (such as sudden spike in traffic or abnormal user behaviors). After all, those are extreme scenarios, and they don’t happen that often. That’s why synthetic monitoring is created to fill in the gap that real user monitoring can’t provide.        
 

In this article, we will dive deep into the concept of synthetic monitoring, present the key components of this approach, as well as best practices you should follow to better implement it. You can even jump straight to our comprehensive guide on how to do synthetic monitoring.

What is Synthetic Monitoring?

Synthetic monitoring is a process where testers create automated scripts simulating real users’ interactions with the application or website to proactively detect performance issues and assess overall system health. These automated scripts are called synthetic transactions, and they help testers gauge the system’s performance when real user data is not yet available, or too challenging to capture.

Benefits of Synthetic Monitoring

Real user data doesn't always tell us the full picture for the software’s performance. It is admittedly great to have data captured directly from real users to provide actual user experience metrics, but what if the feature is still under development and not yet released for real users? How can you proactively capture the data to build a performance benchmark for it?       
 

Even if the feature is already released, user activity tends to be quite limited in the early stage. Performance testing teams have no choice but to passively wait until enough data has been gathered so their test results can be statistically significant.       
 

Similarly, to ensure the highest level of coverage, testers also need to look at more extreme cases, such as sudden traffic spikes or server outages, and you don’t usually have data for such outlier scenarios. To know how the system behaves under such pressures, testers can create synthetic transactions to mimic large volume users and send them to the website and document the results.       
 

When it comes to dependencies, the need for synthetic monitoring becomes even more pronounced. There are so many variables at play that can mess with the final test result. Having a controlled environment exclusively built for the testing of that specific feature is a must if you want the test results to be more reliable. It also reduces the risk of unintentionally breaking other features.        
 

In short, you need synthetic monitoring for two primary reasons:

1. Real user data is not yet available, or difficult to acquire.
2. Testers want to have greater control and less risk over the testing process.       
    

There are so many other benefits of synthetic testing, not because of its synthetic nature, but rather because of its automated nature:

1. Scheduled testing: You can schedule your synthetic transactions to run at specific times, preferably during peak usage periods, maintenance windows, or any time periods suitable for your test strategy.
2. Improved analytics: Historical data generated by synthetic monitoring allows you to dive deep into the testing effort and uncover underlying patterns which you can turn into valuable insights for future decision-making.
3. Customization: Synthetic monitoring can be tailored to your specific testing requirements, allowing you to send traffic on specific journeys through the website to address your unique testing needs.

Synthetic Monitoring vs. Real User Monitoring

Have a look at the table below to better understand the differences between synthetic monitoring and real user monitoring:       
 

Examples of Synthetic Monitoring

Have a look at some examples for synthetic monitoring below. You should notice a pattern: all of them have a fairly specific set of requirements that would be quite challenging to capture if you go with real user monitoring:

1. Regularly test the load time of your website's homepage from different geographic locations.
2. Simulate user searches for products on your e-commerce site to confirm that search results load quickly and accurately.
3. Automate the registration process by creating test accounts to verify that users can successfully register and log in.
4. Test the submission of forms on your website to ensure that data is processed correctly and validation messages appear when necessary.
5. Simulate the entire e-commerce checkout process from adding items to the cart, selecting shipping options, to completing a purchase.
6. Create synthetic tests that send HTTP requests to specific API endpoints to measure response times and identify potential issues.
7. Automate user logins, providing credentials and checking that the authentication process is secure and efficient.
8. Automate the launch of your mobile app and simulate various user interactions like screen transitions, selections, and form inputs to ensure a smooth user experience.
9. Generate specific error conditions to verify that error pages are correctly displayed and provide useful information to users.
10. Monitor images/videos to ensure they are available and load quickly for users.

A Comprehensive Guide to Do Synthetic Monitoring

Before you start, make sure you choose the right approach:

• Write a test script with an automation testing framework (Selenium is a good example. You also need to install Node.js and WebDriver for the specific browser you want to run the test script, such as ChromeDriver).
• Write test scripts with the help of an automation testing tool. Installing such a tool is more straightforward, and you can choose from the list of the top automation testing tools on the market here.       
   

Each tool comes with their own advantages that you should take into consideration. Writing your own test scripts grants a great degree of customization, but the downside is that you have to update all of them when updates are rolled out. Automation testing tools, on the other hand, often come with robust features that support you throughout the entire testing life cycle, but you need some initial investment to get access to the tool.

1. Synthetic Monitoring With Selenium

Scenario: In this example, we'll create a Selenium test script in JavaScript to automate the process of checking a website's homepage for its availability and response time.       
 

Dependencies:

1. Node.js: Download here
2. Selenium WebDriver: You'll need the Selenium WebDriver library for Node.js, which you can install using npm (Node Package Manager). Open your terminal or command prompt and run the following command:

npm install selenium-webdriver
 

For ChromeDriver, also run the following command in your terminal:

npm install chromedriver
 

Test steps:

1. Open a web browser.
2. Navigate to the website's homepage.
3. Measure the response time of the homepage.
4. Verify that the page loads successfully (HTTP status code 200).
5. Close the browser.       
    

Test script:

The following script performs the test steps we mentioned above. Make sure to change the URL from https://www.example.com to the website you want to test.       
 

const { Builder, By, Key, until } = require('selenium-webdriver');
const assert = require('assert');
(async function () {
  // Set up the Selenium WebDriver
  const driver = await new Builder().forBrowser('chrome').build();
  try {
    // Navigate to the website's homepage
    await driver.get('https://www.example.com'); // Replace with the URL you want to monitor
    // Measure the response time
    const startTime = Date.now();
    // Wait for the page to load (e.g., by checking for an element on the page)
    await driver.wait(until.elementLocated(By.id('someElement')), 10000); // Adjust the element and timeout as needed
    const endTime = Date.now();
    const responseTime = endTime - startTime;
    // Verify the page loads successfully (HTTP status code 200)
    const statusCode = await driver.executeScript('return window.performance.timing.responseStart');
    assert.equal(statusCode, 200, 'Expected HTTP status code 200');
    // Log the response time
    console.log(`Response time: ${responseTime} ms`);
  } catch (error) {
    console.error(`Test failed: ${error.message}`);
  } finally {
    // Close the browser
    await driver.quit();
  }
})();
 

That was just a basic example of how to use Selenium WebDriver to measure the response time of a web page and check if it loads successfully. Make sure that you understand the pros and cons of synthetic monitoring with Selenium:       
 

2. Synthetic Monitoring With Katalon

For automation testing tools, the process of synthetic monitoring is much more straightforward. You don’t have to script much thanks to all of the low-code test authoring features (such as Record-and-Playback, which essentially records your on-screen activities and turns that sequence into a code snippet that you can freely execute on environments of your choice.        
 

Automation testing platforms take this to an even higher level when centralizing all stages of the testing life cycle, so that you can plan your synthetic monitoring, write scripts based on that plan, schedule them to be executed locally or on-cloud for web, desktop, mobile, and even API, then generate detailed reports to help you understand your testing effectiveness, all in one place.

Here’s how you can do synthetic monitoring with Katalon - a comprehensive software testing platform for web, desktop, API, mobile:

1. Create a new test case or test suite in Katalon Studio.
2. Use the built-in WebUI automation capabilities to create scripts that simulate user interactions with your web application.
3. Set up the tests to run at scheduled intervals, such as every hour or every day.
4. Use Katalon Analytics to view the results of your tests and identify performance issues.
5. Integrate Katalon Studio with APM vendors.       
    

Step 1: Sign up and download Katalon Studio. This will primarily be where you create your test cases.

Download Katalon & Start Testing Within Minutes       
 

Step 2:  Once you have downloaded Katalon Studio, launch it. You should see the Katalon Studio interface as below. You can then click the “Create new Test Case” button.

Let’s call this test case “synthetic_monitoring”. After you name it and write some description, click OK.

You are now ready to create your test case. Katalon offers a wide variety of keywords (which are essentially automation code snippets for specific actions) that you can use to craft a full on test case. 

Below is a snapshot of our keyword list—quite plenty for all of your synthetic monitoring needs, and there are hundreds to choose from. You can even create your own custom keywords or leverage the Record-and-Playback feature to record your on-screen actions and turn that sequence into a full script that can be executed across environments. 

But it's not just about quickly creating test cases for web (and also desktop, mobile, or API), without having to write any code, but also about scheduling, executing, and viewing reports for them in one place. Here you can see how to schedule a specific test case and test suite to run on specific elements, at what time, and even what intervals.

After that, you can check the results in Katalon Analytics, with detailed information on passed/failed tests, before diving deep into the granularity. 

Start Testing With Katalon       
 

Read More: Enabling Synthetic Monitoring with Katalon

Challenges of Synthetic Monitoring

• Limited visibility: Be aware of your assumptions for user behavior. Synthetic monitoring is usually based on speculations on what your users do, especially in earlier stages of testing where usage data is not yet available. Can you be sure that they are representative enough for real-world usage? Is it too simplistic? Also, you should note that user behavior is not static: it evolves over time. That is why you need to complement synthetic monitoring with some real-user monitoring to add some realism.
• Adjustment for growth: As your website grows over time, it is essential to also adjust your performance benchmark accordingly. What was seen as “traffic spikes” in your outdated test script can actually become “normal traffic” after a long enough period of time. If the assertions in your test scripts are not frequently updated, you will see false positives being reported. These script maintenance tasks can occur alongside infrastructure upgrades.
• Scalability: As your organization grows, your synthetic monitoring infrastructure also needs to evolve accordingly. This introduces additional costs and technical challenges.

Frequently Asked Questions

1. What is the synthetic monitoring process?

The synthetic monitoring process involves creating automated test scripts that simulate user interactions with a web application or service. These scripts are executed at regular intervals from predefined locations or nodes, measuring performance metrics and identifying issues like latency, errors, and downtime.

2. What is synthetic monitoring also known as?

Synthetic monitoring is also known as "active monitoring" or “proactive monitoring.”

3. What is synthetic monitoring also known as?

Synthetic monitoring is a type of active monitoring. Active monitoring broadly refers to any monitoring method that involves actively generating traffic or probes to assess the performance and availability of a system. Synthetic monitoring specifically involves creating and running scripted tests to mimic user interactions.

4. What is synthetic monitoring also known as?

1. Ping monitoring: This involves sending Internet Control Message Protocol (ICMP) ping requests to check if a network host or server is reachable.
2. HTTP/HTTPS monitoring: It tests web services and websites by sending HTTP requests to verify their availability, response times, and content.
3. TCP/UDP monitoring: This type of monitoring assesses the status and performance of network services by establishing and monitoring TCP or UDP connections.
4. DNS monitoring: It focuses on the Domain Name System (DNS) and involves sending DNS queries to check the resolution of domain names to IP addresses.