Run Katalon Studio Tests in CI/CD: Jenkins, GitHub Actions, Azure DevOps, and GitLab

What you need to run Katalon Studio tests in CI

Every setup in this guide uses the same four things:

1. A Katalon Studio project in Git. Your CI tool checks out the repository, so the project (including its .prj file, test suites, and test suite collections) needs to be committed. Build your tests in Katalon Studio first and confirm they pass locally.
2. A Katalon Runtime Engine (KRE) license. CI execution needs a KRE license. KRE is the execution add-on that runs Studio tests from the command line, and the Katalon docs list an active KRE license as a requirement for the Jenkins, GitHub Actions, Azure DevOps, and GitLab integrations. The Docker-based setups call for a KRE floating license. See Katalon Runtime Engine (KRE) for license options.
3. A Katalon API key. KRE uses it to verify your credentials and activate the license. Generate it in your Katalon account, then store it as a secret or variable in your CI tool rather than in the config file itself.
4. A test suite or test suite collection. KRE runs suites and collections, not single test cases.

Katalon publishes integrations for many other tools too; see the full list of CI/CD integrations.

How Katalon Runtime Engine runs tests from the command line

KRE runs your Studio project with the katalonc command. A CI job is a script that calls katalonc with the right arguments, so it helps to understand the command before you wire it into a pipeline.

These are the required arguments, from the KRE command-line syntax docs:

Common optional arguments include -retry (how many times to rerun), -statusDelay (seconds between status updates), -executionProfile, -maxFailedTests (stop the run after a set number of failures), and -orgID if you belong to more than one organization with KRE licenses.

Here is the macOS/Linux example from the Get started with KRE docs, run from the KRE Contents folder:

./MacOS/katalonc -noSplash -runMode=console -projectPath="/Users/username/Katalon Studio/Sample healthcare/test.prj" -retry=0 -testSuiteCollectionPath="Test Suites/healthcare-tests - TS_RegressionTestCollection" -browserType="Chrome" -executionProfile="default" -apiKey="" --config -proxy.auth.option=NO_PROXY -proxy.system.option=NO_PROXY -proxy.system.applyToDesiredCapabilities=true

You don't need to write this by hand. In Studio, click Build CMD on the main toolbar to open Command Builder, pick the suite, browser, profile, and API key, then click Generate Command. Paste the result into your CI config.

There are two ways to get KRE onto a CI machine. You can install it on the agent, or you can run it inside the Katalon Docker image (katalonstudio/katalon on Docker Hub), which includes Chrome, Firefox, and Katalon Studio. With the image you don't need Studio or KRE installed on the machine, and you call katalonc.sh instead of katalonc.

Jenkins test automation with Katalon Studio

Jenkins is an open-source automation server with hundreds of plugins. The Jenkins integration overview lists two ways to run Katalon tests in Jenkins: the Katalon Plugin for Jenkins, or the Katalon Studio Docker image. You can also call an installed KRE from a Jenkinsfile.

Option 1: The Katalon plugin for Jenkins

The plugin downloads and deploys KRE for you, so you don't install it on the agent first.

1. In Jenkins, go to Manage Jenkins > Manage Plugins, search the Available tab for the Katalon plugin, and choose Install without restart.
2. Create a Freestyle project. Under Source Code Management, choose Git and enter your repository URL and branch.
3. In Build, click Add build step and choose Execute Katalon Studio Tests.
4. Enter the KRE version to download, or the folder of a version that's already installed.
5. Paste your command arguments, then click Save and Build Now.

The sample command arguments from the docs:

-browserType="Chrome" -retry=0 -statusDelay=15 -testSuitePath="Test Suites/TS_RegressionTest" -apikey=<YOUR_API_KEY>

If you copy a command from Command Builder, remove -noSplash and -runMode=console first; the plugin docs say to leave them out.

Option 2: A Jenkinsfile

If you prefer pipeline as code, create a Pipeline project and set Definition to Pipeline script, or Pipeline script from SCM to read a Jenkinsfile from your repository.

This is the docs-linked sample Jenkinsfile from the Katalon CI samples repository. It runs the test suite inside the Katalon Docker image, then archives the reports and publishes the JUnit results:

pipeline {
    agent any
    stages {
        stage('Test') {
            steps {
                dir('<your-project-folder>'){
                    sh 'docker run -t --rm -v "$(pwd)":/tmp/project katalonstudio/katalon katalonc.sh -projectPath=/tmp/project -browserType="Chrome" -retry=0 -statusDelay=15 -testSuitePath="Test Suites/TS_RegressionTest" -apiKey=<your-api-key>'
                }
            }
        }
    }
    post {
        always {
            archiveArtifacts artifacts: 'Reports/**/*.*', fingerprint: true
            junit 'Reports/**/JUnit_Report.xml'
        }
    }
}

For the Docker route, install the Docker plugin and Docker Pipeline plugins in Jenkins and make sure Jenkins can find the docker command on its PATH. On a Windows agent, replace sh with bat.

If KRE is installed on the agent instead, the Jenkinsfile changes into the KRE folder and calls katalonc directly, with sh on macOS or Linux and bat on Windows. The Jenkinsfile docs have both versions. To see the run, open the build and choose Console Output.

GitHub Actions testing with Katalon Studio

For GitHub Actions testing, Katalon publishes the Katalon Studio GitHub Action on the GitHub Marketplace. Before you start, add your Katalon API key to your repository's encrypted secrets.

Create a .yml file in the .github/workflows directory of your repository. This is the sample from the Katalon Studio GitHub Action docs:

name: CI Sample
on:
  push:
    branches: [ "main" ]
  pull_request:
    branches: [ "main" ]
jobs:
  build:
    runs-on: macos-latest
    steps:
    - name: Checkout
      uses: actions/checkout@v4
    - name: Katalon Studio GitHub Action
      uses: katalon-studio/katalon-studio-github-action@v4.0
      with:
          version: '${{ your_KRE_version }}'
          projectPath: '${{ github.workspace }}'
          args: '
            -noSplash 
            -retry=0 
            -testSuitePath='${{ KS_path_to_testSuite }}'
            -browserType="Chrome (headless)"
            -apiKey= ${{ your_API_key_from_TestOps }}
            --config -webui.autoUpdateDrivers=true
            '

Replace the placeholders before you commit:

• version: the KRE version to run.
• projectPath: leave it as ${{ github.workspace }}.
• -testSuitePath: your suite, for example Test Suites/Simple Test Suite. To run a collection, use -testSuiteCollectionPath instead.
• -apiKey: a reference to the secret that holds your API key, not the key itself.

The workflow runs on every push and pull request to main. Change the on: triggers to suit your branching.

Azure DevOps testing with Katalon Studio

For Azure DevOps testing, install the Katalon for Azure DevOps extension from the Visual Studio Marketplace. It downloads, deploys, and runs KRE in your pipeline.

1. Install the extension in your Azure DevOps organization.
2. Open your project, go to Repos > Files, and click Set up build. Choose Starter pipeline.
3. Click Show assistant, search for the Execute Katalon Studio Tests task, and enter the KRE version and your command arguments.
4. Click Add to insert the task, then save and run the pipeline.

Store your API key as a pipeline variable named katalon_api_key, so the arguments can reference it as $(katalon_api_key). This is the Windows sample azure-pipelines.yml linked from the Azure DevOps integration docs:

trigger:
- master

pool:
  vmImage: 'windows-latest'

steps:
- task: katalonTask@1
  displayName: 'Execute Katalon Studio project'
  inputs:
    version: '8.2.5'
    executeArgs: '-browserType="Chrome" -retry=0 -statusDelay=15 -testSuitePath="Test Suites/TS_RegressionTest" -apiKey=$(katalon_api_key)'

Set version to the KRE version you use. Two platform notes from the docs:

• Windows agents render at 1024 x 768 by default. If your app needs a larger screen, add the Screen Resolution Utility extension and use the azure-pipelines-windows-srs.yml sample.
• Linux agents use Xvfb-run for UI tests. Leave the X11 DISPLAY field blank and set the resolution in the Xvfb-run field, for example 1024x768x24.

GitLab CI testing with Katalon Studio

For GitLab CI testing, run your job inside the Katalon Docker image. You need a Katalon Studio project in GitLab, GitLab Runners for the project (self-hosted runners need Docker installed), and a KRE floating license.

First, add your API key as a CI/CD variable. Go to Settings > CI/CD, expand Variables, and add a variable with the key KATALON_API_KEY and your API key as the value.

Then edit .gitlab-ci.yml in the pipeline editor (CI/CD > Editor). This is the sample from the GitLab integration docs, without its comment lines:

image: 'katalonstudio/katalon'
services:
  - docker:dind
variables:
  KATALON_PROJECT_DIR: $CI_PROJECT_DIR
stages:
  - test
test_job:
  stage: test
  script:
    - katalonc.sh -projectPath=$KATALON_PROJECT_DIR -apiKey=$KATALON_API_KEY -browserType="Chrome" -retry=0 -statusDelay=20 -testSuitePath="Test Suites/TS_RegressionTest"

The image line uses the latest Katalon Docker image by default. To pin a version, add a tag, for example katalonstudio/katalon:8.1.2. The Katalon Docker image docs recommend the 11-latest-slim tag for production, and the major version in the tag should match your major version of Studio. If your project isn't in the repository root, update KATALON_PROJECT_DIR.

Push a change to trigger the pipeline, then open CI/CD > Jobs to see the job log and the katalonc output.

Headless testing in CI

Headless testing runs the browser without showing its window. CI agents often have no display, so headless runs are the usual choice there. The Katalon docs describe headless mode as quicker to run and suited to CI/CD.

To run headless, set the browser type to a headless option, as the GitHub Actions sample does:

-browserType="Chrome (headless)"

A few points to check:

• Linux agents: KRE on Linux supports Chrome, Firefox, and Remote only.
• Window size: if layouts break in headless mode, set a window size in Project > Settings > Desired Capabilities > WebUI > Chrome (headless) (or Firefox (headless)), for example args with --window-size.
• Non-headless runs on Linux: if you need a visible browser, run it under a virtual display. That's what Xvfb-run does in the Azure DevOps Linux setup.

Run test cases in parallel within a test suite

From Katalon Studio 11.4.0, you can run the test cases in one test suite at the same time instead of one after another. This needs a Katalon Studio Enterprise license and the Enable New Execution Viewer setting, which is on by default.

To set it up:

1. Open the test suite and expand Execution Information.
2. In the Parallel Execution group, select Enable Parallel Execution.
3. Set Max Concurrent Instances, the number of test cases to run at once. The default is 1.
4. Save the test suite.

The setting is stored per test suite and applies to every execution environment except Katalon TestCloud, which handles parallel runs separately. Before you turn it on for a CI suite, make each test case self-contained. Test cases that depend on run order, share a browser session, or write to shared variables can fail in parallel. Only the Retry after executing all option applies to a parallel run.

Reports and exit codes: fail the build on failed tests

A CI job is only useful if it tells you when tests fail. KRE gives you two signals: an exit code and report files.

Exit codes

When katalonc finishes, it returns one of these exit codes:

When katalonc runs as a shell step (sh or bat in a Jenkinsfile, or a script line in .gitlab-ci.yml), Jenkins and GitLab CI mark the step as failed on any nonzero exit code, so failed tests fail the build without extra configuration. Exit codes 4 to 7 point to setup problems, not test failures, so check the job log before you look at the tests. To stop a long run early, add -maxFailedTests=<number>.

Reports

KRE writes reports to the Reports folder in your project, and Katalon Studio can export them as HTML, CSV, PDF, and JUnit. To publish them from CI:

• Jenkins: the sample Jenkinsfile archives everything under Reports/ and publishes JUnit_Report.xml with the junit step, so results show in the build page.
• Azure DevOps: the Linux sample pipeline publishes the report/ folder as a pipeline artifact.
• Any tool: set -report.folderPath=<path> (Katalon Studio 11.1.2 and later) to copy reports to a fixed folder your CI tool collects. It replaces the older -reportFolder argument.

To keep results from every CI tool in one place, connect the project to Katalon TestOps, part of Katalon True, and view the test reports there whichever CI tool started the run.

Where to go next

Once one suite runs in CI, add more. One option is a short smoke suite on every pull request and a full regression suite on a schedule. For more on running suites, see our test execution guide. To compare CI servers before you pick one, read our roundup of CI/CD tools. And if you're building browser tests to run in these pipelines, see AI web testing with Katalon.