Skip to content

Testkube Test and Test Suite Scheduling

In order to run Tests and Test Suites on a regular basis, we support a scheduling mechanism for these objects. CRDs both for test and test suite contain a schedule field used to define rules for launching them in time. Testkube's schedule data format is the same that is used to define Kubernetes Cron jobs (check Wikipedia Cron format for details https://en.wikipedia.org/wiki/Cron).

Scheduling Architecture

Testkube uses the scheduling engine from Kubernetes Cron jobs. In fact, for each scheduled Test or Test Suite, a special cron job is created from this template: https://github.com/kubeshop/helm-charts/tree/main/charts/testkube-api/cronjob-template.yml. Technically, it is a callback to Testkube API server method launching either Test or Test Suite execution. This works similarly to scheduled Test and Test Suite executions done by external scheduling platforms.

Create a Test with a Schedule

Let's create a test with a required schedule using Testkube CLI command:

kubectl testkube create test --file test/postman/TODO.postman_collection.json --name scheduled-test --schedule="*/1 * * * *"

Output:

Detected test type postman/collection
Test created  / scheduled-test 🥇

We successfuly created a scheduled test and can check a list of the available tests:

kubectl testkube get tests

Output:

  NAME              | TYPE               | CREATED                       | LABELS | SCHEDULE    | STATUS | EXECUTION ID              
+-------------------+--------------------+-------------------------------+--------+-------------+--------+--------------------------+
  scheduled-test    | postman/collection | 2022-04-13 12:37:40 +0000 UTC |        | */1 * * * * |        |                           

The scheduled test was created but it was not yet executed.

Run a Scheduled Test

In order to start the execution of the test on a defined schedule, use the Testkube CLI command:

kubectl testkube run test scheduled-test

Output:

Type          : postman/collection
Name          : scheduled-test



Test queued for execution

Use the following command to get test execution details:
$ kubectl testkube get execution 

Test was successfully scheduled for execution and put into the queue. Let's check a Cron job connected to this test.

kubectl get cronjobs -A

Output:

NAMESPACE   NAME                   SCHEDULE      SUSPEND   ACTIVE   LAST SCHEDULE   AGE
testkube    scheduled-test-tests   */1 * * * *   False     1        42s           3m22s

The Cron job for this test was successfully created and the test was executed. To the check Cron job details:

kubectl describe cronjob scheduled-test-tests -n testkube

Output:

Name:                          scheduled-test-tests
Namespace:                     testkube
Labels:                        testkube=tests
Annotations:                   <none>
Schedule:                      */1 * * * *
Concurrency Policy:            Forbid
Suspend:                       False
Successful Job History Limit:  3
Failed Job History Limit:      1
Starting Deadline Seconds:     <unset>
Selector:                      <unset>
Parallelism:                   <unset>
Completions:                   <unset>
Pod Template:
  Labels:  <none>
  Containers:
   curlimage:
    Image:      curlimages/curl
    Port:       <none>
    Host Port:  <none>
    Command:
      sh
      -c
    Args:
      curl -X POST -H "Content-Type: application/json" -d '{}' "http://testkube-api-server:8088/v1/tests/scheduled-test/executions?callback=true"
    Environment:     <none>
    Mounts:          <none>
  Volumes:           <none>
Last Schedule Time:  Wed, 13 Apr 2022 15:50:00 +0300
Active Jobs:         scheduled-test-tests-27497570
Events:
  Type    Reason            Age                  From                Message
  ----    ------            ----                 ----                -------
  Normal  SuccessfulCreate  5m41s                cronjob-controller  Created job scheduled-test-tests-2749757

As mentioned above, we have a scheduled callback for launching our test.

Getting Scheduled Test Results

Now we can check whether the test is executed every minute for the schedule we provided.

kubectl testkube get execution

Output:

  ID                       | NAME                | TYPE               | STATUS  | LABELS  
+--------------------------+---------------------+--------------------+---------+--------+
  6256c98f418062706814e1fc | scheduled-test      | postman/collection | passed  |         
  6256c953418062706814e1fa | scheduled-test      | postman/collection | passed  |         
  6256c91e418062706814e1f8 | scheduled-test      | postman/collection | passed  |         
  6256c8db418062706814e1f6 | scheduled-test      | postman/collection | passed  |         
  6256c89f418062706814e1f4 | scheduled-test      | postman/collection | passed  |         
  6256c885418062706814e1f2 | scheduled-test      | postman/collection | passed  |         
  6256c87e418062706814e1f0 | scheduled-test      | postman/collection | passed  | 

The test is successfully regulary executed.

Create a Test Suite with a Schedule

Let's create a Test Suite with a required schedule using the Testkube CLI command:

cat test/suites/testsuite.json | kubectl testkube create testsuite --name scheduled-testsuite --schedule="*/1 * * * *"

Output:

TestSuite created scheduled-testsuite 🥇

We successfuly created a scheduled Test Suite and can view a list of the available Test Suites:

kubectl testkube get testsuites

Output:

  NAME                | DESCRIPTION            | STEPS | LABELS | SCHEDULE    | STATUS | EXECUTION ID  
+---------------------+------------------------+-------+--------+-------------+--------+--------------+
  scheduled-testsuite | Run test several times |     2 |        | */1 * * * * |        |    

The scheduled test suite was created, but it was not yet executed.

Run a Scheduled Test Suite

In order start the execution of the test suite on a defined schedule, we need to run it using Testkube CLI command:

kubectl testkube run testsuite scheduled-testsuite

Output:

Name          : scheduled-testsuite
Status        : queued


Test Suite queued for execution

Use the following command to get test execution details:
$ kubectl testkube get tse 

The Test Suite was successfully scheduled for execution and put into the queue. We will skip the Cron job details, they are fully similar to test one described above.

Getting Scheduled Test Suite Results

The Test Suite is executed every minute for the schedule we provided.

kubectl testkube get tse

Output:

  ID                       | TEST SUITE NAME     | EXECUTION NAME                             | STATUS | STEPS | LABELS  
+--------------------------+---------------------+--------------------------------------------+--------+-------+--------+
  6256ce3f418062706814e210 | scheduled-testsuite | scheduled-testsuite.abnormally-in-lark     | passed |     2 |
  6256ce04418062706814e20c | scheduled-testsuite | scheduled-testsuite.kindly-evolved-primate | passed |     2 |
  6256cdcc418062706814e208 | scheduled-testsuite | scheduled-testsuite.formerly-champion-dodo | passed |     2 |

The Test Suite is successfully executed according to the schedule set.