Newman

The JUnit check type runs a new kubernetes pod with the specified image, in this example we are running a postman collection test suite using the newman cli.

apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
  name: newman-junit
spec:
  interval: 120
  severity: high
  junit:
    - testResults: "/tmp/"
      name: junit-newman
      display:
        template: |
          ✅ {{.results.passed}} ❌ {{.results.failed}} in 🕑 {{.results.duration}}
          {{  range $r := .results.suites}}
          {{- if gt (conv.ToInt $r.failed)  0 }}
            {{$r.name}} ✅ {{$r.passed}} ❌ {{$r.failed}} in 🕑 {{$r.duration}}
          {{- end }}
          {{- end }}
      spec:
        containers:
          - name: newman
            image: ghcr.io/flanksource/canary-newman:latest
            command: ["/start.sh"]

By configuring newman to export JUnit to the testResults folder, canary-checker will pick up the results and make then available display formatting and health evaluation.

start.sh
set -x
mkdir -p /tmp/junit-results
newman run newman.json --reporters junit --reporter-export /tmp/junit.xml
cp /tmp/*.xml /tmp/junit-results/
touch /tmp/junit-results/done

For a complete working example, see canary-checker-examples/newman

Field	Description	Scheme
name*	Name of the check, must be unique within the canary	string
spec*	Pod specification	Spec
testResults*	Path to JUnit test results	string
artifacts	Specify what files/folders created in the pod need to be archived	[]Artifact
timeout	Timeout in minutes to wait for specified container to finish its job. Defaults to 5	integer
description	Description for the check	string
display	Expression to change the formatting of the display	Expression
icon	Icon for overwriting default icon on the dashboard	Icon
labels	Labels for check	map[string]string
metrics	Metrics to export from	[]Metrics
test	Evaluate whether a check is healthy	Expression
transform	Transform data from a check into multiple individual checks	Expression

Test Result Variables

See JUnit Test Results for the schema that is ingested and can be used for evaluating health or formatting the display.