Using Testcontainers in a Kubernetes Jenkins pod

2-minute read
Development • hugo
hugo • gradle

This post will show you how to run testcontainers in Jenkins on Kubernetes.

Setting up a Kubernetes cluster and Jenkins is outside the scope of this article.

Testcontainers

Testcontainers is a Java library that makes it easy to create integration tests with external dependencies. Instead of creating and managing new instances of databases, queue systems etc. which your application depends on, you can let testcontainers do the heavy lifting for you. It will create and manage throw-away Docker containers, and you can set up your tests to use those instead. Very highly recommended!

Jenkins on Kubernetes

For a new application I have been working on, I have set up a simple Kubernetes cluster, and installed Jenkins on it. The application source repo is in Gitlab, and I have set up a Webhook to start the Jenkins build when ever I push changes to the code.

As a part of the project I have also created a Jenskinsfile which Jenkins can use to create a new Kubernetes pod for each build.

A rough template looks like:

def label = "worker-${UUID.randomUUID().toString()}"

podTemplate(label: label, containers: [
            containerTemplate(name: 'jdk',
                    image: 'adoptopenjdk/openjdk11:jdk-11.0.6_10-ubuntu-slim',
                    command: 'cat',
                    ttyEnabled: true),
        ],
        volumes: [
                hostPathVolume(mountPath: '/home/gradle/.gradle',
                        hostPath: '/tmp/jenkins/.gradle'),
        ]) {

    node(label) {
            stage('Check') {
                container('jdk') {
                    sh """./gradlew --no-daemon check"""
                }
            }
    }
}

This creates a pod based on the template, using the openjdk11 image for the Docker container in the pod. The project supplies a Gradle wrapper script, so that will be installed on-the-fly. Not optimal, but also not relevant just now.

Running the tests

The project also uses testcontainers to handle the database which the application needs. On my local development environment, this just works. The Gradle check task will run the project tests, and the integration tests that access the database pass correctly.

When I pushed this to Gitlab, and the Jenkins build ran, the tests were failing. This happens because I am using testcontainers inside a Docker container, as described here.

Once I knew what the issue was, the fix was relatively simple: Add the Docker socket as a volume mount:

def label = "worker-${UUID.randomUUID().toString()}"

podTemplate(label: label, containers: [
            containerTemplate(name: 'jdk',
                    image: 'adoptopenjdk/openjdk11:jdk-11.0.6_10-ubuntu-slim',
                    command: 'cat',
                    ttyEnabled: true),
        ],
        volumes: [
                hostPathVolume(mountPath: '/home/gradle/.gradle',
                        hostPath: '/tmp/jenkins/.gradle'),
                hostPathVolume(mountPath: '/var/run/docker.sock',
                        hostPath: '/var/run/docker.sock')
        ]) {

    node(label) {
            stage('Check') {
                container('jdk') {
                    sh """./gradlew --no-daemon check"""
                }
            }
    }
}

With that change the tests were running correctly in the Jenkins build as well.