How to Simplify Docker Container Management in Integration Tests Using Pytest-Docker

Introduction

Today, we're going to discuss an efficient way to set up your automated integration test workflows using Docker and Pytest.

Motivation

In an era where software complexity is rapidly growing, developing an efficient testing strategy is more than a mere best practice — it's the bedrock of stable and reliable applications.

One common strategy involves using Docker to set up a test version of a software component, such as a vector database. There are two popular variations to this strategy, both of which have their own limitations.

The Common Approaches

Approach 1: Manual Intervention

In this approach, the developer is expected to bring up the required containers before running a test suite.

For instance, a developer in the Haystack project aiming to test the weaviate integration has to execute the following command first:

docker run -d -p 8080:8080 \
    --name haystack_test_weaviate \
    --env AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED='true' \
    --env PERSISTENCE_DATA_PATH='/var/lib/weaviate' \
    --env ENABLE_EXPERIMENTAL_BM25='true' \
    semitechnologies/weaviate:1.14.1

But picture this: You're a developer already in The Zone, and your code is flowing like a symphony. Out of the blue, your integration tests explode into a sea of red failures. The oversight? You overlooked starting the Docker container🤦‍♂️🤣.

It's clear - relying on manual processes is a recipe for hiccups. Automation isn't just a fancy perk; it's a need in our quest for smooth, error-free testing workflows.

Approach 2: Boilerplate Overload

The second approach attempts to automate the Docker container's lifecycle with fixtures. The developer writes the code that handles the setup and teardown of the required containers within the test suite.

For example, the Docarray project has the following fixtures to test their integration with weaviate:

HOST = "http://localhost:8080"


cur_dir = os.path.dirname(os.path.abspath(__file__))
weaviate_yml = os.path.abspath(os.path.join(cur_dir, 'docker-compose.yml'))


@pytest.fixture(scope='session', autouse=True)
def start_storage():
    os.system(f"docker-compose -f {weaviate_yml} up -d --remove-orphans")
    _wait_for_weaviate()

    yield
    os.system(f"docker-compose -f {weaviate_yml} down --remove-orphans")


def _wait_for_weaviate():
    while True:
        try:
            response = requests.get(f"{HOST}/v1/.well-known/ready")
            if response.status_code == 200:
                return
            else:
                time.sleep(0.5)
        except requests.exceptions.ConnectionError:
            time.sleep(1)


@pytest.fixture
def weaviate_client(start_storage):
    client = weaviate.Client(HOST)
    client.schema.delete_all()
    yield client
    client.schema.delete_all()

Now Picture this: You're about to kickstart a project, and you know you should be writing tests because, well, that's the best practice. But the mere thought of wading through the docs and assembling the necessary scaffolding feels as daunting as building an IKEA furniture without a manual. So, you tell yourself, 'I'll do it later,' which in developer language often translates to 'Maybe never.' 😀

Automation is crucial, but when it feels like a hurdle, it's high time we look for a better way.

The Solution

A more efficient solution to the automated integration test setup discussed in Approach 2 can be found using the pytest-docker plugin:

def is_ready(url):
    """Check if the weaviate service is ready"""
    try:
        response = requests.get(url)
        return response.status_code == 200
    except requests.exceptions.ConnectionError:
        return False


@pytest.fixture(scope="session")
def weaviate_service(docker_ip, docker_services):
    port = docker_services.port_for("weaviate", 8080)
    url = f"http://{docker_ip}:{port}"
    ready_endpoint = f"{url}/v1/.well-known/ready"

    docker_services.wait_until_responsive(
        timeout=30.0, pause=0.1, check=lambda: is_ready(ready_endpoint)
    )
    return url


@pytest.fixture(scope="session")
def weaviate_client(weaviate_service):
    client = Client(weaviate_service)
    yield client
    client.schema.delete_all()

Right off the bat, you will notice the trimmed down code. This version of the setup does away with unnecessary loops and conditions, leaving you with a sleek and straightforward fixture setup.

In this approach, the weaviate_service fixture automatically manages the container for the duration of the session. The docker_services.wait_until_responsive method ensures that the service is ready before continuing, and this checking function is abstracted away into the is_ready function.

Conclusion

In software development, nothing should stand in the way of best practices. While setting up testing environments can sometimes feel like preparing a seven-course meal just to satisfy a small hunger, there are ways to streamline the process without sacrificing effectiveness. Our exploration of the pytest-docker plugin serves as an example, showing how it simplifies Docker container management in testing workflows, making it more efficient and less error-prone.

By replacing manual and boilerplate-heavy setups with a lean, automated system, we keep the focus on what truly matters: creating reliable, stable software that stands the test of time.