Cypress integration test using docker-compose passes locally but fails in GitLab CI with docker:dind

Current behavior:

A job in my pipeline uses docker-compose with Cypress to run integration tests. The docker-compose.ci.yml file contains the following services:

• postgres
• backend # django webserver
• asgiserver # django channels daphne server
• nginx # serves Vue application that makes websocket requests
• redis

These containers are built and run as described in the integration-tests.sh (see below) and then start the cypress service with docker-compose which runs integration tests.

Out of 6 Cypress integration test, 4 passes locally. One test seems to fail sometimes and pass with the same code on retries. Here is an example of this behavior. The pipeline is passing, but this is only so I can capture the video output, otherwise GitLab CI will not capture the test recordings as job artifacts. The last test that tests websocket behavior fails consistently (but passes when running in docker-compose locally).

Here is a description of my cypress tests:

• test_api.js: tests cy.request can reach an API endpoint that returns a static response
• test_homepage.js: tests that cypress can access the Vue application
• test_login.js: tests login functionality (tests the connection to the postgres database container) (with the same code, this test fails sometimes and passes at other times).
• test_logout.js: tests a user can logout; uses custom login Command instead of login manually as in test_login.js (always passes)
• test_ws_connection.js: presses a button that sends a websocket message and tests that the websocket response from the server triggers the creation of a notification element in the DOM. (this test fails in GitLab CI, but passes locally when running integration-tests.sh)
• test_redis_connection: sets a value to the redis cache and reads the value (this test passes)

Here is the repo: https://gitlab.com/verbose-equals-true/django-postgres-vue-gitlab-ecs

Desired behavior:

All Cypress tests running in docker-compose should pass in GitLab CI as they pass when running locally on docker-compose. When I run docker-compose locally, I see the following logs:

asgiserver    | 192.168.160.6:42158 - - [29/Jul/2019:04:10:29] "WSCONNECTING /ws/ping-pong/" - -
asgiserver    | specific.cHSRkyiZ!lrvzSAdPrLsn
asgiserver    | 192.168.160.6:42158 - - [29/Jul/2019:04:10:29] "WSCONNECT /ws/ping-pong/" - -
asgiserver    | received message
asgiserver    | sending ws response
    ✓ User sees PONG message from websocket PING message (721ms)
cypress       | 

Steps to reproduce: (app code and test code)

To run the tests locally with docker-compose, you can run ./integration-tests.sh that can be found in the root of the project repository. This requires that you have docker-compose installed locally. To see the behavior in GitLab CI, you can clone the project and run the pipeline on a public shared runner.

integration-tests.sh contains docker-compose commands:

#!/bin/bash

# set -e

echo "Starting services"
docker-compose -f docker-compose.ci.yml up -d --build

sleep 20
echo "Services are up and ready"

echo "Seeding database with user"
# docker-compose -f docker-compose.ci.yml exec backend python manage.py create_default_user

docker-compose -f docker-compose.ci.yml -f cypress.yml up --exit-code-from cypress

echo "Cypress tests passed successfully."

echo "Stopping docker compose..."
docker-compose -f docker-compose.ci.yml -f cypress.yml down

Versions

Versions are documented in docker-compose.ci.yml and cypress.yml. Running the tests locally, I am on Docker version 18.09.7 with Ubuntu 16.04.

[joelvandiver]

For what it’s worth to whoever may land here in the future, I ran into a similar problem.

I found out that the handling of volumes in docker-dind didn’t work the same as when I ran docker-compose locally. GitLab mounts the code of the job onto a volume. I then was trying to mount the volume into the cypress image through docker compose.

I was able to solve this (after some help from a friend) by finding the original volume mount in the GitLab CI:

.gitlab-ci.yml

  script:
    # Mount the volume through dind
    - export CONTAINER_ID=$(docker ps -q -f "label=com.gitlab.gitlab-runner.job.id=$CI_JOB_ID" -f "label=com.gitlab.gitlab-runner.type=build")
    - export MOUNT_DIR=$(docker inspect $CONTAINER_ID -f "{{ range .Mounts }}{{ if eq .Destination \"/builds\" }}{{ .Source }}{{end}}{{end}}")

I then used the $MOUNT_DIR var in my docker compose for the cypress image when mounting the volume:

docker-compose.yml

...
  # Cypress container
  cypress:
    # https://github.com/cypress-io/cypress-docker-images
    image: "cypress/included:9.1.0"
    depends_on:
      - api
    environment:
      - CYPRESS_baseUrl=http://api:8000
    working_dir: /e2e
    volumes:
      - ${MOUNT_DIR}/:/e2e
    networks:
      api-int:
...

I hope this helps!

[jennifer-shehane]

Unfortunately we have to close this issue due to inactivity. Please comment if there is new information to provide concerning the original issue and we can reopen.