Integration Testing with Cypress

To give a bit of background… I’m working with a small, cross-skilled team of Ruby and dot-net engineers, and we were interested in streamlining our awkward development process. We were feature branch based, but were hoping to incrementally move to trunk based CI/CD.

One part of our process that we wanted to optimise first was the QA’s manual regression testing. Doing the same tests over and over is not only a waste of their time, but can’t be the least bit fun.

For each test our QA was manually conducting we instead wanted to pair up and write an integration test. This would not only give us better confidence to release, but also expand the automation skill-set of our QA.

These tests should automatically run as part of the GitLab pipeline whenever code is committed to source control. If any of the tests fail then the pipeline is to be marked as failed.

What is Cypress?

Enter cypress.io. This is an opensource javascript framework that can run a set of integration tests from the command line. It isn’t coupled to either Ruby or dotnet, which is exactly what we need.

Getting Set Up

cd to the root of your project and install cypress with npm install --save-dev cypress@3.6.0. Npm will add cypress as a dependency in your package.json, and will create a folder structure where your tests should be created.

Writing a Simple Test

Under ./cypress/integration create a new file named server_status.js with the following contents:

describe('Server status endpoint', function () {
  it('Returns OK', function () {
    cy.request('/api/server_status')
      .then(response  => {
        expect(response.status).to.eq(200)
    })
  })
})

This will simple test that our API’s server_status endpoint returns a 200 response. Lovely.

Running it in Development

To run this integration test in development we can run cypress run --config baseUrl=http://localhost:24555. You can of course plop this into your Makefile:

run-cypress:
  cypress run --config baseUrl=http://localhost:24555

Debugging a Test

Sticking console.log in your tests doesn’t appear to output anywhere that I could see, but you can use the regular javascript debugging technique of adding debugger; where you want to break.

Run the test with cypress open --config baseUrl=http://localhost:24555, which will open a UI and you can run the test from within your browser. From here you can debug as you normally would.

Running it in a Pipeline

Great. We now have a simple test and we can now work to get it running in GitLab.

Config

In the cypress.json file, we’ll configure the baseUrl:

{
  "baseUrl": "http://foobar-api:24555",
  "video": false
}

Docker

Create a new tests.Dockerfile in the root of your project with this:

FROM cypress/included:3.6.1 as test
WORKDIR /src
COPY ./cypress.json ./package.json ./
RUN npm install
COPY /cypress/ ./cypress/

Make

Add these bits to your Makefile:

REPO=your-docker-registry/yourorg/foobar
GIT_SHA?=$(shell git rev-parse --verify HEAD)

docker-build-tests:
  docker build --pull -t ${REPO}_tests:${GIT_SHA} -f "tests.Dockerfile" .

docker-push-tests:
  docker push ${REPO}_tests:${GIT_SHA}

GitLab CI

In your .gitlab-ci.yml (assuming you already have your app build steps in here), add the following bits:

stages:
  - build-tests
  - run-tests

build-tests:
  image: docker:stable
  stage: build-tests
  script:
    - apk add --no-cache make
    - make GIT_SHA=$CI_COMMIT_SHA docker-build-tests
    - make GIT_SHA=$CI_COMMIT_SHA docker-push-tests

integration-test:
  image:
    name: your-docker-registry/yourorg/foobar_tests:$CI_COMMIT_SHA
    entrypoint: ['']
  stage: run-tests
  variables:
    GIT_STRATEGY: none
    BASE_URL: "http://foobar-api:24555"
  services:
    - name: your-docker-registry/yourorg/foobar_api:$CI_COMMIT_SHA
      alias: foobar-api
  script:
    - cd /src
    - npm run cypress:run -- --config baseUrl=$BASE_URL

Keh?

So what’s going on here?

We’re building the foobar_tests image and immediately pushing it to the docker repository. This is done because each stage can run on different GitLab runners, so we can’t rely on the runners’ local image cache to run the next stage.

The docker image is being tagged with the $CI_COMMIT_SHA to aid with concurrent pipelines that are executing against different branches. We know that we’re going to be running the

The run-tests stage then uses that freshly built foobar_tests image as its’ base image, which is built on cypress/included:3.6.1.

The foobar-api is the API that we’re going to run our tests against (which isn’t detailed here). This is hosted as a GitLab service and is magically configured to be accessible by the foobar_tests on the http://foobar-api alias.

When npm run cypress:run... is run, our test will run in GitLab. This is great because we don’t need to deploy anything, anywhere in order for these tests to run!

If the tests fail for whatever reason; cypress returns a non-zero return-code, which GitLab will pick up as a failure. This causes the pipeline to fail with lots of red indicators.

//TODO:

Ideally I should have created these integration tests in a separate git repository, such that any down-chain app can pull, build, and run the test suite whenever a commit is pushed.

Popular posts from this blog

Taking a memory dump of a w3wp process

sp_blitzIndex

GitLab Badges