Integration Testing with Cypress

-

To give a bit of background… I’m working with a small, cross-skilled team of Ruby and dot-net engineers, and we were interested in streamlining our awkward development process. We were feature branch based, but were hoping to incrementally move to trunk based CI/CD.

One part of our process that we wanted to optimise first was the QA’s manual regression testing. Doing the same tests over and over is not only a waste of their time, but can’t be the least bit fun.

For each test our QA was manually conducting we instead wanted to pair up and write an integration test. This would not only give us better confidence to release, but also expand the automation skill-set of our QA.

These tests should automatically run as part of the GitLab pipeline whenever code is committed to source control. If any of the tests fail then the pipeline is to be marked as failed.

What is Cypress?

Enter cypress.io. This is an opensource javascript framework that can run a set of integration tests from the command line. It isn’t coupled to either Ruby or dotnet, which is exactly what we need.

Getting Set Up

cd to the root of your project and install cypress with npm install --save-dev cypress@3.6.0. Npm will add cypress as a dependency in your package.json, and will create a folder structure where your tests should be created.

Writing a Simple Test

Under ./cypress/integration create a new file named server_status.js with the following contents:

describe('Server status endpoint', function () {
  it('Returns OK', function () {
    cy.request('/api/server_status')
      .then(response  => {
        expect(response.status).to.eq(200)
    })
  })
})

This will simple test that our API’s server_status endpoint returns a 200 response. Lovely.

Running it in Development

To run this integration test in development we can run cypress run --config baseUrl=http://localhost:24555. You can of course plop this into your Makefile:

run-cypress:
  cypress run --config baseUrl=http://localhost:24555

Debugging a Test

Sticking console.log in your tests doesn’t appear to output anywhere that I could see, but you can use the regular javascript debugging technique of adding debugger; where you want to break.

Run the test with cypress open --config baseUrl=http://localhost:24555, which will open a UI and you can run the test from within your browser. From here you can debug as you normally would.

Running it in a Pipeline

Great. We now have a simple test and we can now work to get it running in GitLab.

Config

In the cypress.json file, we’ll configure the baseUrl:

{
  "baseUrl": "http://foobar-api:24555",
  "video": false
}

Docker

Create a new tests.Dockerfile in the root of your project with this:

FROM cypress/included:3.6.1 as test
WORKDIR /src
COPY ./cypress.json ./package.json ./
RUN npm install
COPY /cypress/ ./cypress/

Make

Add these bits to your Makefile:

REPO=your-docker-registry/yourorg/foobar
GIT_SHA?=$(shell git rev-parse --verify HEAD)

docker-build-tests:
  docker build --pull -t ${REPO}_tests:${GIT_SHA} -f "tests.Dockerfile" .

docker-push-tests:
  docker push ${REPO}_tests:${GIT_SHA}

GitLab CI

In your .gitlab-ci.yml (assuming you already have your app build steps in here), add the following bits:

stages:
  - build-tests
  - run-tests

build-tests:
  image: docker:stable
  stage: build-tests
  script:
    - apk add --no-cache make
    - make GIT_SHA=$CI_COMMIT_SHA docker-build-tests
    - make GIT_SHA=$CI_COMMIT_SHA docker-push-tests

integration-test:
  image:
    name: your-docker-registry/yourorg/foobar_tests:$CI_COMMIT_SHA
    entrypoint: ['']
  stage: run-tests
  variables:
    GIT_STRATEGY: none
    BASE_URL: "http://foobar-api:24555"
  services:
    - name: your-docker-registry/yourorg/foobar_api:$CI_COMMIT_SHA
      alias: foobar-api
  script:
    - cd /src
    - npm run cypress:run -- --config baseUrl=$BASE_URL

Keh?

So what’s going on here?

We’re building the foobar_tests image and immediately pushing it to the docker repository. This is done because each stage can run on different GitLab runners, so we can’t rely on the runners’ local image cache to run the next stage.

The docker image is being tagged with the $CI_COMMIT_SHA to aid with concurrent pipelines that are executing against different branches. We know that we’re going to be running the

The run-tests stage then uses that freshly built foobar_tests image as its’ base image, which is built on cypress/included:3.6.1.

The foobar-api is the API that we’re going to run our tests against (which isn’t detailed here). This is hosted as a GitLab service and is magically configured to be accessible by the foobar_tests on the http://foobar-api alias.

When npm run cypress:run... is run, our test will run in GitLab. This is great because we don’t need to deploy anything, anywhere in order for these tests to run!

If the tests fail for whatever reason; cypress returns a non-zero return-code, which GitLab will pick up as a failure. This causes the pipeline to fail with lots of red indicators.

//TODO:

Ideally I should have created these integration tests in a separate git repository, such that any down-chain app can pull, build, and run the test suite whenever a commit is pushed.

Popular posts from this blog

Taking a memory dump of a w3wp process

-
Taking a memory dump of a w3wp process Taking a memory dump of an IIS worker process in a live system can be problematic, as IIS will kill the process if it has been suspended for more than 60 seconds. Only use these steps if your application is on a high availability cluster, else you’ll take your app down! copy sysinternals procdump.exe and pssuspend.exe to target machine create powershell file with: $iispid = Get-Process svchost | ? { $_ . modules . ModuleName -eq "iisw3adm.dll" } | Select - First 1 - ExpandProperty Id $workerpid = Get-Process w3wp | Sort ws - Descending | Select - First 1 - ExpandProperty Id & ".\pssuspend.exe" $iispid Write-Output "Creating memory dump for w3wp PID $workerpid " & ".\procdump.exe" - ma $workerpid & ".\pssuspend.exe" $iispid - r (taken from stackoverflow ) 3) check the w3wp process has reached the high level of RAM usage that you’re intending to
Read more

GitLab Badges

-
Image
GitLab has built in functionality to allow you to add badges, to provide an at-a-glance status of your app's quality.  In order to set project badges up, you need to be a maintainer of the project so that you can access the settings. Pipeline Status This shows whether your master branch pipeline has passed or failed.  The badge is super easy to set up.  In GitLab go to Settings > General > Badges > Add Badge, and enter this: Name:  Pipeline Link:  https://gitlab.com/%{project_path}/commits/%{default_branch} Badge: https://gitlab.com/%{project_path}/badges/%{default_branch}/pipeline.svg These links should be right for any project, as they're using variables. Coverage % Code coverage is a little more involved, as it relies on you having already configured code coverage in your app.  For the app I've been working on, which is a dotnet application, I opted to use the opensource MiniCover .  This is configured to writes a code coverage report to the
Read more

sp_blitzIndex

-
Some more notes after watching Brent’s How I Use the First Responder Kit: sp BlitzIndex . This sproc will analyse every index across all databases in your sql server and return a prioritised list of potential issues. EXEC sp_blitzIndex @GetAllDatabases = 1 The MoreInfo column is interesting. If you copy the SQL out and run it, it will give you more information on the indexing issue in question. E.g: EXEC dbo . sp_BlitzIndex @DatabaseName = 'YourDatabase' , @SchemaName = 'dbo' , @TableName = 'tFoo' ; If you run this, the top result set it a list of indexes that are on the target table. It shows how many seeks and scans each index has had, the size of the index, etc. Note the URL column for more information on each issue category. Missing indexes are shown in the 2nd result set. The 3rd result set shows the table definition, which is nice to have without having to mess with the SSMS UI.
Read more