Load testing with artillery

So you need to load test an application to find its maximum capacity? This set of basic scripts will get you started.

Install artillery with: npm install -g artillery … assuming you already have npm installed.

The premium version of artillery comes with all sorts of fancy features, such as the ability to run distributed load tests. But the free version is more than enough to do a basic set of tests.

As a word of warning, if you’re running this against a production system, you’ll want to schedule it to run out of hours.

loadtest.ps1

Npm only allows a certain amount of CPU to be in use by a process before it starts throwing warnings out. So to get around this, we can spawn multiple processes using this script:

param([int]$clients = 1)
<#
.SYNOPSIS
Run load tests

.PARAMETER  clients
Runs this many concurrent load tests

.EXAMPLE
.\loadtest.ps1
This will run the load test with a single client

.EXAMPLE
.\loadtest.ps1 -clients 3
This will run the load test with three clients
#>

Push-Location  $PSScriptRoot

WRITE-HOST  'Running load test' -ForegroundColor Green

For($i=0; $i -lt $clients - 1; $i++) {
   Start-Process  "powershell.exe" -ArgumentList "artillery run -c config.yml -o .\Output\stats_ scenarios.yml" -WindowStyle Minimized
}

artillery run -c config.yml -o .\Output\stats scenarios.yml

artillery report .\Output\stats

Pop-Location

config.yml

As for configuring artillery, there’s loads you can do. Generally you’ll want a few phases of test, each of which defining a different level of load.

Here you can set your common headers, and configure your payload files. For consistency, it’s a good idea to set the order to sequential, rather than the default of random.

config:
  target: "http://yourtargetwebsite.co.uk/Service"
  
  phases:
    - duration: 60
      arrivalRate: 5
      maxVusers: 200
      
  ensure:
    maxErrorRate: 1

  defaults:
    headers:
      accept-encoding: "gzip-deflate"
      content-type: "text/xml"
      connection: "keep-alive"

  payload: 
    - path: "./payload.csv"
      fields:
        - "payload"
      order: sequence

scenarios.yml

This is where you define your… scenarios. I’ve just defined one here, but if you define many you can weight each scenario differently to mirror real life traffic.

For debugging purposes, I’ve included some commented out logging. There’s a bunch of options for defining the fields to capture in the response, but I went with regex just for ease. Here I’m just dumping out the entire request and response for inspection.

scenarios:
  - name: "Search"
    weight: 100/100
    flow:
    #- log: "request: {{ payload }}"
    - post:
        url: "/OldSchoolWebService.asmx"
        body: "{{ payload }}"
        capture:
          - header: "content-length"
            as: length
          # - regexp: ".*"
          #   as: "root"
    - log: "length: {{ length }}"
    #- log: "root: {{ root }}"

payload.csv

This is the file where we have a request body per line. For the case of my SOAP OldSchoolWebService, I didn’t need to include the soap body, which was nice.

<FooSearch><Foo bar="abc"/></FooSearch>
<FooSearch><Foo bar="xyz"/></FooSearch>

Run the test

Now for the fun bit! In your nearest terminal, execute ./loadtest.ps1 and wait for it to complete. A html report will be generated and will open automatically in your browser.

If you’ve run this load test with multiple clients, you should know that the generated report is only for one of those clients. If you need more accurate results, you can either buy their pro version, or rely on other monitoring tools like NewRelic.