Continuous web page performance testing

Web page performance is very important in a website, as it directly impacts your ROI. A slow website will have a high rate of abandonment and is a source of frustration for your visitors.

One of the ways to ensure you are not caught in this situation is to measure web page performance every time you deploy a new feature.

To demo this we are going to use:

  • webpagetest.org –  The tests will be invoked using its REST API via a Node module
  • Appveyor – A CI service to run the tests

First we need to create an API key at webpagetest.org so we can use its API. Browse to http://www.webpagetest.org/getkey.php and request a key.

Next, edit your project (or create a new one from one of your repositories) and add the API key as a secure variable. Navigate to your profile menu and select “Encrypt data”: https://ci.appveyor.com/tools/encrypt

appveyor-encrypt-data

In this screen, enter the API key given by webpagetest.org.

appveyor-encrypt-data-2

You will see your API key has been encrypted. You can use it in config files and Appveyor will decrypt them when they are used in your build steps.

 

Now you add the encrypted variable in your environment section as “webpagetestkey”.

appveyor-variable

You need to install node.js and the webpage test module at build startup/

Now you run the tests as an “after build step”. Add this PS script to invoke them. This script invokes a webpagetest.org test in Germany using Chrome, compares them against your test specs and adds the results in Appveyor’s dashboard.

Replace http://xxxxxxx/ with the site you want to test. Typically this will be the url of your test site.

webpagetest test http://xxxxxx/ --key %webpagetestkey% --location ec2-eu-central-1:Chrome --first --poll --specs testspecs.json --reporter xunit > testresults.xml

[xml]$xml = Get-Content -Path .\\testresults.xml

$nodes = $xml.SelectNodes("/testsuite/testcase")
foreach ($node in $nodes) 
{    
    $name = $node.attributes['name'].value
    $duration = $node.attributes['time'].value
    $outcome = "Passed";
    $errorMessage = "";
    if ($node.SelectSingleNode("failure") -ne $null)
    {
        $outcome = "Failed";
        $errorMessage = $name;        
        $host.SetShouldExit(1);    
    }    
    
    echo $name    
    Add-AppveyorTest "$name" -Outcome $outcome -Duration $duration -ErrorMessage $errorMessage
}

Alternatively you can save the script below as “appveyor.xml” in the root of your repository to install node, webpagetest.org’s API and invoke the tests.

Do replace the secure variable and the url to test.

version: 1.0.{build}
environment:
  webpagetestkey:
    secure: xxxxxxxxxxxxx
nuget:
  account_feed: true
  project_feed: true
install:
  - ps: Install-Product node 0
  - npm install webpagetest -g
before_build:
- cmd: nuget restore
build:
  verbosity: minimal
after_build:
- ps: "webpagetest test http://xxxxxx/ --key %webpagetestkey% --location ec2-eu-central-1:Chrome --first --poll --specs testspecs.json --reporter xunit > testresults.xml\n\n[xml]$xml = Get-Content -Path .\\testresults.xml\n\n$nodes = $xml.SelectNodes(\"/testsuite/testcase\")\nforeach ($node in $nodes) \n{    \n    $name = $node.attributes['name'].value\n    $duration = $node.attributes['time'].value\n    $outcome = \"Passed\";\n    $errorMessage = \"\";\n\n    if ($node.SelectSingleNode(\"failure\") -ne $null)\n    {\n        $outcome = \"Failed\";\n        $errorMessage = $name;\n        $host.SetShouldExit(1);\n    }\n    echo $name\n    Add-AppveyorTest \"$name\" -Outcome $outcome -Duration $duration -ErrorMessage $errorMessage\n}"
deploy: off

 

Next, add a “testspecs.json” file to the root directory of your repository

This file defines the criteria that will be checked against your website. In this example we are testing the page load time, rendering time, gzip and cache scores based on webpagetest.org’s rules:

{
  "median": {
    "firstView": {
      "render": 3000,
      "loadTime": 3000,
      "score_gzip": {
        "min": 20
      },
      "score_cache": {
        "min": 50
      }
    }
  }
}

Commit and push into your repository and start a build in Appveyor.

You will see the tests being run in the console. In case one of your changes causes a performance metric to fail then your build will fail.

median.firstView.render: 1780 should be less than 3000
median.firstView.loadTime: 2755 should be less than 3000
median.firstView.score_gzip: 36 should be greater than 20
median.firstView.score_cache: 57 should be greater than 50
median.firstView.responses_200: 36 should be greater than 1

 

By following the steps above you can integrate continuous page performance testing in your continuous integration pipeline.

You can see an example setup in Appveyor here:

https://ci.appveyor.com/project/nimeshjm/episerver-test-automation/build/tests

An example repository with these tests configured:

https://github.com/nimeshjm/EPiServer-test-automation

Webpage test node API wrapper:

https://github.com/marcelduran/webpagetest-api

Changing the test specs:

https://github.com/marcelduran/webpagetest-api/wiki/Test-Specs