wP Input

Input data and Encoding webPageTest Scripts

This page is about the main input into webPageTest, the webPageTest script. For full details of webPageTest scripting, see here.

Scripting gives a lot more scope to this solution and allows a particular test run to consider full user work flows, including login, filling out forms etc. It also allows for changing DNS settings, which I use in this example to filter out 3rd party calls whilst testing our web site whilst it's under construction.

Scripts are input via a csv file, one per line. The csv file effectively allows you to queue up jobs for webPageTest. Each line is seen as a separate test run but the whole file constitutes a Jenkins Build and graphs will be produced for each line in the file.

The CSV is loaded into Jmeter:


There are 3 fields in the CSV, url_id and scriptDescription are used for the graphs in Jenkins. The id allows you to order the graphs and the description allows a reasonable context to be given to the test. So for example below, we have ids of '01.' and '02.' and descriptions of 'programmes page' and 'a to z page'. the cached and uncached are added by jmeter depending on other settings used. Critically, Jenkins orders the graphs alphabetically, hence the separate ids:


The script field gets passed down to webPageTest and defines the test. If you want to run simple tests, that's fine, such as

So if you wanted to check a series of URLs and have them report up into Jenkins one after the other, you simply make a list of them in your CSV file:

    01. ,Yahoo front page,navigate%09http%3A%2F%2Fuk.yahoo.com
    02. ,Nicks site,navigate%09http%3A%2F%2Fwww.webwob.com
    03. ,BBC front page,navigate%09http%3A%2F%2Fwww.bbc.com

I guess you'll notice the url encoding! This threw me for a while as I was doing a bit of noddy self coding. This is really why I've written this page!

I have found a quick and easy way to get the encoding right, including the newlines that are needed for more complex scripts. So now I use notepad++ and an online encoder. Note the encoder builder does say it's ok to copy the source and use it locally. This is good because it gets around any security issues you may have with passwords etc.

Two sample scripts are shown below before and after decoding. The first one involves logging in and going to a secure page:



The next examples shows where I have removed 3rd party calls before testing our internal application. This is at an early stage of the development cycle and we want to see issues in our code before moving on to check the full solution. I've also shown below the full csv file as it will be fed into jmeter, which consisted of 4 key pages being checked in this manner:



4 urls are sent down through JMeter for a CI build to check key pages using this scenario:


Single Points of Failure (SPF) Investigations

It's worth mentioning here a 3rd set of scripts that are proving very valuable in highlighting a particular problem, that of single points of failure (SPF). If you are loading 3rd party content synchronously then your site could be at risk if the 3rd party site goes down (or has connection issues etc.). We can test our pages for this directly with a specially designed webPageTest script that can be fed in through this solution. This script uses this method


using a black hole server which correctly mimics services going down. This is unlike setting your 3rd party content to point to localhost, as above. In that case, the calling page returns immediately. In this case, the page waits, as it would in Live, for the 3rd party content to time out. An example script to acheive this is (from the link above):

setDnsName ajax.googleapis.com blackhole.webpagetest.org
setDnsName apis.google.com blackhole.webpagetest.org
setDnsName www.google-analytics.com blackhole.webpagetest.org
setDnsName connect.facebook.net blackhole.webpagetest.org
setDnsName platform.twitter.com blackhole.webpagetest.org
navigate your.url.com

[Home] [About (CV)] [Contact Us] [JMeter Cloud] [webPageTest] [wP Highlights] [wP Overview] [wP Jenkins] [wp Jmeter] [wP Input] [wP Start] [wP While] [wP Results] [wP Utilities] [_64 images] [asset moniitor] [Linux Monitor] [Splunk ETL] [Splunk API] [AWS bash] [LR Rules OK] [LR Slave] [LR CI Graphs] [LoadRunner CI] [LR CI Variables] [LR Bamboo] [LR Methods] [LR CI BASH] [Bash methods] [Jenkins V2] [Streaming vid] [How fast] [Finding Issues] [Reporting] [Hand over] [VB Scripts] [JMeter tips] [JMeter RAW] [Dynatrace] [Documents] [FAQ] [Legal]

In the Cartesian Elements Ltd group of companies