JM 95th

JMeter/Jenkins 95th Percentile Pass/Fail the Build

See 95th Percentile version 2 if you are using assertions

For me this is the most useful addition to the original solution and in fact itís an add-on and can be used with the original solution even if the other changes are not required. There is one caveat to that statement: The original solution outputs data to .jtl files. This needs changing to use .csv files if you want to use Jenkins as used here. See below.

For CI performance testing we need a pass fail criteria that can be used to stop the build. Typically I would recommend using something like 95th percentile of particular transactions (as used here). This could be changed to 90th percentile if you have a particularly variable test environment and you donít want the build to break unnecessarily. Or other metrics could be designed from the available data, perhaps using some of the specific results files described on the JM Results page.

If you look at the collated output from the ĎGenerate Summary Resultsí listener (that this solution must include), you can see that all the named requests from the test script are listed, with their response times in the second column:

95th percentile1

95th percentile2

For this solution to work it is important that

    Requests in the JMeter Script have unique names

The 95th percentile script analyses this output file, pulls out the response times for all the calls for the transaction in question, sorts them, finds the 95th percentile and reports that value:

95th percentile3

The solution shown here can be used outside of Jenkins of course but the script also builds a bespoke jtl output file specifically designed for the Jenkins JMeter plugin. I have used this jtl file building method extensively before (with webpagetest.org front end testing for example) and it seems robust and offers the casual observer a quick overview of the performance in the form of graphs on the Jenkins front end. More details of using this are on the Jenkins page. This is why we specify the guideline not to use .jtl files for other purposes:

95th percentile4

See the JM Jenkins page for details of how to use al this

[Home] [About (CV)] [Contact Us] [JMeter Cloud] [JM Highlights] [JM Overview] [JM Control] [JM Inject] [JM Threads] [JM Results] [JM Assertions] [JM TPS] [JM Metrics] [JM Runtime] [JM Collation] [JM Logs] [JM 95th] [JM 95th v2] [JM Jenkins] [JM Corporate] [JM Scripts] [JM Variables] [JM Embedded] [JM Hosts] [JM Running] [JM Example] [JM Versions] [webPageTest] [_64 images] [asset moniitor] [Linux Monitor] [Splunk ETL] [Splunk API] [AWS bash] [LR Rules OK] [LR Slave] [LR CI Graphs] [LoadRunner CI] [LR CI Variables] [LR Bamboo] [LR Methods] [LR CI BASH] [Bash methods] [Jenkins V2] [Streaming vid] [How fast] [Finding Issues] [Reporting] [Hand over] [VB Scripts] [JMeter tips] [JMeter RAW] [Dynatrace] [Documents] [FAQ] [Legal]

In the Cartesian Elements Ltd group of companies