JM 95th v2

JMeter/Jenkins 95th Percentile Pass/Fail the Build

See 95th Percentile (version 1) for an introduction and if you are NOT using assertions

This page explains version 2 of the 95th percentile data analysis. Both versions are useful but version 2 is more efficient for the controller and could have a significant effect on controller performance for long running tests.

Version 2 includes:

    1. Improved efficiency

    2. Separate calculations for pass/fail or both

    3. Adjustment for the percentile value used

Version 2 is similar to version 1 but it makes use of the bespoke assertions file for any particular transaction. Version 1 uses the general results summary which contains results for all transactions and all other jmeter samplers, including  data processors and csv loading routines for example. Thus version 1 is more resource hungry as it needs to filter through all the results data to pull out the one transaction you are interested in.

Note, if you are not using assertions, perhaps for a simpler unit performance test or basic services test, then version 1 still fits the bill and this is why it is left in the solution.

Other enhancements to version 2

One more difference is pass fail status of your transactions. Version 1 does not consider pass fail. It simply gives the 95th percentile for all transaction requests with that name, regardless of return code and of course the general results file does not have any more detailed pass fail information in it.

Version 2 allows you to analysis 95th percentile of passes / fails or both. And the pass /fail criteria is determined by your code rather than just the return value, which is often more accurate as a page could return a 200 but still not be rendered correctly.

Finally, version 2 allows adjustment of the percentile value , so you can get values for 90th percentiles if that fits your requirements better, for example.

Running version 2 with the same settings as version 1 gives the same results (see below). But version 2 uses the smaller data file (you can see the assertions file is used rather than the general project results file) and now you can check other percentiles and put different limits on them:

Version 1 results:

percv2-1

Version 2 results:

percv2-2

command line options into version 2 are as follows:

percv2-4

And the Jenkins build incorporating this now looks like:

percv2-3

[Home] [About (CV)] [Contact Us] [JMeter Cloud] [JM Highlights] [JM Overview] [JM Control] [JM Inject] [JM Threads] [JM Results] [JM Assertions] [JM TPS] [JM Metrics] [JM Runtime] [JM Collation] [JM Logs] [JM 95th] [JM 95th v2] [JM Jenkins] [JM Corporate] [JM Scripts] [JM Variables] [JM Embedded] [JM Hosts] [JM Running] [JM Example] [JM Versions] [webPageTest] [_64 images] [asset moniitor] [Linux Monitor] [Splunk ETL] [Splunk API] [AWS bash] [LR Rules OK] [LR Slave] [LR CI Graphs] [LoadRunner CI] [LR CI Variables] [LR Bamboo] [LR Methods] [LR CI BASH] [Bash methods] [Jenkins V2] [Streaming vid] [How fast] [Finding Issues] [Reporting] [Hand over] [VB Scripts] [JMeter tips] [JMeter RAW] [Dynatrace] [Documents] [FAQ] [Legal]

In the Cartesian Elements Ltd group of companies