ADDENDUM (blog for anyone struggling to get this going...)
Iíve really struggled to get a proper scenario going with real load (even just a 10 minute one). For some reason I was not getting the xml results file created in the slave workspace. This stopped the LR analysis proceeding.
I tried getting rid of spaces in the workspace name (by changing the Jenkins build name) and that worked for one run and then failed again.
Tried increasing the time out in the LoadRunner specific settings in the Jenkins plugin - you MUST set the scenario time out large enough to allow your test to run to conclusion.
Tried ignoring errors in the same settings panel.
So far, I can run my proper scenario for 9 minutes including ramp up and down but not 11 minutes. Will carry on investigating...
Finally, It is now working but I canít be clear why (!). It could be that I was not being patient enough with the analysis step finishing. I thought it was hanging and it looked like all the relevant files were not being produced in the workspace. But I may have jumped the gun. I will continue to monitor my CI process with a real project and report back here.
To check this whole setup, I have copied the same scenario and scripts from my Jenkins master to the slave. Set up the same directory structure (C:\backup\Loadrunner\scenarios and C:\backup\Loadrunner\scripts\bips\Stage\). And copied the Jenkins project from the master to the slave (setup the same Jenkins build steps). I DID HAVE TO add the cygwin path to my slave path so I could run the Jenkins bash shells on this box. Iíll add that step above. Add c:\cygwin\bin and /cygdrive/c/cygwin/bin to the system path (to be on the safe side).
I ran the scenario though Jenkins directly on the Master and it worked fine and I repeated this process on the slave. I then found I could run it from master to slave...
It does look like analysing results does take a LONG time with a real project. This could be an issue for CI as time is important for dev team turn around. I guess scaling up the cloud box would help here. This may need further investigation...
Her we go (test output just now):
Error: Timeout for Analysis passed.
Exception while trying to analyze results:
Process must exit before requested information can be determined.
at System.Diagnostics.Process.EnsureState(State state)
at HpToolsLauncher.TestRunners.PerformanceTestRunner.generateAnalysisReport(TestRunResults runDesc)
at HpToolsLauncher.TestRunners.PerformanceTestRunner.RunTest(String scenarioPath, String& errorReason, RunCancelledDelegate runCancelled)
Test result: Error
Iíll try scaling up the AWS instance from a c1.medium to a c3.2xLarge (new generation). Mind you, I probably shouldnít just throw hardware at it as an instant solution. Letís see if this works, can always hone in on the problem properly if it does.
Just seen this (helps to read the manual!):
Scenario execution timeout (in minutes) Ė timeout for lengthy parts of a scenario execution, which include running Vusers, collating results, analyzing results and shutting down the Controller. If any of these operations within any of the selected scenarios is not completed after the specified number of minutes, the step is terminated and declared ďfailedĒ.
So the bigger machine may improve things (make it faster) but the solution should also include increasing the timeout setting in the plugin. I think I may need a faster machine anyway just to keep the CI rolling. Lets see if the new hardware works. I currently have the timeout set to 40 minutes and the scenario runs for 30 minutes. Analysis therefore had in the region of 10 minutes without finishing.
I think that should be sufficient for a CI solution. In fact, ultimately, for our continuous deployment model, I need to do the full performance step within a 15 minute time slot, so something has to change. I will have to aim at a 10 minute intensive test with 5 minutes analysis. Iíll try this later and see what size machine I need to do that.
So far I still havenít managed to get a working test with 20 users for 30 minutes (at 2tps) using the Jenkins plugin. At this point I need to try and get some HP support. Although CI may be aimed at smaller intensive tests, I do also need longer test runs to work for my general day-to-day workload.
Iíll revisit this later. For the moment I will have to carry on with JMeter.
March 2015 Additional Notes
Things are now working a lot better, just about fully there in fact. See here for more up-to-date details about the plugin etc.