Learning Experience with Apache JMeter
Sharing my versatile experience of using this smart tool ‘ Apache JMeter ’ from checking elementary web & API performance tests to High RPM Load tests for Aarogya setu app.
Year 2007 -2016 - JMeter Induction & Exploration
So I introduced myself to JMeter in 2007 and got formal hands on it of using all basic required features by 2009 — taking out time to explore it more in an intermittently. During that tenure, I used it for recording & executing granular web performance tests for my Organization. Further with added new features & enhanced versions — JMeter evolved more and in 2011 , I used it for executing lite Performance tests of APIs via JMeter UI platform. That gave me more confidence in exploring the tool and later I tested WSDL requests.
Year 2017 -2020 — A big Transformation & Learning
Then it was a revolutionary experience for me in 2017- when I along with my colleagues Radha Krishan & Mohit Gupta successfully got set up JMeter as an reliable inhouse Performance / Load testing tool in our Organization. Entire set up was done on Amazon AWS EC2 instance using JMeter’s dynamic command line arguments , JMeter output file ( .jmx) conversion to static reports for build results comparison & native JMeter dashboard reporting. Parallely run time test execution output gets dumped to ‘InfluxDB ’ for maintaining historical trends & real time Grafana dashboard reporting.
In this entire set up, Github along with Jenkins got integrated with JMeter to get executed in an interconnected way for running Performance tests on APIs during git merge requests & deployment. My teammate Mohit Gupta helped me out here in building an important piece — ‘ reading JMeter’s Aggregate report via shell script to filter out values for APIs response time comparison and further triggering alerts ’.
Impact
That helped Engineering team in receiving high response time & failure alerts of critical APIs during deployment — before release gets available to large set of users. Below pasted are few samples of JMeter native dashboard reporting, Grafana dashboard plotted using JMeter ..
This inhouse setup helped too in executing High RPM load tests for multiple Big Marketing campaigns supported by our Organization and cracking out required infra set up & technical concerns, latencies in internal major release before rolling out changes to live.
Further Assignments & Challenges
Then there came an interesting ask where we have to support IPL Campaign having expected 150K hits/sec. Here Engineering Team asked to hit their firebase code directly and NOT the APIs to meet the expected target of writes on multiple firebase shards from different users. So there JMeter helped in encapsulating the firebase code within its component ‘JSR223 Sampler’ by storing ‘firebase jar’ was inside JMeter. To achieve said RPM, we implemented Distributed Load Testing of JMeter using Master Slave Strategy -where we cloned multiple EC2 instances from Parent EC2 instance. Another interesting problem here we encountered was ‘java.lang -out of memory heap space’ error -while increasing the threads count & executing test. Then it got solved by creating ‘setenv.sh’ file inside JMeter & providing required config to it.
Another challenging stuff came around where we have to break server side cache for every single request to get triggered — as due to external service provider’s caching , all requests fired were getting cached. Due to this , RPM Graph on NewRelic was not reaching to the expected benchmark. Here we tried different Thread Groups of JMeter but ice was not breakable and also removing this caching would have badly impacted front end applications consuming the same API . Then I got deep dived into this problem with dev team and summed up with solution of passing timestamp as an additional parameter in request. This worked out like a charm for us & load test was successful by helping out dev team in identifying few code level issues & required infra scalability concerns for live set up.
Pasted below is screengrab of NewRelic insight displaying expected Throughput ( RPM) of requests fired via JMeter for the targeted API.
Benchmark 75K RPM for a single API
This got followed by another assignment of achieving ~ 75K RPM for a single API supporting another IPL Marketing campaign. To achieve this, combination of JMeter’s basic thread group along with its component ‘constant throughput timer’ was used on single JMeter setup instance to fire requests. This test helped out dev team in identifying 503 service unavailable errors and taking the justified action of changing the strategy to avoid using any external service provider for the caching support.
below is screengrab of NewRelic insight displaying achieved expected RPM
High Scalable APIs Load tests for Aargoya Setu App
Then recently load testing requirement for APIs supporting project came up. Using JMeter’s distributed load testing approach, We internally scaled load up to 90–100k RPM unique requests for APIs and got fixed errors, improved response time & pointed towards expected infra scalability optimisation to be done to handle initial high load .
Below pasted are samples of APIs high response time & errors during the early phase when application just got started building and then further with fixes & changes done by Engineering team — how response time got improved & errors reduced to minimal state for High RPM.
Here using JMeter again , bottlenecks related to slow DB queries/functions , few errors in Application / infra and problems related to high response time internally got digged out before passing App to Indian Govt Partners for their own scalability testing. In this load test activity, my colleague Radha Krishan collaborated with me in meeting the expected criteria for this crucial Project.
TakeAway & Credits
So that’s all in above mentioned multiple ways, JMeter helped me in understanding & implementing the Performance -Load Testing Concepts in my career path. And I’m really thankful to below mentioned JMeter Engineering Team , Tech Support Team & Users Community for helping me out in solving my queries while getting juggled somewhere in between problems .
Also this book — ‘ Master JMeter from Load Testing to DevOps ‘ penned down by legendary JMeter Team members — Bruno Demion ,Philippe Mouawad & Antonio Gomes help in understanding the JMeter terminology in detail.
Cordially,