Skip to content

Commit e050dfa

Browse files
committed
Update based on review
1 parent 1915f02 commit e050dfa

1 file changed

Lines changed: 22 additions & 21 deletions

File tree

technical-reports/Performance-Testing.md

Lines changed: 22 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -13,14 +13,14 @@ Authors: [Ronak Lakhotia](https://github.com/RonakLakhotia), [Amrut Prabhu](http
1313

1414
## Introduction
1515

16-
This report gives a brief overview of the profiling operations performed on TEAMMATES. In particular, it includes a detailed discussion of the Load and Performance (L&P) testing framework and justification for our solution.
16+
This report gives a brief overview of the profiling operations performed on TEAMMATES. In particular, it includes a detailed discussion of the Load and Performance (L&P) testing framework and the process we followed.
1717

1818
## Problem
1919

20-
TEAMMATES is one of the biggest student projects in the open source community. As of April 2019, TEAMMATES boasts a developer community of over 450 contributors and a codebase with ~130 KLoC.
20+
TEAMMATES is one of the biggest student projects in the open source community. As of April 2019, TEAMMATES boasts a codebase with ~130 KLoC. More importantly, it has over 350,000 users.
2121
Maintaining such a project demands high quality standards to ensure long term survival.
22-
This means, continuously monitoring code health and product performance. As the number of developers and user base continue to grow, we need to ensure optimal performance at all times.
23-
To do so, we need to be able to identify performance issue-prone operations with a quantitative measure so that they can be rectified.
22+
There are many factors that can cause decelerating performance of the production software like increased number of database records, increased number of simultaneous requests to the server, and a larger number of users accessing the system at any given point.
23+
This means, continuously monitoring code health and product performance. To do so, we need to be able to identify performance issue-prone operations with a quantitative measure so that they can be rectified.
2424

2525
## Overview of Solution
2626

@@ -38,54 +38,55 @@ In this report we will discuss the reasons behind why we chose JMeter and a more
3838

3939
Some of the tools that we considered before deciding on JMeter were:
4040

41-
* [Gatling](https://gatling.io/) - It higher barrier to entry for potential contributors.
42-
* [LoadRunner](https://www.guru99.com/introduction-to-hp-loadrunner-and-its-archtecture.html) - This is a license tool and cost of using it is high. LoadRunner has a lot of protocols, such as HTTP, Oracle and SAP.WEB., but we don’t need this.
43-
* [BlazeMeter](https://www.blazemeter.com/)
41+
* [Gatling](https://gatling.io/) - It has higher barrier to entry for potential contributors.
42+
* [LoadRunner](https://www.guru99.com/introduction-to-hp-loadrunner-and-its-archtecture.html) - This is a licensed tool and cost of using it is high. LoadRunner has a lot of protocols, such as HTTP, Oracle and SAP.WEB., but we do not need those.
43+
* [BlazeMeter](https://www.blazemeter.com/) - The reports generated by BlazeMeter are basic as compared to JMeter. Also, the free version of the tool has limited functionalities.
4444

4545
## Reasons for using JMeter
4646

47-
One of the main reasons we use JMeter over the other tools was the **extensive documentation** we found online. There are a number of resources to help you get started. Some of which we have listed below:
47+
One of the main reasons we use JMeter over the other tools was the **extensive documentation** we found online. There are a number of resources to help one to get started. Some of which we have listed below:
4848

4949
* [JMeter Tutorial for beginners](https://www.guru99.com/jmeter-tutorials.html)
5050
* [How to use JMeter](https://www.blazemeter.com/blog/how-use-jmeter-assertions-three-easy-steps)
51-
* [The official website](https://jmeter.apache.org/usermanual/build-web-test-plan.html) also offers a good documentation on how to get started.
51+
* [The official website](https://jmeter.apache.org/usermanual/build-web-test-plan.html)
5252

5353
Some other reasons why we found JMeter to be useful:
5454

5555
**Open Source** - JMeter is an open source software. This means that it can be downloaded free of cost. The developers can use its source code, can modify and customize it as per their requirement.
5656

57-
**Ease of Integration** - It is easier to integrate JMeter into the project because of the [JMeter Java API](https://jmeter.apache.org/api/index.html). There is also a [jmeter-gradle plugin](https://github.com/jmeter-gradle-plugin/jmeter-gradle-plugin) if you want to make it a part of your build process.
57+
**Ease of Integration** - It is easier to integrate JMeter into the project because of the [JMeter Java API](https://jmeter.apache.org/api/index.html).
5858

59-
**Roust Reporting** - JMeter can generate effective reporting. The test results can be visualized by using Graph, Chart, and Tree View. JMeter supports different formats for reporting like text, XML, HTML and JSON.
59+
**Robust Reporting** - JMeter can generate effective reporting. The test results can be visualized by using Graph, Chart, and Tree View. JMeter supports different formats for reporting like text, XML, HTML and JSON.
6060

6161
## Current implementation of the solution
6262

6363
JMeter offers us a couple of ways to perform the tests. We had the choice of performing these tests with automating tools like jmeter-gradle plugin and the JMeter Java API.
6464
We explored both possibilities but ended up using the JMeter Java API. Some key observations we made:
6565

66-
* The jmeter-gradle-plugin is not well maintained and does not have easy-to-find documentation.
67-
* The existing resources are outdated and are not in sync with the latest version of JMeter.
68-
* The JMeter Java API, on the other hand, fits well with TEAMMATES' backend testing framework.
69-
* It is also easier to integrate it into the CI pipeline with a TestNG gradle task.
70-
* The entire process is more coherent while allowing the same level of configuration.
66+
* The jmeter-gradle-plugin is not well maintained and does not have easy-to-find documentation. The existing resources are outdated and are not in sync with the latest version of JMeter.
67+
* The JMeter Java API, on the other hand, fits well with TEAMMATES' backend testing framework. It is also easier to integrate it into the CI pipeline with a TestNG gradle task.
68+
* With the JMeter Java API, the entire process is more coherent while allowing the same level of configuration.
7169

7270
A brief description of the process:
7371

74-
* Create a test json and csv file for the test.
72+
* Determine the failure threshold criteria according to which endpoint is being tested.
73+
74+
* Create a test JSON and CSV file for the test.
75+
* The purpose of the JSON and CSV files are to store data that is needed to test the endpoints. With the data stored in these files we can parameterize HTTP requests and simulate multiple users accessing the endpoint being tested.
7576
* Since the data files are large (at least 5 times the size of test data used for E2E tests), they are not committed to the repo. This way, we can easily change the scale of the test without having to rewrite the code for generating the data.
7677

7778
* Create the JMeter test and run.
78-
* Each test configures the test plan, similar to how it is done in the GUI. We also considered using a Builder pattern, but it didn’t make complete sense to do so (since we can’t say for sure what the components of the class are, and what order they should be in). Instead, we have created abstractions and default configurations which make it easier to create new tests.
79+
* Each test configures the test plan, similar to how it is done in the GUI. We also considered using a Builder pattern, but it did not make complete sense to do so (since we can not say for sure what the components of the class are, and what order they should be in). Instead, we have created abstractions and default configurations which make it easier to create new tests.
7980

80-
* Determine the failure threshold criteria and display the summarised results for that endpoint.
81+
* Display the summarised results for that endpoint.
8182

8283
* Delete the entities and data files created.
8384

8485
A more detailed overview of the tasks performed can be seen in the [Continuous Profiling Project page](https://github.com/teammates/teammates/projects/7).
8586

8687
## Findings and Recommendations
8788

88-
Currently the performance issue-prone operations in TEAMMATES are as follows:
89+
Currently, the performance issue-prone operations in TEAMMATES are as follows:
8990

9091
* Instructor page: Enrolling students
9192

@@ -99,6 +100,6 @@ This is still a work-in-progress as we are yet to consolidate the results but ou
99100
## Future Work
100101

101102
We need to fine-tune the L&P test parameters and set suitable thresholds for failure. These should align with the goals of the application.
102-
Currently login takes a lot of time (compared to student profile, at least). So, we can explore the idea of using a delay after login, and testing the endpoint after that.
103+
Currently logging in takes a lot of time (compared to student profile, at least). So, we can explore the idea of using a delay after logging in, and testing the endpoint after that.
103104

104105
We can also explore elements like Timers and JSON Extractors. By synchronizing, Timer JMeter **Spike Testing** can be achieved.

0 commit comments

Comments
 (0)