iAksesoris API— Performance Testing
This article is written as a part of Individual Review of Fasilkom UI’s Software Engineering Project Course 2021
Another Testing?
Imagine you’ve built your API and ready to be consumed by your Applications. Have you ever thought about your API’s performance? How many requests can your API handle in one second? How many loads can your API take until it is down? These questions are crucial if you want to build a good system that can handle many requests, and here Performance Testing do its job.
Performance Testing
But what’s the difference with other testing, such as Unit Testing and Integration Testing? Performance Testing doesn’t test the logic or correctness of your code, like Unit Testing and Integration Testing do. Performance Testing is a software testing process used for testing the speed, response time, stability, reliability, scalability and resource usage of a software application under particular workload.
Why do Performance Testing?
The main purpose of performance testing is not to find bugs but to eliminate performance bottlenecks that can slow down our system. A good system won’t only consider the logic and correctness of the system, but also to measure, maintain, and serve requests efficient and effectively with its limited resources. Hence, application’s performance like its response time, reliability, resource usage and scalability do matter.
Performance Testing can provide information regarding speed, stability, and scalability. This test can uncover what needs to be improved before our system deployed into production. So we can prevent our system from issues such as: running slow while several users use it simultaneously, inconsistencies across different operating systems and poor usability.
Imagine you’ve built a system that is critical, such as space launch programs or life-saving medical equipment should be performance tested to ensure that they run for a long period without deviations. Hence, performance testing is important.
Types of Performance Testing
There are many types of Performance Testing, and we can customize it to suit our business needs, but here are some of the 3 most common Performance Testing:
- Load Testing — is used to determine how an application performs based on a certain volume of users. Usually, a load test will increase the number of requests during the duration of the test, when given an expected workload (i.e. expected number of users or requests according to the design).
- Stress Testing — is very similar to load testing, except they are specifically designed to test the system’s performance with the maximum, or over maximum, capacity of requests and jobs. This test determines at what point a system breaks down.
- Endurance Testing — is done to make sure the software can handle the expected load over a long period of time.
iAksesoris API Endpoint Performance Testing
Our group — Magic People — implements backend system for iAksesoris in Software Engineering Project Course 2021. Let’s take a look and demonstrate Performance Testing on one of the most accessed iAksesoris API’s Endpoint, which is to get all products.
The Endpoint
We will use an endpoint that will retrieve the information of all products in iAksesoris: /api/product/v1/product/ . It is a read-only endpoint and require user’s authentication. This endpoint is chosen because of its high chance to have huge loads, being the first endpoint accessed by user after logged in.
For testing purpose, this endpoint returns all products in the database (54 products). Each of them has about one or two images to be shown.
The Tool — Apache Benchmark
ApacheBench is an application for benchmarking an HTTP server. It sends simultaneous requests with level of concurrency that we can adjust. Then, the analysis result will contain connection time, how many bytes has transferred, average response time, etc.
Apache Benchmark is packaged on apache2-utils package on Linux system. For other Operating Systems, you can check the official documentation or instructions on the internet.
Testing!
Apache Benchmark can be used with Command Line Interface (CLI) mode. There are so many parameters that can be adjusted for our needs. The most used parameters are -n to specify how many requests and-c to specify the level of concurrency.
Before we are going to test the specified endpoint that requires authentication, let’s take a look at performance testing on unauthenticated iAksesoris API’s Endpoint / that returns Hello World message.
ab -n 100 -c 50 http://127.0.0.1:8000/This command sends 100 HTTP GET requests to /with concurrency level 50. You can think it as dropping 100 litre of water from 50 water faucets. Here is the result:
We can see that the iAksesoris API Endpoint /can serve 376.89 requests per second, and it took about 2.653 ms per request. We can also see that all requests are served without fail (failed requests: 0).
Real Performance Testing
In order to access authentication-required endpoints, we need to pass the Bearer JWT token into Authorization header. Now, we can use another parameter -H to add the Authorization header.
We will also increase the number of requests to 1000 with concurrency level of 10, to the endpoint /api/product/v1/product/ , which is the endpoint for logged in users to retrieve all available products.
ab -n 1000 -c 10 -H "Authorization:Bearer <your_jwt_token>" http://127.0.0.1:8000/api/product/v1/product
We need 47.6 seconds to complete 1000 requests with concurrency level of 10, which each request is served in about 47.609 ms. There are no failed requests, so the iAksesoris API is doing good with retrieving 54 products from database. We can also see that the longest request took 666 ms.
Bonus: Performance Testing on Django, Flask, Go, and NodeJS
This section will show us comparison between languages in Performance Testing. I’ve already built four Hello World’s REST API with Django, Flask, Go, and NodeJS. You can get the code here.
Here is the Performance Testing’s result on Django, Flask, Go, and NodeJS, with 10000 requests and concurrency level of 100.
As we can see, other languages can perform better from Python Django Web Framework, at least it can win against Python Flask Web Framework. But, compared to Go and NodeJS, they are still performing better than Python Django and Flask, which can handles more than 1000 requests per second.
Analysis & Thoughts
Note that the result of Performance Testing depends on how big our database is and how do we query the database. For the query, we can improve it by using index (SQL-based database). Because the API is built with Django, database queries are handled by Django, so in this case, we can only consider the size of the database. The bigger the database is, searching and querying process will take more time.
Endpoint selection also matters. In order to give the best out of our API, we can analyze which endpoints are often accessed by users and test it out with Performance Testing. For example, the previous endpoint that serves all products to users can be improved with pagination. For example, if there are many users logged-in in one second, our API can return 10 most-recent products. As the user scrolls down, the Frontend Application will make another request to get next 10, and so on.
We’ve already seen Performance Testing with Apache Benchmark that requires more analysis to suit our needs. There are also other tools, such as Apache JMeter, Autocannon, Locust, etc. Performance Testing is a good approach to build a high-quality system that can always serve request.
Thank you for reading! Happy Testing!
