In Uncategorized
Performance testing. Be prepared.
An unexpected load jump may lead to so many different consequences: temporary (when the site can be quickly restored) or fatal (when the business suffers to sustain losses due to lack of sales). Careful performance testing will help avoid this situation. Let's talk about it today.
THIS can happen at any time. If you have a streaming service, THIS happens exactly at the moment when the current popular boy band releases a new song. If you sell movie tickets online, you'll see THIS when Frozen II is released. If you have a big bridal shop, put a "sale up to 90% off" banner on the site and wait. THIS will not take long.
As you may have guessed, by THIS we mean huge traffic. An unexpected load jump may lead to so many different consequences: temporary (when the site can be quickly restored) or fatal (when the business suffers to sustain losses due to lack of sales). Careful performance testing will help avoid this situation. Let's talk about it today.
As you may have guessed, by THIS we mean huge traffic. An unexpected load jump may lead to so many different consequences: temporary (when the site can be quickly restored) or fatal (when the business suffers to sustain losses due to lack of sales). Careful performance testing will help avoid this situation. Let's talk about it today.
Performance testing is a generic concept that often denotes different types of software testing.
Nowadays, the growth of computing power is gradually decreasing. At the same time, the volume of data processed, the number of users and requests to systems continue to grow. That is why issues of performance and its testing are widely discussed in a professional environment.
In this article, we will describe how performance testing is conducted and what is measured at each of the stages.
In this article, we will describe how performance testing is conducted and what is measured at each of the stages.
Stress Test
This test is usually carried out first. Its goal is to set the maximum level of product performance. The stress test helps to analyze how the key characteristics of the system (response time of the most important business processes, the number of requests / transactions per second) depend on the number of simultaneous users.
During the stress test, the load on the system gradually increases until the application stops working correctly. The result of the test is the "performance limit" of the application, that is, the maximum number of users who can work with the application at the same time.
Usually, this type of testing is carried out if the application's user load should increase in the short term. For example, for e-commerce, this could be sales days. It is hard to imagine how much the online shopping sites have to withstand when Christmas sale hunters are going on the warpath. Spooky!
During the stress test, the load on the system gradually increases until the application stops working correctly. The result of the test is the "performance limit" of the application, that is, the maximum number of users who can work with the application at the same time.
Usually, this type of testing is carried out if the application's user load should increase in the short term. For example, for e-commerce, this could be sales days. It is hard to imagine how much the online shopping sites have to withstand when Christmas sale hunters are going on the warpath. Spooky!
Load Test
It is carried out at the second stage. This test evaluates the ability of the system to cope with a continuous load. The number of users for a load test is defined as 80% of the maximum performance detected during the stress test. The system is being under load for 4-8 hours.
At this time, various performance metrics are collected: the number of requests per second, transactions per second, server response time, the percentage of errors in the responses, etc. As a result of the test, we understand whether the system meets the performance requirements. Additionally, using this test, we can localize application breakpoints and other defects.
At this time, various performance metrics are collected: the number of requests per second, transactions per second, server response time, the percentage of errors in the responses, etc. As a result of the test, we understand whether the system meets the performance requirements. Additionally, using this test, we can localize application breakpoints and other defects.
How to load the system properly?
The load during the tests should simulate the actual user behavior. Only in this case, you can get a result that will show the real system performance. Running any other form of the load is a waste of resources and time.
To protect yourself from scripting errors, we recommend carefully checking the network traffic generated by load scripts and real users. The structure of requests and responses should be identical.
The load during the tests should simulate the actual user behavior. Only in this case, you can get a result that will show the real system performance. Running any other form of the load is a waste of resources and time.
To protect yourself from scripting errors, we recommend carefully checking the network traffic generated by load scripts and real users. The structure of requests and responses should be identical.
When conducting load testing, it is important to correctly install a special tool. The tool is installed on the generator – a virtual or physical machine which resources are used to create load on the system. The load generator should be located as close as possible to the test environment.
For example
The environment load is 1000 users, and the actual load during the test is only 200 users. In this case, you won't get the accurate response time data and estimate the performance of the entire system correctly.
The environment load is 1000 users, and the actual load during the test is only 200 users. In this case, you won't get the accurate response time data and estimate the performance of the entire system correctly.
It is also recommended to place the load generator and the app server on the same local network. This must be done to eliminate deviations when serving the load. These deviations usually occur due to network delays, and their range can vary from a few milliseconds to a few tens of seconds.
For example
You got an average response time of 10 seconds during the test. But the response time consisted of the following: 9.5 seconds – data sending, 0.5 seconds – response making. It turns out that the server is able to quickly process requests, but the network conditions do not allow it to happen.
You got an average response time of 10 seconds during the test. But the response time consisted of the following: 9.5 seconds – data sending, 0.5 seconds – response making. It turns out that the server is able to quickly process requests, but the network conditions do not allow it to happen.
Volume Test
The task of the test is to find out what impact the increase in data volume will have on the system. The test requires a database filled with the required amount of information. The load technique remains the same as during load testing. With the help of this technique, we can find out just how the app performance changes when the traffic increases several times.
Stability Test
The stability test assesses the performance of the system with a long-expected load in the 24/7 operating mode. For example, if a website is visited by users located in different time zones, the load level may be kept constant. The duration of the load may vary depending on the goals and capabilities of the project. Will the performance level decrease? Is the application able to withstand a stable load without critical failures? We can answer these questions through a stability test.
With this test, you can determine whether the system will restart itself if served with a long-term load. The effect of rare events, for example, the work of garbage collectors, can also be studied.
With this test, you can determine whether the system will restart itself if served with a long-term load. The effect of rare events, for example, the work of garbage collectors, can also be studied.
What do we get as a result? We get answers to the following questions:
• Will application performance increase if additional hardware resources are added?
• Will productivity increase in proportion to the amount of added hardware?
• Will application performance increase if additional hardware resources are added?
• Will productivity increase in proportion to the amount of added hardware?
Performance test readings
Response time.
It is measured from the moment a request is sent to the server until the last byte is received from the server.
Requests per second.
The client application generates an HTTP request and sends it to the server. The server software processes the request, generates a response and sends it back to the client. The total number of requests per second is the reading we need.
Transactions per second.
User transactions are a sequence of user actions. This reading shows how successfully the system has passed load testing. To do this, you must compare the real transaction time with the expected one. You can also estimate the number of transactions per second.
The number of virtual users per unit of time.
This reading allows you to find out if the application's performance meets the established requirements. If your scripts are as close as possible to the actual user's behavior, then you are doing everything correctly. In this case, one virtual user will be equal to one real user.
It is measured from the moment a request is sent to the server until the last byte is received from the server.
Requests per second.
The client application generates an HTTP request and sends it to the server. The server software processes the request, generates a response and sends it back to the client. The total number of requests per second is the reading we need.
Transactions per second.
User transactions are a sequence of user actions. This reading shows how successfully the system has passed load testing. To do this, you must compare the real transaction time with the expected one. You can also estimate the number of transactions per second.
The number of virtual users per unit of time.
This reading allows you to find out if the application's performance meets the established requirements. If your scripts are as close as possible to the actual user's behavior, then you are doing everything correctly. In this case, one virtual user will be equal to one real user.
The errors percentage is calculated as the ratio of invalid to valid responses over a period of time.
Standard (desired) application readings are usually specified in the requirements.
We also need to talk about the differences between client-side testing and server performance testing. Let's think, for instance, of a simple web application.
The best solution to the problem of web servers' stability is a two-tier application architecture: the client side (front-end) and the server side (back-end).
The user opens the browser and sends a request to the site page on the front-end server. The request is received and sent to the executive part of the web application – the back-end server. The back-end is responsible for storing application logic, executing PHP scripts, and generating HTML pages.
The best solution to the problem of web servers' stability is a two-tier application architecture: the client side (front-end) and the server side (back-end).
The user opens the browser and sends a request to the site page on the front-end server. The request is received and sent to the executive part of the web application – the back-end server. The back-end is responsible for storing application logic, executing PHP scripts, and generating HTML pages.
The front-end accepts the generated page from the back-end and sends it to the browser as an answer to the user's request. After receiving the page, the user's browser displays it, sending to the front-end the requests for graphic content and CSS. The front-end accepts and processes these requests without back-end participation.
Thus, the web-application client and server parts performance can be estimated with the help of two different types of testing. These are client-side testing for the front-end and server performance testing for the back-end.
Thus, the web-application client and server parts performance can be estimated with the help of two different types of testing. These are client-side testing for the front-end and server performance testing for the back-end.
Client-side performance testing.
The main task here is to measure exactly how much time it takes for the browser to display an HTML-page. The main readings are the volume of loading data and the number of requests executed. You can collect these statistics using the built-in browser tools. You can also use special tools or online services.
The main task here is to measure exactly how much time it takes for the browser to display an HTML-page. The main readings are the volume of loading data and the number of requests executed. You can collect these statistics using the built-in browser tools. You can also use special tools or online services.
In addition to the total weight of the page, the tools provide detailed information on each of the components. After examining the parameters of the requests, you can find a number of page displaying problems. For example, the page is loaded with too large a picture or Javascript does not work correctly.
The page display speed can be improved by reducing the size of the elements (CSS, Javascript, graphic content) or shorten variable names and optimize the page code. Also using this test, you can detect the page elements absence or incorrect operation. Another check analyzes the HTTP cache headers.
The page display speed can be improved by reducing the size of the elements (CSS, Javascript, graphic content) or shorten variable names and optimize the page code. Also using this test, you can detect the page elements absence or incorrect operation. Another check analyzes the HTTP cache headers.
Server-side performance testing.
Server-side performance testing analyzes the requests execution and corresponding back-end's requests reception.
Why do we need this test?
1. To measure the response time of important business transactions
2. To determine the load limit
3. To identify system performance bottlenecks
4. To make recommendations for performance improving
5. To find possible defects that appear only when a large number of users work simultaneously
A few more tips on testing performance:
1. Correlation of dynamic parameters and the construction of scripts should be based on real user behavior. Otherwise, testing will be more like a DDOS attack.
2. Before developing and running scripts, it is necessary to conduct a great deal of analytical work in order to prepare a detailed methodology.
3. It is important to take into account the time delays between user actions and correctly place them in scripts according to the behavior of real users.
4. The caching function should be reproduced for both virtual users and real users in the system industrial usage.
5. To find functional defects that appear only under load, you should create scripts with the same sets of actions as real users. This will provide an opportunity to analyze and reproduce requests.
Why do we need this test?
1. To measure the response time of important business transactions
2. To determine the load limit
3. To identify system performance bottlenecks
4. To make recommendations for performance improving
5. To find possible defects that appear only when a large number of users work simultaneously
A few more tips on testing performance:
1. Correlation of dynamic parameters and the construction of scripts should be based on real user behavior. Otherwise, testing will be more like a DDOS attack.
2. Before developing and running scripts, it is necessary to conduct a great deal of analytical work in order to prepare a detailed methodology.
3. It is important to take into account the time delays between user actions and correctly place them in scripts according to the behavior of real users.
4. The caching function should be reproduced for both virtual users and real users in the system industrial usage.
5. To find functional defects that appear only under load, you should create scripts with the same sets of actions as real users. This will provide an opportunity to analyze and reproduce requests.
Let's sum up. Performance testing is important to be conducted consistently. It is also important to receive all the necessary readings and interpret them correctly. The use of this approach will allow you to estimate the system's productivity, as well as help identify some hidden defects.
Now our tools and technicians are bared. We're prepared!
Now our tools and technicians are bared. We're prepared!
We wish you to be always prepared for business challenges and fear nothing.
We love what we do and do it with pleasure.
We love what we do and do it with pleasure.
QAcamp team
You may also like: