Assertion help verify that your server under test returns the expected results.
Following are some commonly used Assertion in JMeter:
• Response Assertion
• Duration Assertion
• Size Assertion
• XML Assertion
• HTML Assertion
Response Assertion
The response assertion lets you add pattern strings to be compared against various fields of the server response.
For example, you send a user request to the website http://www.google.com and get the server response. You can use Response Assertion to verify if the server response contains expected pattern string (e.g. "OK").
Duration Assertion
The Duration Assertion tests that each server response was received within a given amount of time. Any response that takes longer than the given number of milliseconds (specified by the user) is marked as a failed response.
For example, a user request is sent to www.google.com by JMeter and get a response within expected time 5 ms then test case pass, else, test case failed.
Size Assertion
The Size Assertion tests that each server response contains the expected number of byte in it. You can specify that the size be equal to, greater than, less than, or not equal to a given number of bytes.
JMeter sends a user request to www.google.com and gets response packet with size less than expected byte 5000 bytes àtest case pass. If else, test case failed.
XML Assertion
The XML Assertion tests that the response data consists of a formally correct XML document.
HTML Assertion
The HTML Assertion allows the user to check the HTML syntax of the response data. It means the response data must be met the HTML syntax.
Handson - Assertion
We will continue on the script we developed in the earlier.
In this test, we are using Response Assertion to compare the response packet from www.google.com matches your expected string.
Roadmap of test:
The response assertion control panel lets you add pattern strings to be compared against various fields of the response. Step 1) Add Response Assertion
Right-Click Thread Group -> Add -> Assertions -> Response Assertion
Response Assertion Pane displays as below figure:
Step 2) Add Pattern to test
When you send a request to Google server, it may return some response code as below:
• 404: Server error
• 200: Server OK
• 302: Web server redirect to other page. This usually happens when you access google.com from outside USA. Google re-directs to country specific website. As shown below, google.com redirects to google.co.in for Users.
Assume that you want to verify that the web server google.com responses code contains pattern 302,
On Response Field To Test, choose Response Code,
On Response Assertion Panel, click Add -> a new blank entry display -> enter 302 in Pattern to Test.
Step 4) Run your test
Click on Thread Group -> Assertion Result
When you ready to run test, click Run button on the menu bar, or short key Ctrl+R.
The test result will display on Assertion Results pane. If Google server response code contains the pattern 302, the test case is passed. You will see the message displayed as follows:
Now back to Response Assertion Panel, you change the Pattern to test to from 302 to 500.
Because Google server response code doesn't contain this pattern, you will see the test case Failed as following:
Troubleshooting:
Any issue while running the above scenarios ... do the following:
1. Check whether you are connecting to internet via a proxy. If yes, remove the proxy.
2. Open a new instance of JMeter
3. Open the AssertionTestPlan.jmx in JMeter
4. Click on Thread Group -> Assertion Result
5. Run the Test
We parameterize the input to run the test with different set of data for each user. We will provide the data in a file and provide as input for a field.
For this we will create a .csv file in Excel and save it. For this example, we will parameterize four fields. They are from Port, to Port, passFirst0 and passLast0. The details to corresponding fields are as follows:
We have our requests as shown below:
To add the data to the test we need to add the config element as below:
Right Click on Thread Group -> Add -> Config Element -> CSV Data Set Config
The CSV Data Set details were entered as follows:
Parameters for the CSV Data Set config is as below:
Now once the CSV Data Set is configured, we need to set the variables at appropriate position as shown:
The value is set as ${Variablename} for the required fields. For example here passFirst0 is set as ${FirstName} and passLast0 is set as ${LastName}.
Let us now verify the values are passed correctly to the required parameters from the csv file. To perform this we will add a Listener -> View Results tree as below:
In Thread group set the Number of users as 2 and run. We see that there are two sample results as we ran for two users. We check the first thread we see that from Port is Frankfurt and to Port is London which is same as first record of our .csv file.
Now, let us verify the second thread results, we see that from Port is London and to Port is New York. So for two different customers there are two different input provided.
In the same way we can check for the customer firstname and lastname also.
Uses of Controllers in JMeter
JMeter in-built function controller is basically a Logic Controller provides control on “when & how” to send a user request to a web server under test. Logic Controller having command of the order of the request sends to the server.
Logic Controllers
Logic Controllers facilitates users to describe the sequence of processing request sends to the server in a Thread like; JMeter’s Random Controllers can be used to send HTTP requests randomly to the server. Logic Controllers define the order in which user request are executed.
Some useful Logic Controllers comes in operation are,
• Recording Controller
• Simple Controller
• Loop Controller
• Random Controller
• Module Controller
Recording Controller
After recording of testing process happens in JMeter, Recording Controller role is to store those recordings.
Simple Controller
Simple Controller is a container, holds user request. It does not provide any customization, randomization or change of loop count, etc as other logic controllers do.
Loop Controller
Loop Controller creates a situation where user request runs either; for limited time or forever, which is better explained in the figure,
Random Controller
Random Controller creates a situation where user request runs in random order in each loop period. Suppose, the website http://www.google.com is having 3 user requests in following order, 1. HTTP request
2. FTP request
3. JDBC request
Each user request runs 5 times i.e,HTTP request runs 5 times, FTP request runs 5 times and JDBC request runs 5 times,means; total user requests send to the Google server is 15 = 5*3 by JMeter.
In each loop, user requests send sequentially in following order, HTTP request -> FTP request-> JDBC request
In each loop, user requests send randomly in following order, FTP request -> HTTP request-> JDBC request
Or JDBC request -> FTP request-> HTTP request
Module Controller
Module Controller sets the modularity in each module of JMeter function stored in Simple Controller. For example; any web application consists of numbers of functions like; sign – in, sign – out, and account creation, password change, Simple Controller stores these functions as modules then Module Controller chooses which module needs to run.Suppose, simulation happens when 50 users sign-in, 100 users sign-out, and 30 users search on website www.google.com.
Using JMeter, user can create 3 modules where each module simulates each user activity: sign-in, sign-out, and Search. Module controller selects which module wants to run. Some More Vital Controllers Interleave Controller: Select only one user request which runs in one particular loop of thread.
Runtime Controller: Determines that till how long its children are permitted to run.For example; if user has detailed Runtime Controller 10 seconds then JMeter will run the test for 10 seconds. Transaction Controller:Evaluates the overall time taken to complete a test execution Include Controller: It is created to utilize an external test plan. This controller permits you to utilize multiple test plans in JMeter.
Brief Look on Loop Controller
Here you will get to know each and every step involved in adding Loop Controller to at present performance test plan.
The Loop Controller helps samplers run at specific number of times with the loop value you definite for the Thread Group. Suppose user, add one HTTP Request to a Loop Controller with a loop count 50, configure the Thread Group loop count to 2, and then JMeter will send a total of 50 * 2 = 100 HTTP Requests. These examples are explained below in better manner,
• Configuring Thread Group
• Adding Loop Controller
• Configuring Loop Controller
• Add View Result in Table
• Run the Test
First Step: Configuring Thread Group Add Thread Group:
To add the Thread Group, first run JMeter, from opened interface of JMeter choose Test Plan from the tree and right click to choose Add -> Threads (Users) -> Thread Group.
After opening thread Group, enter Thread Properties as shown in figure below,
In the above figure, Number of Threads: 1 user is connected to target website, Loop Count: run it 2 times, and Ramp-Up Period: 1.
The Thread Count and The Loop Counts both are different. In the Thread Count, it simulates 1 user trying to connect the targeted website. In the Loop Counts, it simulates 1 user trying to connect the targeted website 2 times. Add JMeter elements:
Here we will Add HTTP request Default” element.
To get this element, go to Thread Group and right-clicking, from context menu select Add -> Config Element -> HTTP Request Defaults.
In the HTTP Request Defaults page, enter the Website name (www.google.com) under Web server -> Server name or IP.
Adding Loop Controller:
To open the Loop Controller pane, right click on Thread Group -> Logic Controller -> Loop Controller.
Second Step: Configuring Loop Controller:
Given below figure shows Loop Count: 50 which means one user request sends to the web server google.com run 50 times.Jmeter with take loop value =2 specified for Thread Group in the above figure to send the total HTTP Requests: 2 * 50 = 100.
Right click on Loop Controller, from context menu, click on Add -> Sampler -> HTTP request.
Third Step: Add View Results in Table:
To view the test result in tabular format, right click on Add -> Listener -> View Result in Table
View Results in Table will display like given below figure,
Fourth Step: Run the test:
After opening View Results in Table, click on Start button on Menu bar (Ctrl+R) to run test.
In previous article we have cover How to install JMeter on your machine to start running your first JMeter script. The different part or component of JMeter is called Element which co-relates with each other but designed for different-different purpose. Before start working on Jmeter, it is best to know all components or Elements of Jmeter with full detail description. From this article you will get to know the Elements of JMeter and why they are useful in JMeter.
Given below figure presents you some common elements in JMeter,
• Test Plan
• Thread Group
• Controllers
• Listeners
• Timers
• Configuration Elements
• Pre-Processor Elements
• Post-Processor Elements
• Execution order of Test Elements
Test Plan
A test plan is the top level body of JMeter, explains sequence of steps execute at run time. A final test plan made up of one or more Thread Groups, Sampler, logic controllers, listeners, timers, assertions, and configuration elements. Each Sampler can be preceded by one or more Pre-processor element, followed by Post-processor element, and/or Assertion element. It saves in Java Management Extensions (JMX) format.
JMX is an open text based format, used across dissimilar systems. This format makes test plan to be open in a text editor and can use the “find and replace” function to speedily change a CMS name, web server name, or document ID that might appear hundreds of times throughout the test plan with very little determination.
Thread Group
Thread Group is an initial stage of test plan. The name, Thread Groups represent a group of Threads. Under this group, each thread simulates one real user requests to the server.
A control on a thread group facilitates to set the number of threads on each group. Also can control Ramp Up Time and the number of test iterations.
If you mention 10 numbers of threads then JMeter will generate and simulate 100 user requests to the server under test.
Controllers
There are two types of controllers: 1) Samplers 2) Logical Controllers
1) Samplers
Samplers facilitate JMeter to deliver explicit types of requests to the server. It simulates a user’s request for a page from the target server. So that, to avail POST, GET, DELETE functions on a HTTP service, user can add HTTP Request sampler. Apart from HTTP Request sampler, there are other simpler too,
2) Logical Controllers
Logic Controllers decide the order of processing of Samplers in a Thread. It offers a mechanism to control the flow of the thread group. Logic Controllers facilitates to customize the logic that JMeter uses to resolve when to send requests. Logic Controllers can alter the order of requests coming from their child elements.
Logic Controllers of JMeter provide,
• Runtime Controller
• IFController
• Transaction Controller
• Recording Controller
• Simple Controller
• While Controller
• Switch Controller
• ForEach Controller
• Module Controller
• Include Controller
• Loop Controller
• Once Only Controller
• Interleave Controller
• Random Controller
• Random Order Controller
• Throughput Controller
A ForEach Controller Control Panel looks like the following figure,
Test Fragments
Test Fragments is a different type of element located at the same level as Thread Group element. It is eminent from a Thread Group in that it is not performed unless it is referenced by either a Module Controller or an Include_Controller. This element is morally for code re-use within Test Plans.
Listeners
Listeners facilitates viewers to view Samplers result in the form of tables, graphs, trees or simple text in some log files and also give pictorial access to the data collected by JMeter about those test cases as a Sampler component of JMeter is executed. Listeners offer a means to collect, save, and view the results of a test plan and store results in XML format, or a more efficient (but less detailed) CSV format. Their output can also be viewed directly within the JMeter console.
It basically adjusts anywhere in the test, moreover under the test plan, gather data only from elements at or below their level.
Listeners of JMeter provide these many things,
• Graph Results
• Spline Visualizer
• Assertion Results
• Simple Data Writer
• Monitor Results
• Distribution Graph (alpha)
• Aggregate Graph
• Mailer Visualizer
• BeanShell Listener
• Summary Report
• Sample Result Save Configuration
• Graph Full Results
• View Results Tree
• Aggregate Report
• View Results in Table
Timers
To perform load/stress testing on your application, you are using threads, controllers and samplers then JMeter will just blast your application with nonstop requests. This is not the real environment or characteristic of the real traffic. JMeter thread sends request without stopping between each sampler. This not exactly what you want. We can add a timer element which will permit us to define a period to wait between each request.
So, to simulate real traffic or create extreme load/stress – orderly, JMeter provides you Timer elements.
Just have a look on the types of Timers available,
• Synchronizing Timer
• JSR223 Time
• BeanShell Time
• Gaussian Random Timer
• Uniform Random Timer
• Constant Throughput Timer
• BSF Time
• Poisson Random Time
As an example, the Constant Timer Control Panel looks like this:
Configuration Elements
Configuration Element is a simple element where you can collects the corporate configuration values of all samplers like webserver’s hostname or database url etc.
These are some commonly used configuration elements in JMeter,
• Java Request Defaults
• LDAP Request Defaults
• LDAP Extended Request Defaults
• Keystore Configuration
• JDBC Connection Configuration
• Login Config Element
• CSV Data Set Config
• FTP Request Defaults
• TCP Sampler Config
• User Defined Variables
• HTTP Authorization Manager
• HTTP Cache Manager
• HTTP Cookie Manager
• HTTP Proxy Server
• HTTP Request Defaults
• HTTP Header Manager
• Simple Config Element
• Random Variable
Pre-Processor Elements
A Pre-Procesor is an interesting thing which executes before a sampler executes. It is basically used to modify the settings of a Sample Request before it runs, or to update variables that are not extracted from response text.
These are the list made up of all the Pre-Processor Elements JMeter,
• JDBC PreProcessor
• JSR223 PreProcessor
• RegEx User Parameters
• BeanShell PreProcessor
• BSF PreProcessor
• HTML Link Parser
• HTTP URL Re-writing Modifier
• HTTP User Parameter Modifier
• User Parameters
Post-Processor Elements
A Post Processor gets executed after finishing a sampler execution process. This element is basically useful to procedure the response data, for example, to retrieve particular value for later use.
These are the list made up of all the Post-Processor Elements JMeter,
Introduction to Apache JMeter
Apache JMeter is a great open source application with awesome testing abilities. Web Server is a platform which carries loads of numbers of applications and users, so that it is necessary to know that how does it works or performs means; how effective it is to handle simultaneous users or applications.
For example; how the “Gmail” supporting server will perform when numbers of users simultaneous access the Gmail account – basically have to do performance testing using performance testing tools like JMeter, Loadrunner etc.
To check the high performance of the application or server, do high performance testing using JMeter for exceptional results.
Before understanding Overview of JMeter, let us have a look on three testing approach,
Performance Test: This test provides the best possible performance of the system or application under a given configuration of infrastructure. Very fast, it also highlights the change need to be made before application goes into production.
Load Test: This test is useful for determining and preceding the system under the top load it was aimed to work under.
Stress Test: This test tries to break the system by crushing its resources.
Introduction to JMeter
JMeter design is based on plugins. Most of its “out of the box” features are applied with plugins. Off-site developers can simply spread JMeter with custom plugins.
JMeter works on these many protocols: Web Services – SOAP / XML-RPC, Web – HTTP, HTTPS sites ‘web 1.0′ web 2.0 (ajax, flex and flex-ws-amf), Database – JDBC drivers, Directory – LDAP, Messaging Oriented service – JMS, Service – POP3, IMAP, SMTP, FTP.
Features Supported by JMeter:
People really think of that why they should go for JMeter? So, to clear their drought, let us get to know JMeter awesome features,
Open source application: JMeter is a free open source application which facilitates users or developers to use source code for other development or modification purpose.
User – friendly GUI: JMeter comes with natural GUI. It is very simple and easy to use and users get familiar very soon with it.
Platform independent: Although, it is totally a Java based desktop application. That’s why; it can run on any platform. It is highly extensible and capable to load the performance test in many different server types: Web – HTTP, HTTPS, SOAP, Database via JDBC, LDAP, JMS, Mail – POP3.
Write your own test: Using plugins, write your own test case to extend the testing process. JMeter uses text editor to create a test plan and supplies in XML format.
Support various testing approach: JMeter supports various testing approach such as Load Testing, Distributed Testing, and Functional Testing, etc.
Simulation: JMeter can simulate various users with simultaneous threads, generate heavy load against web application under test.
Support multi-protocol: JMeter works on web application testing and database server testing, and also supports protocols such as HTTP, JDBC, LDAP, SOAP, JMS, and FTP.
Script Test: JMeter is capable to perform automation testing using Bean Shell &Selenium.
Totally multi-threading framework: It’s a full multi-threading framework permits simultaneous sampling by many threads and simultaneous sampling of different functions by distinct thread groups.
Easily understandable test result: It visualizes the test result in a format such as chart, table, tree and log file are very simple to understand.
Easy installation step: Just run the“*.bat” file to use JMeter. In Linux/Unix – JMeter can be approached by clicking on JMeter shell script. In Windows – JMeter can be approached by starting the JMeter.bat file.
JMeter architecture working process:
JMeter simulates number of users send request to an appropriate server which shows the performance/functionality of an appropriate server / application via tables, graphs etc. The figure below depicts this process:
1. Step by step method to test single web page in Apache JMeter
2. Manually creating a test plan in Apache JMeter
3. Simulating 50 users through Apache JMeter test plan
Let us see what we want to achieve first. We want to create a Test Plan in Apache JMeter so that we can test performance of one web page say page shown by the URL: http://www.google.com/. We will need many elements in the JMeter Test plan let us see what are they:
So this is what we shall see when we are done with creating the Test Plan. We have to start with a blank JMeter window
Before adding anything more to this window let us change the name of Test Plan. We can type a new name for the node in the Name text box. In this version of JMeter the name does not get reflected immediately when you type it in the text box. You have to change focus to workbench node and back to Test Plan node to see the name getting reflected.
Let us give new name "First Test" to the Test Plan node
Now we will add our first element in the window. We will add one Thread Group, which is placeholder for all other elements like Samplers, Controllers, Listeners. We need one so we can configure number of users to simulate. The Thread Group represents one set of actions performed by User.
In JMeter all the node elements are added by using the context menu. You have to right click the element where you want to add a child element node. Then choose the appropriate option to add.
We will click the Thread Group option and it will get added under the Test Plan (First test) node.
Like we changed the name of Test Plan let us also change the name of Thread Group. For us this is the element which represents multiple users. So let us name it "Users". For us this element means Users visiting the Google Home Page.
Now we have to add one Sampler in our Thread Group (Users). As done earlier for adding Thread group, this time we will open the context menu of the Thread Group (Users) node by right clicking and we will add HTTP Request Sampler by choosing Add > Sampler > HTTP request option.
This will add one empty HTTP Request Sampler under the Thread Group (Users) node.
We will need most configurations in this node element, HTTP Request Sampler. The settings we have to configure are:
Name: we will change the name to reflect the action what we want to achieve
This will will name "Visit Google Home Page" Server Name or IP: here we have to type the web server name
In out case it is www.google.com
Note http:// part is not written this is only the name of the server or its IP Protocol: we will keep this blank
That will mean we want HTTP as the protocol Path: we will type path as / (slash)
This means we want the root page of the server
This also means now the server will decide what page will be sent to us and it is the default page of the web site
After add the HTTP Request Sampler we will need one Listener. Let us add View Results Tree Listener under the Thread Group (User) node. This will ensure that the results of the Sampler will be available to view in this Listener node element.
Adding a listener is also same like adding a new node under Thread Group. Open the context menu and choose Add > Listener > View Results Tree option to add the listener.
With this much of setup we are now ready to run our first test. Coming back to the configuration of the Thread Group (Users) we have kept it all default values. This means JMeter will execute the sampler only once. It will be like a single user and only one time.
This is similar to like a user visiting a web page through browser, only here we are doing that through JMeter sampler. We will execute the test plan using Run > Start option.
Apache JMeter asks us to save the test plan in a disk file before actually starting the test. This is important if we want to run the test plan again and again. If we say not to save by clicking No option it will run without saving. Let us save it first so we can reuse it again.
We have kept the setting of the thread group as single thread (that means one user only) and loop for 1 time (that means run only one time), hence we will get result of one single transaction in the View Result Tree Listener.
This means JMeter was successful in fetching the web page at the given URL just like a browser. It has stored all the headers and the response sent by the web server and ready to show us the result in many ways.
Let us take a closer look at the three tabs available in the View Result Tree Listener view.
The first tab is Sampler Results. It shows JMeter data as well as data returned by the web server. Usually browsers hide this data from us as it is not related to showing the web page. This data is utilized internally by the browser, for example when there is a Response Code: 404 then the browser shows a page not found error page, and when the Response Code is 200 the browser shows the received web page HTML.
The second tab is Request, where all the data which was sent to the web server as part of the request is shown.
The last tab is Response data. In this tab the listener shows the data received from server as it is in text format. It also have facility to show the data in XML, HTML formats but that will be just the data in other way (no JavaScript will be executed). This may look different that the page actually seen in the browser.
We can select different ways of viewing this response data like as below:
This was just one request but JMeter's real strength is in sending the same request like many users are sending it. To test the web servers with multiple users we will have to change the Thread Group (Users) settings.
We will set 50 users and these requests will be sent with some delay of (10/50) seconds as we have set the ramp up time to 10.
To visualize the results, a better aggregating listener will be required so we will add Summary Report Listener in the Thread Group node. Adding Summary Report Listener is similar to adding a View Results Tree Listener. Just right click on the Thread group node and choose Add > Listener > Summary Report.
We will run the test plan again after adding the Summary Report Listener ans when the test is done we will see the following results. Your results may be different than mine due to different environments but mostly similar.
This is how we tried testing one single page with JMeter. The summary report shows values about the measurement JMeter has done while calling the same page as if many users are calling the page. Let us look at some of the value headings and what they mean:
• Samples: A sample mean one sampler call. One request to web page in our case. So the value of 51 means total 51 web page requests to the page http://www.google.com were made by JMeter
• Average: This value is the average time taken to receive the web pages. In our case there were 51 values of receiving time which were added and divided by 51 and this value is arrived by JMeter. This value is a measure of performance of this web page. This means on an average 334 milliseconds time is required to receive this web page for our network conditions.
• Min and Max: These are the minimum and maximum value of time required for receiving the web page.
• Std. Dev: This shows how many exceptional cases were found which were deviating from the average value of the receiving time. The lesser this value more consistent the time pattern is assumed.
• Error %: This value indicated the percentage of error. In our case 51 calls were made and all are received successfully this means 0 error. If there are any calls not received properly they are counted as errors and the percentage of error occurrence against the actual calls made is displayed in this value.
To check the performance of the web page the average value of receiving the page is an important parameter to check.
