The [General] section defines global settings of the test, default values of certain parameters and options controlling the behavior of the virtual clients and the format of reports. These global settings will be used if not overridden by the script run parameters or local HTTP request declarations.
Syntax example:
clients=5
Sets the amount of virtual clients.
Syntax example:
requests=10
Sets the number of runs of the main cycle [TestSeq] for each virtual client. E.g. if the settings are Clients=10, Requests=20 and there is a single HTTP request declared in [TestSeq], the overall number of [TestSeq] cycle runs performed by all virtual clients will make 10x20=200, thus 200 HTTP requests will be sent to the server. If there are 5 HTTP requests declared in [TestSeq], then the overall number of [TestSeq] runs in this test will remain the same - 10x20=200, but the amount of HTTP requests sent to the server will make 10x20x5=1000.
Syntax examples:
server=www.webapplicationstesting.com
server=194.149.225.75Name or IP address of the server being tested. E.g., www.webapplicationstesting.com or 194.149.225.75
Syntax example:
port=5555
The server port to which the requests are sent.
If equal to 1, turns on the use of HTTPS. You should understand that when you are testing an application using HTTPS it is impossible to record a test script by capturing the sequence of user actions in the browser. This is due to the fact that requests in this case are encrypted and Web Roller is unable to decrypt them. The test scripts for such a case can be developed using one of two possible alternatives:
During development of test scripts switch the tested application (if possible) to HTTP (instead of HTTPS). Record the sequence of user browser actions using the standard procedure, debug the testing scripts if required. After developing the test scripts, switch the application back to HTTPS and set the Secure option in the test scripts to 1. Develop the scripts manually or within the IDE (refer to Manual - “Manual development of test scripts”).
The maximum time of waiting for server response. Set in seconds! If you specify 0, a very big value (ca. 18 hours) will be used, but this will not completely switch off the check. Timeout=1 will most probably result in an error. When the response time exceeds the specified value of Timeout, the request is considered failed and is closed; the report will list "Ò/Î" instead of response time (if other symbols were not specified by the Timeouterrorsymbol setting).
The version of HTTP protocol used by the virtual clients.
1 – Version 1.1;
0 – Version 1.0.
Syntax example:
method=GET
The default method of HTTP request (GET or POST) used in the declared requests (see also $method).
Syntax example:
uri=/categories/category1.html
Default URI (see also $URI).
Determines whether to use keep-alive requests (see also $keepalive):
1 – make all requests keep-alive;
0 – make all requests non-keep-alive.
Delay after each request (milliseconds). Sets the delay inserted after each of requests within a sequence (InitSeq, FinalSeq, TestSeq).
Delay between starting times of different virtual clients. By default, if the number of virtual clients is big enough, the initial load on the server rises, but not instantly. E.g. if Clients=200, the chart of virtual clients' start times will look as follows,
potentially leading to poorer precision of the measured Requests per second value. If you set Delaystart to 0, all virtual clients will start exactly at the same time:
Limits the maximum overall amount of requests made by all virtual clients. When this number is reached, all virtual clients will be stopped - disregarding the amount of requests made by each particular client.
This setting can be useful when measuring the precise value of Requests per second. Usually the chart of server load chart with multiple virtual clients looks as follows:
The reason of the stepwise character of the load rise at the initial stage was described earlier (see Delaystart). The reason of the stepwise character of the final stage is similar: it is both the fact that not all virtual clients start at the same time, and the various work times of particular clients (which can differ significantly).
Web Roller calculates the Requests per second value in the following manner:
Requests per second = Total requests made / Working period
where:
Working period - the overall time spent for processing all requests - from the time of sending the very first request till the end of processing the last one;
Total requests made – the overall number of requests made. Calculated as:Clients x Requests x (Total number of requests in [TestSeq] section)
Thus the Requests per second value is measured incorrectly, since constant load is not provided during the Working period. You cannot provide accurate conclusions about application performance tested under a specific load (200 virtual clients in our case).
The method of avoiding the starting "steps" is described in Delaystart. The MaxRequests setting allows in turn to avoid the final "steps". The required actions are:
- Set an excessive value of Requests (e.g. 1 000 000);
- Set the MaxRequests value so that the amount is sufficient for correct calculation of Requests per second (i.e. if we set MaxRequests=10, then the value of Requests per second calculated using only 10 requests can be claimed to have a great calculation error and thus cannot be used as a realistic estimate; if we set MaxRequests=10000, then you can consider the calculation error to be acceptable).
It also a good idea to consider also the running time of the script when setting the MaxRequests value - e.g. if the script successfully performs 10000 requests in 5 seconds, than the Requests per second will be calculated with a noticeable error).
Attention!
You need to remember that Requests sets the amount of runs for the [TestSeq] sequence which can include multiple HTTP requests, while MaxRequests sets the maximum amount of processed HTTP requests for ALL clients. The MaxRequests value should be significantly less than the value of Total requests made (see above for the calculation of this value), calculated for the case when MaxRequests equals to 0 (i.e. not used).
When this value is non-zero, a special Delay mode is used. This is not a value, but a switch! If the Softdelay mode is active, the full time spent on processing the previous request (Previous request response time) is subtracted from the Delay value, yielding the Resulting delay value. The delay equal to the Resulting delay value will be inserted after the next request. The Resulting delay value is calculated dynamically and is not a constant. This mode allows to provide a more even server load, since constant time between sending requests to the server will be maintained in no dependence, of the response times. The approximate value of the time period for sending the requests to the server is set by the Delay value. If the response time is greater than the value of Delay, then the Resulting delay value for the next request will be set to 0.
This setting was introduced for convenience. This value specifies the desired amount of requests per hour FOR EACH CLIENT! It controls only the frequency of requests. The test must not necessarily run for an hour - it can be 10 minutes, but if it works one hour, the required amount of requests will be made (of course, if the response time is not too slow to keep the specified rate). This command is equivalent to calculating the required Delay value and turning on the SoftDelay mode. Caution! The Delay value should not be changed after setting Rate - otherwise you will obtain unexpected results (the new Delay value will override the one calculated using the Rate setting).
The name of the test run textual report file. Since MS Excel accepts data in Tab-Separated format used for the resulting report, the generated file has the xls extension, to avoid unneeded renaming.
Outfile can be set both in plain format and using the naming templates. The following macros can be used:
%s - the server setting of the General section;
%p - the port setting of the General section;
%r - the requests setting of the General section;
%c – the clients setting of the General section;
%f - the full name of the test script file;
%n - the name of the test script file (no extension);Current date/time macros:
%y - year;
%m – month;
%d – day;
%t – hhmmss;
%% - the % character.The default template for the file name is %n-%s-%cx%r.xls
The maximum time for running the test (seconds). The test will run until all requests will be processed, or until the run time exceeds Maxtime.
Caution!!!
For this setting to be active, the amount of requests should be big enough so that all requests are not processed before Maxtime elapses.
Determines whether to abort the current sequence [TestSeq] for all virtual clients if Maxtime is reached, or to wait for normal sequence termination.
Handling of redirects:
0 – redirects are processed;
1 - redirects are ignored.By default Web Roller handles redirects. Thus, if a redirect is obtained in response to a HTTP request, Web Roller will make an additional request to the location specified by the redirect. If yet another redirect is obtained, another request will be made, and so on. By default the total time of processing all redirects in a series is accounted as the response time for the current request. If some of the requests in this chain of redirects results in an error, the request is accounted as failed and the respective error code is written to the test report.
Defines whether to add up request times for redirected requests:
0 – redirect times are not added, only the response time of the very first request is written to the report;
1 - redirect times are added, the total for all redirects is written in the report.
International settings for the report. E.g. to use “,” as the decimal separator you should set the locale to Russian. The default decimal separator is a period, setting locale to Russian switches to a comma.
Toggle whether to print supplementary test info in the beginning of the report file - including the run date, amount of clients and requests, the user and workstation names of the computer which ran the test. If Projectname and Description are specified, these will be listed as well.
Information string. Used mainly for Reportinfo.
Information string. Used mainly for Reportinfo.
Upon launching the test script the results are either appended to the existing report file, or the existing file is replaced with the new one each time the test script is launched.
Defines whether to print Summary times in the report.
0 – do not print;
1 – printYou should not use Delay or other resource-consuming options (e.g. regular expressions) together with this option, otherwise the results will not be correct. Only the time period when all clients were launched and none of them has already terminated its run should be used for calculation of Summary times. If the amount of requests is not big enough, probably this requirement will not be met and the respective warning about results approximation will be added to the report.
Rounding precision.
10 – round to 0.1;
0 – no rounding, minimal possible value is 0.01 ms.
All the run results are written to the log file. Each virtual client produces a separate log file.
The comprehensive log of the Web Roller client run is created, including all requests, responses, redirects and the returned HTML code. If the response is obtained in chunked transfer encoding, it will be converted to normal presentation.
All HTTP requests generated by Web Roller will be made through the specified proxy server. Provide host:port.
Toggles the use of additional fields of the HTTP header for imitating different users working through a proxy server. Useful to emulate requests coming from multiple computers.
0 – do not use emulation;
1 – use emulation.
IP address mask used in Proxyemulation:
$C1 – low byte of the client number;
$C2 – high byte of the client number;
$R1 – low byte of the request number;
$R2 - high byte of the request number.All values span form 1 to 254. To emulate requests coming from unique IP's, set the mask to:
$C2.$C1.$R2.$R1
Obviously if the amount of requests exceeds 64262, the $Rx values will be repeated. You can use a static value instead of $C2 if the amount of clients is lower than 253.
Toggles whether to write the starting and ending times for each request cycle to the report:
0 – do not write;
1 – write.
A string (up to 8 characters) used in the report instead of response time if a HTTP error was detected (403, 404, 500 etc.). For instance, "HTTPERR".
A string (up to 8 characters) used in the report instead of response time if a network (socket) error was detected.
A string (up to 8 characters) used in the report instead of response time if the $fail error condition was detected.
A string (up to 8 characters) used in the report instead of response time if the response time exceeded the Timeout value.
Toggles whether to log the requests which lead to errors.
1 – write to the log;
0 – do not write to the log.
Toggles whether to write to the report the time of individual requests or only the summary statistics. It is a good idea to turn off the details when measuring overall performance, i.e. when a precise value of Requests per second is required.
Turns off the timeout check. Allows to provide significant server load using a single workstation, thanks to saved resources. Consequently, can lead to hang-ups of virtual clients in case of a big amount of stuck requests (which would be normally closed using the timeout check).
Sets the value of minimum time period (milliseconds) for simultaneous work of all clients required to correctly estimate SummaryTimes.
Defines whether to abort the sequence if an error was detected for a request. Useful for “negative” tests, i.e. when an error is an expected result.
The mode of time measurement:
0 – WebTransaction, the time period between the last request byte sent and the last response byte received;
1 – ResponseTime, the server response time. Works only under HTTP/1.1. Time period between the last request byte sent and the first response byte received;
2 – RequestTime, full request processing time. Time period between the first request byte sent and the last response byte received.
Sets the network connection bandwidth limit for virtual users (in kbps). Useful to emulate e.g. a modem connection.
Syntax:
authorization=login:pass
Basic authorization. Substitutes the specified login:pass in requests requiring basic authorization.
Toggles use of test scripting (inline or external). See details in Scripting.
Each HTTP request declared in the test uses the sending parameters set in the [General] section. But these parameters can be modified when required, and made unique for each request. The essence of these local settings is the same as for the respective [General] section ones. The parameter names are obtained by adding a $ character to the name of the respective parameter from the [General] section ($URI, $Server, $Port and so on). The locally defined parameters have a higher priority then the global ones, i.e. the global values will be used only if no local parameters were set. Examine the following test script:Example 1
[General]
Method=GET
Server=myserver
Port=5000
URI=/servlet
[Request1]
param1=value1
param2=value2
In this case Request1 will use the values of Server, Port, URI set globally (in the [General] section), since these are not locally redefined. The resulting HTTP request will look as follows:
GET /servlet?param1=value1¶m2=value2 HTTP/1.1
Host: myserver:5000Now let us redefine locally some parameters, and we will obtain
Example 2
[General]
Method=GET
Server=myserver
Port=5000
URI=/servlet
[Request1]
$Server=server2
$Port=8000
$URI=/servlet/view
param1=value1
param2=value2Here the Request1 will use the locally redefined values - $URI, $Port. The resulting HTTP request will be:
GET /servlet/view?param1=value1¶m2=value2 HTTP/1.1
Host: myserver:8000Not all of the settings from the [General] section can be redefined locally. The parameters that can be redefined are listed below.
Syntax:
$Server={servername}
Locally defined name (or IP) of the server to send the request to.
Syntax:
$Port={portnumber}
Locally defined server port to send the request to.
Syntax:
$URI={uri|$href(text)|$action|$var(name,default)}
Locally defined request URI. Unlike the globally defined one, it can use either a static value or one of the following functions:
$href(text) – searches for the link with the specified text in the HTML code of response to the previous request and takes the URI from this link;
Example:
The HTML code of the response contains:
<A href="/manual_3.html">preparing test scripts</a>
If we use in the subsequent request the $URI=$href(Preparing test scripts) function, we will finally obtain the actual URI of this link, which is equivalent to $URI=/manual_3.html.
$action – the first form in the respnonse to previous request is found and the URI is extracted from the form action.
Example:
The HTML code of the response contains:
<FORM NAME="search" METHOD="GET" ACTION="/cgi-bin/search.pl">
<INPUT TYPE="text" NAME="text" VALUE="" MAXLENGTH=80>
<INPUT TYPE="submit" VALUE="Start search">
If we use in the subsequent request the $URI=$action function, we will finally obtain the actual URI of this form action, which is equivalent to $URI=/cgi-bin/search.pl.
$var(name,default) – the URI value will be taken from the variable previously set by the $exportvar function. If the specified variable is undefined, the default value will be used.
Syntax:
$Method={GET|POST}
Request method.
Syntax:
$Keepalive={0|1}
Toggles the use of keep-alive requests.
Syntax:
$Delay={number}
Delay (in ms) after request execution.
Syntax:
$Fail=[!]/regexp/
If the specified regular expression is matched in the HTML code of the response to the current request, the request will be accounted as failed. (see Failerrorsymbol). If you place the ! character (exclamation point) before the regular expression, the result of matching will be inverted, i.e. the request will be accounted as failed if the response HTML code DOES NOT contain the matching pattern.
Example:
The test script is:
[General]
requests=5
server=www.yahoo.com
[Request1]
$uri=/
$method=GET
$fail=/Yahoo/The HTML code of the response contains:
<HTML><HEAD><TITLE>Yahoo!</TITLE>
If the request description contains $Fail=/Yahoo/, the request will be accounted as failed and the string specified by the Failerrorsymbol setting will be written to the report instead of request processing time. This setting is useful for functional testing.
Syntax:
$Exportvar=(varname),/regexp/,{replacestring}
Defines a variable and its value (see also $Var, $RegExp). Extracts the pattern matching the regular expression from the current response HTML code and stores it in a variable named varname. $0-$9 metacharacters can be used in {replacestring} to specify the groups in the text matched by the regular expression.
In this chapter the term "parameters" is used to describe the parameters specified to the right of the “?” sign in the HTTP request string. The functions are used to define the values of HTTP request parameters and provide the ability to dynamically set the required values for the Param=Value pairs. If a function is used, the declaration of a parameter looks like this: Param=$function. The functions substitute the resulting values, with one exception being the $rxname function (see below).
Now let us examine the available functions and the resulting values that can be generated.
Syntax:
ParamName=$NRand({from},{to}[,{prefix}])
A random integer from the [from,to] range; precede by a {prefix} (if one specified).
Example:
The test script is:
[General]
clients=1
requests=3
server=myserver
uri=/view
[Request1]
param=$NRand(1,20,ID)
If you run this script , three HTTP requests will be sent to the server:
http://myserver/view?param=ID5
http://myserver/view?param=ID12
http://myserver/view?param=ID2
The part of the param value after ID is generated randomly. If there is nothing specified instead of ID in the script, i.e. the script is as follows:
[General]
clients=1
requests=3
server=myserver
uri=/view
[Request1]
param=$NRand(1,20)
then three different requests will be sent to the server upon the script run:
http://myserver/view?param=5
http://myserver/view?param=12
http://myserver/view?param=2
Syntax:
ParamName=$ARand({from},{to})
Random alphanumeric sequence with length in the range [from, to].
Example:
Launching the test script below:
[General]
clients=1
requests=3
server=myserver
uri=/view
.. ..
[Request1]
param=$ARand(1,20)
results in issuing three HTTP requests:
http://myserver/view?param=g
http://myserver/view?param=fndy5lgs5pck
http://myserver/view?param=xfglfgjl4asd098ljk6c
Here all values of param are randomly generated.
Syntax:
ParamName=$LRand(value1[,{value2},…])
Random value from the list of values {value}.
Example:
Launching this test script:
[General]
clients=1
requests=3
server=myserver
uri=/view
.. ..
[Request1]
param=$LRand(test4,bobby,alpha)
Results in sending three HTTP requests to the server:
http://myserver/view?param=alpha
http://myserver/view?param=test4
http://myserver/view?param=bobby
Syntax:
ParamName=$List(value1[,{value2},…])
Similar to $LRand, but the elements are used in the order they are listed (not randomly). The 1st request uses the 1st value from the list, the 2nd one uses the second value and so on. After reaching the last value the list is looped, starting again from the first listed value.
Syntax:
ParamName=$Seq({prefix}[,{startvalue}[,{clientshift}]])
Augmenting sequence. Increases by 1 with each run of [TestSeq]. The initial value is {startvalue} (if specified, otherwize 0). If the {clientshift} is specified, then the starting value for different virtual clients will be shifted using this value. You can also specify several leading zeroes for the startvalue, so that the numeric component always keeps the same length.
Example 1:
The following test script:
[General]
clients=1
requests=3
server=myserver
uri=/view
.. ..
[Request1]
param=$Seq(test,2)
Results in sending three HTTP requests to the server:
http://myserver/view?param=test2
http://myserver/view?param=test3
http://myserver/view?param=test4
Example 2:
If the following test script is launched:
[General]
clients=3
requests=3
server=myserver
uri=/view
.. ..
[Request1]
param=$Seq(test,2,5)
the virtual client #1 will send three HTTP requests to the server:
http://myserver/view?param=test2
http://myserver/view?param=test3
http://myserver/view?param=test4
virtual client #2 will send three different HTTP requests:
http://myserver/view?param=test7
http://myserver/view?param=test8
http://myserver/view?param=test9
and virtual client #3, again, three different HTTP requests:
http://myserver/view?param=test12
http://myserver/view?param=test13
http://myserver/view?param=test14
Syntax:
ParamName=$UrlParam[({name})]
The response to the previous request will be searched for a GET request containing a parameter with name {name}. The value of this parameter will be substituted. If {name} is not explicitly set, then the name of the target parameter (ParamName) will be used - i.e., Myparam=$UrlParam is completely equivalent to Myparam=$UrlParam(Myparam).
Example 1:
The HTML code of the response contains:
<A href=”http://myserver/view?sessionid=fchhab02df¶m=test111”</a>
Use of Myparam=$UrlParam(SessionID) in the next request will result in the actual value of Myparam=FCHHAB02DF.
This function is very useful when you need to use the parameter values generated by the server on-the-fly. This relates, e.g., to various Session variables.Example 2:
You need to automate a simple scenario of navigating through a certain site using the respective HTTP requests:
1.User registration:
http://myserver/register?User=Alex&Pass=qwerty (a POST request)
2.After registration the user is redirected to the main category listing the links to all resources of the site. At this stage the SessionID session variable is generated. The user selected the Main category by clicking the respective link:
http://myserver/view?SessionID=FCHHAB02DF&Category=main
3.The user selects a link to a particular resource:
http://myserver/view?SessionID=FCHHAB02DF&Category=main&Item=67
If we record this sequence of HTTP request, it would result in the following test script:
[General]
clients=1
requests=1
server=myserver
[Req1]
$Uri=/register
$Method=POST
User=Alex
Pass=qwerty
[Req2]
$Uri=/view
$Method=GET
SessionID=FCHHAB02DF
Category=main
[Req3]
$Uri=/view
$Method=GET
SessionID=FCHHAB02DF
Category=main
Item=67
But we will not be able to use this script - after sending Req2 an error will occur. The reason is that we try to use the old value of SessionID. Consequently, we need that each time after user registration the test script operates with a correct (currently generated) value of SessionID. We can modify the script to achieve this:
[General]
clients=1
requests=1
server=myserver
[Req1]
$Uri=/register
$Method=POST
User=Alex
Pass=qwerty
[Req2]
$Uri=/view
$Method=GET
SessionID=$UrlParam(SessionID)
Category=main
[Req3]
$Uri=/view
$Method=GET
SessionID=$UrlParam(SessionID)
Category=main
Item=67
and thus solve the problems related to SessionID. Now before sending the Req2 request the $UrlParam function will parse the response (the returned HTML code) to request Req1, find the first link containing the required SessionID and substitute the actual value of SessionID into Req2 (in the Example this value is FCHHAB02DF).
Syntax:
ParamName=$Hidden[({name})]
The function searches for an element containing <.... name=”{name}” value=”{value}”> using the specified {name}. The found {value} will be used. If {name} is not specified explicitly, the name of the target parameter (ParamName) will be used. The syntax of this type is used typically to define input elements in HTML forms. The advantage of this function is that it allows to work e.g. with hidden form elements with dynamically generated values (again the Session variables).
Example:
The response to the previous HTTP request contains a form with a set of hidden elements:
<form action="/cgi-bin/select.pl" method="get">
<input type="text" name="index" size="25" maxlength="25">
<input type="submit" value="Search">
<input type="hidden" name="restrict1" value="yes">
<input type="hidden" name="restrict3" value="no">
....
....
</form>
and the values of these elements should be used in the subsequent HTTP request. Let us compose a test script for solving this problem:
[General]
clients=1
requests=1
server=myserver
[Req1]
$Uri=/
[Req2]
$Uri=/cgi-bin/select.pl
$Method=GET
index=AnyText
restrict1=$Hidden
restrict3=$Hidden
In the test script above, before sending the Req2 request the $Hidden function will parse the HTML code of response to request Req1. For the restrict1 parameter the function will find the fragment <….name="restrict1" value=……>, for restrict3 - respectively, <….name="restrict3" value=……>. Respective values will be returned and used as values of parameters restrict1 and restrict3 in Req2.
Syntax:
ParamName=$RegExp(/{regexp}/,{replacestring})
The response to the previous request will be scanned for a match to the specified regular expression {regexp}, and the resulting function value will be built according to the search results and the {replacestring} syntax. The syntax is similar to Perl syntax (see description in Syntax Reference ->Regular Expressions).
Example:
You need to automate the submitting of a form with comboboxes. Only the description of combobox values are static (in this case - minimal, middle and maximum), but the values are generated by the server and not initially known. The HTML code of such a form may look like this:
<form action=”send” method=”post”>
.....
.....
<select name="weight">
<option value="20">minimal</option>
<option value="40">middle</option>
<option value="60">maximum</option>
</select>
</form>
Let us compose a test script for solving this problem:
[General]
clients=1
requests=1
server=myserver
[Req1]
$Uri=/
[Req2]
$Uri=/send
$Method=POST
weight=$RegExp(/value=\"(\d)\">minimal</,$1)
Before sending the Req2 request $RegExp parses the HTML code of response to the Req1 request. For the weight parameter it will find the fragment <option value="20">minimal</option>, which matches the regular expression. The value 20 will be extracted and returned by the $RegExp function.
Syntax:
$RxName(/{regexp}/,{replacestring},{value})
Modifies the parameter name according to the regular expression. No matter which parameter name was initially used in the script, it will be replaced by the result built according to {replacestring}. The rules are the same as for $regexp(). The {value} can be any correct value used in the request parameters, e.g. the ones listed above.
Example:
You need to submit a form for which the names of input elements are dynamically generated by the server, and thus initially unknown. The form is like follows:
<form action="send">
<input type="checkbox" name="Option124">Food
<input type="checkbox" name="Option78">Juice
<input type="checkbox" name="Option471">Biscuit
</form>
The test script for performing this task looks as follows:
[General]
clients=1
requests=1
server=myserver
[Req1]
$Uri=/
[Req2]
$Uri=/send
AnyText=$rxname(/name="(Option\d+)">Juice/i,$1),on
Before sending the Req2 request the $RxName function parses the HTML code returned in response to Req1 and finds the code fragment name="Option78">Juice matching the regular expression. The value Option78 is extracted and used as the parameter name, the value of this parameter is then set to on.
Syntax:
ParamName=$Var({name}[{,default}])
Uses a variable set earlier by the $exportvar function. If the variable with name {name} is not defined, the {default} value will be used instead (if present).
Example:
Let us examine once again the example for the $RegExp function and solve the problem in a different manner. You need to automate the submitting of a form with comboboxes. Only the description of combobox values are static (in this case - minimal, middle and maximum), but the values are generated by the server and not initially known. The HTML of such a form may look like this:
<form action=”send” method=”post”>
.....
.....
<select name="weight">
<option value="20">minimal</option>
<option value="40">middle</option>
<option value="60">maximum</option>
</select>
</form>
Let us compose a test script for solving this problem:
[General]
clients=1
requests=1
server=myserver
[Req1]
$Uri=/
$ExportVar=VarName,/value=\"(\d)\">minimal</,$1
[Req2]
$Uri=/send
$Method=POST
weight=$Var(VarName)
After sending the Req1 request the $ExportVar function parses the HTML code returned in response to Req1 and finds the code fragment <option value="20">minimal</option> which matches the regular expression. The value 20 is extracted and assigned to the VarName variable. Before sending the Req2 request the $Var function returns the value of the VarName variable (defined earlier), and assigns this value to the weight parameter.
All functions parsing the HTML code of the responses have one common limitation - they can operate only with the code of the previous request. Defining and using external variables with $ExportVar and $Var functions allows to overcome this limitation and transfer data between non-subsequent HTTP requests.
Syntax:
$File({path filename}[{,mimetype}])
File upload. Multipart/form-data will be used. If {mimetype} is not explicitly specified, application/octet-stream will be used. Used to upload a file to the server, e.g. when sending e-mail messages with attachments via the Web interface of the mailing system.
Example:
Usually the code for file upload looks as follows:
<form method="post" action="upload" enctype="multipart/form-data">
<input type="file" name="filename">
</form>
If you need to upload the JPEG image image.jpg located in the C:\Graphics\Jpg\ folder, the test script will look as the one below:
[General]
clients=1
requests=1
server=myserver
[Req1]
$Uri=/upload
$Method=POST
filename=$File(C:\Graphics\Jpg\image.jpg,image/pjpeg)
Syntax:
ParamName=$Clid({prefix})
{prefix} plus the ordinal number of the virtual client. Can be used to create values which should be different for different clients, but the same for a single client.
Syntax:
Paramname=$gvar(OptionName, DefaultValue)
Defines the parameter value when running as a Win32 console application (see Console mode).
OptionName – the name of command line option,
DefaultValue – default value of $gvar function.Upon the launch of a test script woth a $gvar function the latter substitutes the value of the respective option from the command line (or the default value, if the requested OptionName was not found in the command line). Can be used for defining any parameters (or their components). CANNOT be used to define a parameter name (i.e., cannot stay to the left of the “=” sign).
Examples (correct use):
Server=$gvar(server, myserver)
$Server=$gvar(server, myserver)
param=$gvar(param, value1)
param=abc$gvar(param, value1)Example (incorrect use):
$gvar(paramname, value)=value9
When a HTTP request is listed in any of the sequences (InitSeq, FinalSeq, TestSeq), both the request sending parameters and the values of its local parameters can be set or redefined.
The syntax is as follows:
[TestSeq]
Request1 (param1=value, param2=value2, $uri=servlet)Example:
Launching the following test script:
[General]
Method=GET
Server=myserver
Port=5000
URI=/servlet
[Request1]
param1=value1
param2=value2will result in sending the following HTTP request:
Now let us redefine several parameters when calling Request1, and we will obtain the following example:GET /servlet?param1=value1¶m2=value2 HTTP/1.1
Host: myserver:5000[General]
Method=GET
Server=myserver
Port=5000
URI=/servlet
[TestSeq]
Request1 (param1=value3, $URI=/servlet/view, port=5555)
[Request1]
param1=value1
param2=value2
Launching this script will result in sending the following HTTP request:
GET /servlet/view?param1=value3¶m2=value2 HTTP/1.1
Host: myserver:5555
You can modify all request parameters (except for $exportvar and $fail) by setting request parameters within a sequence.
The support of scripting in Web Roller allows to replace the sequence of requests with a program.
Web Roller scripting is implemented via the ActiveScriptingHost, i.e. any scripting language using the respective interface can be used. Currently JScript, VBScript and PerlScript (the latter requires installation of ActiveState ActivePerl) are supported. The name of the script file is set by the Script parameter of the [General] section. If this parameter is specified, the scripting is turned on automatically. It is also possible to include the script text in the body of a test script. In this case $inline.ext should be used as the name of script file.
Caution!: It is important to specify a correct extension - it is used to determine the script language:
- Perl - .pl
- Jscript - .js
- VBScript - .vbs
Scripting imposes certain limitations:
- [InitSeq], [FinalSeq] and [TestSeq] sequences are ignored;
- $Seq and $List functions cannot be used;
- The following parameters are ignored: requests (to use requests you will need to set up a cycle - see Example 2 in GetOpt); timestamp; maxtime; abortseq; nodetails; maxrequests; exacttiminginterval; ignoreerrors;
The rest of the settings and functions described earlier remain functional. Web Roller transfers the control to the script program and does not issue any requests itself.
To provide operation with HTTP requests the wapt object is added to the script's namespace, offering the following methods:
int AddRequest(string RequestName, string Uri);
Declare a new HTTP request with name RequestName. Returns 1 upon success, 0 if a reauest with this name already exists.
Example:
$wapt->AddRequest("test","/");
$wapt->AddRequest("test","http://myserver");
int AddParam(string ReqãuestName, string ParamName, string ParamValue);
Adds the Param=Value pair in the RequestName request.
Example:
$wapt->AddParam("test","param1","value4");
int SetParam (string RequestName, string Param[, string Value]);
Sets the value of existing Param parameter in the existing RequestName request. If Value is not specified, then the parameter will be reset to the original value specified in the request declaration. Returns 1 upon success, otherwise 0.
Example:
$wapt->SetParam("test","param1","value9");
int ResetRequest (string RequestName)
Restores the initial request parameters values modified by SetParam, AddParam for the RequestName request.
Example:
$wapt->AddParam("test","param1","value4");
$wapt->SetParam("test","param1","value9");
$wapt->ResetRequest("test");
int Request (string RequestName);
Issue a RequestName request. Returns 0 if the request does not exist (or unsuccessful), 1 upon successful request termination.
Example:
$wapt->Request("test");
int SetRequestOpt(string RequestName, string Options);
Sets the parameters ($uri, $server, $port, $delay, $method, $keepalive) for the RequestName request. The Options string should be in the same format as when redefining parameters from within a sequence.
Note: If Perl is used, the Options string should be in single quotes:
$wapt->SetRequestOpt("test",’$uri=/view’);
string GetOpt (string Option);
returns the value of the Option parameter from the [General] section. Additionallly returns clid – the ordinal number of the current virtual client.
Example:
$wapt->GetOpt("requests");
string Response ();
Returns the response to the last HTTP request.
int IsAborting ();
Returns 1 if Ctrl-Break was pressed, otherwise 0. If you want the script to properly terminate on pressing, this value should be checked frequently.
void Logit (string logstr);
Writes the logstr string to the logfile, if logging is on ([General] section, Log=1).
Example1. Primitive Perl script:
[General]
script=$inline.pl
[script]
$wapt->AddRequest("test","http://myserver/");
$wapt->Request("test");
[/script]
This will result in sending the following HTTP request to the server:
GET / HTTP/1.1
Host: myserverExample2. Using the Requests parameter
[General]
log=1
script=$inline.pl
requests=3
[script]
$wapt->AddRequest("test","http://myserver/");
for($i=0;$i<$wapt->GetOpt("requests");$i++)
{
$wapt->Request("test");
}
[/script]
After the execution of a test script the textual and graphical execution reports are created.
The textual report is generated as a file with .xls extension saved in the current folder; the concise version of the report is also presented on the Report->Report tab.
Let us examine a typical execution report. Henceforward all comments and explanations are in bold:
Report generated by Web Roller v3.5 build 939 |
|||||
Test started at:Sun Mar 31 20:41:26 2002 |
|||||
Test finished at:Sun Mar 31 20:41:26 2002 |
|||||
Project name: Test project |
Project name (Projectname setting) |
||||
Description: Test |
Project desctription (Description parameter) |
||||
Server name: kort |
Server tested (Server parameter) |
||||
Clients: 3 |
Number of virtual clients |
||||
Requests: 5 |
Number of [TestSeq] runs |
||||
Test executed by: Igor |
User name |
||||
Test executed at: kort |
Workstation name |
||||
Clnt/Rqst Req1 Req2 Date and time of request execution (if timestamp=1) |
|||||
C00R00000 5.07 51.06 2002/03/31 20:41:26 2002/03/31 20:41:26 |
C00R00000 – Client0, Request0. First run of [TestSeq] by virtual client #1, processing times for each request from [TestSeq] are listed in milliseconds. |
||||
C00R00001 0.22 125.02 2002/03/31 20:41:26 2002/03/31 20:41:26 |
|||||
C00R00002 61.18 89.67 2002/03/31 20:41:26 2002/03/31 20:41:26 |
|||||
C00R00003 42.41 112.37 2002/03/31 20:41:26 2002/03/31 20:41:26 |
|||||
C00R00004 6.81 69.35 2002/03/31 20:41:26 2002/03/31 20:41:26 |
|||||
C01R00000 4.57 89.27 2002/03/31 20:41:26 2002/03/31 20:41:26 |
|||||
C01R00001 32.19 90.65 2002/03/31 20:41:26 2002/03/31 20:41:26 |
|||||
C01R00002 22.72 79.29 2002/03/31 20:41:26 2002/03/31 20:41:26 |
|||||
C01R00003 41.30 114.60 2002/03/31 20:41:26 2002/03/31 20:41:26 |
|||||
C01R00004 26.73 81.37 2002/03/31 20:41:26 2002/03/31 20:41:26 |
|||||
C02R00000 16.14 68.92 2002/03/31 20:41:26 2002/03/31 20:41:26 |
|||||
C02R00001 6.52 80.96 2002/03/31 20:41:26 2002/03/31 20:41:26 |
|||||
C02R00002 29.99 99.36 2002/03/31 20:41:26 2002/03/31 20:41:26 |
|||||
C02R00003 26.81 61.01 2002/03/31 20:41:26 2002/03/31 20:41:26 |
|||||
C02R00004 18.99 62.73 2002/03/31 20:41:26 2002/03/31 20:41:26 |
|||||
--- Basic statistics --- |
Minimum, average and maximum processing times for each of HTTP requests (Req1, Req2) - milliseconds. |
||||
Min: 0.22 51.06 |
|||||
Avg: 22.78 85.04 |
|||||
Max: 61.18 125.02 |
|||||
--- Network traffic details --- |
|||||
Total bytes sent : 2604 |
Total amount of data sent and received. |
||||
Total bytes received : 25320 |
|||||
--- Summary times --- |
|||||
Virtual Clients statistics: |
Virtual clients statistics: Count – amount of HTTP requests sent; Time – virtual client operation time; AR/S – average amount of HTTP requests per second; AT/R – average time of HTTP request processing. |
||||
Count Time AR/S AT/R |
|||||
10 601 16.64 60.10 |
|||||
10 601 16.64 60.10 |
|||||
10 500 20.00 50.00 |
|||||
Total work time: 631 |
Script operation time |
||||
Total requests made: 30 |
Total amount of HTTP requests made by all virtual clients |
||||
Total average time per request: 21.0333 |
Average HTTP request processing time (using all requests) |
||||
Total average requests per second: 47.5436 |
Average amount of HTTP requests per second |
||||
WARNING: approximation used! |
The warning is displayed if the overall time of the test script operation is lower than required for precise measurement (and calculation) of the “Total average time per request “ and “Total average requests per second” values. In this case the results will be approximated. |
||||
Not enough requests per client for exact timing! |
|||||
--- HTTP response codes details --- |
Statistics on HTTP response codes: |
||||
Code Count |
|||||
200 30 |
If scripting is used, the textual execution report is simplified. Execution of the script below:
[General]
project=Test Project
Description=test
script=$inline.pl
requests=3
clients=2
[script]
$wapt->AddRequest("test1","http://myserver/");
$wapt->AddRequest("test2","http://myserver/");
for($i=0;$i<$wapt->GetOpt("requests");$i++){
$wapt->Request("test1");
$wapt->Request("test2");
}
[/script]
will result in the following report:
Report generated by Web Roller v3.5 build 957 |
|
Test started at:Fri Apr 19 16:28:42 2002 |
|
Test finished at:Fri Apr 19 16:28:43 2002 |
|
Project name: Test Project |
Project name (Projectname setting) |
Description: Test |
Project desctription (Description parameter) |
Server name: myserver |
Server tested (Server parameter) |
Clients: 2 |
Number of virtual clients |
Requests: 3 |
Number of [TestSeq] runs |
Test executed by: Igor |
Computer user name |
Test executed at: Kort |
Computer workstation name |
--- Client 0 --- |
Statistics on processed requests for virtual client “Client 0” |
test1 09.20 |
|
test2 12.10 |
|
test1 11.60 |
|
test2 09.70 |
|
test1 12.10 |
|
test2 11.60 |
|
--- Client 1 --- |
Statistics on processed requests for virtual client “Client 1” |
test1 09.20 |
|
test2 12.10 |
|
test1 11.60 |
|
test2 09.70 |
|
test1 12.10 |
|
test2 11.60 |
|
--- HTTP response codes details --- |
Statistics on HTTP response codes: |
Code Count |
|
200 12 |
If in the [General] section Log=1, all the results of operation are logged to a file. Each virtual client generates a separate log file. The complete log of the Web Roller client operation is written to this file. This includes all requests, responses, redirects and the HTML code returned.
A test script like this:
[General]
clients=1
requests=2
server=kort
log=1
[TestSeq]
Req1
Req2
[Req1]
$uri=/
$method=GET
[Req2]
$uri=/view.pl
$method=GET
will result in the following log file:
----- [TestSeq]: Req1(0) ----- |
Separator line: pass #0 of [TestSeq], request Req1 |
Request: |
Header of the HTTP request sent |
GET / HTTP/1.1 |
|
Host: kort |
|
Connection: close |
|
Response: |
Header of the server response. |
HTTP/1.1 200 OK |
|
Date: Sun, 21 Apr 2002 16:30:45 GMT |
|
Server: Apache/1.3.12 (Win32) |
|
Connection: close |
|
Transfer-Encoding: chunked |
|
Content-Type: text/html |
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"> |
HTML code of response |
<html> |
|
</body> |
|
.. |
|
.. |
|
</body> |
|
</html> |
|
----- [TestSeq]: Req2 (0) ----- |
Separator line: pass #0 of [TestSeq], request Req2 |
Request: |
|
GET /view.pl HTTP/1.1 |
|
Host: kort |
|
Referrer: http://kort/ |
|
Connection: close |
|
Response: |
|
HTTP/1.1 200 OK |
|
Date: Sun, 21 Apr 2002 16:30:45 GMT |
|
Server: Apache/1.3.12 (Win32) |
|
Connection: close |
|
Transfer-Encoding: chunked |
|
Content-Type: text/plain |
|
<html> |
|
</body> |
|
.. |
|
.. |
|
</body> |
|
</html> |
|
----- [TestSeq]: Req1(1) ----- |
Separator line: pass #1 of [TestSeq], request Req1 |
Request: |
|
GET / HTTP/1.1 |
|
Host: kort |
|
Referrer: http://kort/view.pl |
|
Connection: close |
|
Response: |
|
HTTP/1.1 200 OK |
|
Date: Sun, 21 Apr 2002 16:30:45 GMT |
|
Server: Apache/1.3.12 (Win32) |
|
Connection: close |
|
Transfer-Encoding: chunked |
|
Content-Type: text/html |
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"> |
|
<html> |
|
</body> |
|
.. |
|
.. |
|
</body> |
|
</html> |
|
----- [TestSeq]: Req2(1) ----- |
Separator line: pass #1 of [TestSeq], request Req2 |
Request: |
|
GET /view.pl HTTP/1.1 |
|
Host: kort |
|
Referrer: http://kort/ |
|
Connection: close |
|
Response: |
|
HTTP/1.1 200 OK |
|
Date: Sun, 21 Apr 2002 16:30:45 GMT |
|
Server: Apache/1.3.12 (Win32) |
|
Connection: close |
|
Transfer-Encoding: chunked |
|
Content-Type: text/plain |
|
<html> |
|
</body> |
|
.. |
|
.. |
|
</body> |
|
</html> |
The wr.exe executable can be used to launch test scripts and projects in console mode (from the command line). To launch a test project enter:
wr.exe projectname.wpj
To launch a single test script enter:
wr.exe testscriptname.tst [-option=value,., , ]
[Options] can be set if the $gvar function is used within the test script.
Example:
The server parameter in the test script testscript.tst is set as follows:
Server=$gvar(server, myserver)
If we launch the script this way:
wr.exe testscript.tst
the requests will be sent to myserver.
But if we launch the script as:
wr.exe testscript.tst – server=testserver
then the requests will be sent to testserver.
To ensure correctness and sufficient precision of the measurements, the testing procedure should meet the following requirements:
Recommended settings for the [General] section:
[General] |
|
delaystart=0 |
Provides simultaneous start of virtual clients (see description of delaystart) |
maxrequests=50000 |
The maxrequests value should not exceed the product (clients x requests). Required to eliminate the “final steps” (see the description of maxrequests) |
nowait=0 |
Do not use the timeout check. Allows to provide significant server load using a single workstation, thanks to saved resources. Consequently, can lead to hang-ups of virtual clients in case of a big amount of stuck requests (which would be normally closed using the timeout check).. |
log=0 |
Logging is off - to reduce resource consumption of the workstation running the test. |
nodetails=1 |
Turns off the printing of detailed statistics on separate requests for all virtual clients. Useful if measuring only the value of Requests per second. |
summarytimes=1 |
Turns on printing of the summary statistics |
Example: a test script for measuring the performance of loading the main page of a Web server:
[General]
clients=50
requests=10000
server=myserver
keepalive=1
delaystart=0
maxrequests=50000
nowait=0
log=0
nodetails=1
summarytimes=1
[TestSeq]
Req1
[Req1]
$uri=/
A typical mistake when automating scripts using session variables is using the static value for the session variable (obtained by recording the sequence of HTTP requests with the Record function) while these session variables are unique for each session; thus an attempt of running such a test script will lead to errors (since other sessions will generate different values of session variables). To provide proper functioning of test scripts in cases like this you need to extract respective session variables from the responses to previous requests (see the description of $UrlParam, $Hidden, $RegExp).
Usually when test scripts with redirects are recorded, the redirects are not included in Web Roller's proxy log. But if “meta http-equiv” redirects are used, then the situation is possible when the log will include both the initial pages (containing redirects) and the redirects themselves. In this cases the extra redirects should be removed from the log manually.