Web Roller - description of parameters and functions

Global settings

The [General] section defines global settings of the test, default values of certain parameters and options controlling the behavior of the virtual clients and the format of reports. These global settings will be used if not overridden by the script run parameters or local HTTP request declarations.


Clients

Syntax example:

clients=5

Sets the amount of virtual clients.


Requests

Syntax example:

requests=10

Sets the number of runs of the main cycle [TestSeq] for each virtual client. E.g. if the settings are Clients=10, Requests=20 and there is a single HTTP request declared in [TestSeq], the overall number of [TestSeq] cycle runs performed by all virtual clients will make 10x20=200, thus 200 HTTP requests will be sent to the server. If there are 5 HTTP requests declared in [TestSeq], then the overall number of [TestSeq] runs in this test will remain the same - 10x20=200, but the amount of HTTP requests sent to the server will make 10x20x5=1000.


Server

Syntax examples:

server=www.webapplicationstesting.com
server=194.149.225.75

Name or IP address of the server being tested. E.g., www.webapplicationstesting.com or 194.149.225.75


Port

Syntax example:

port=5555

The server port to which the requests are sent.


Secure

If equal to 1, turns on the use of HTTPS. You should understand that when you are testing an application using HTTPS it is impossible to record a test script by capturing the sequence of user actions in the browser. This is due to the fact that requests in this case are encrypted and Web Roller is unable to decrypt them. The test scripts for such a case can be developed using one of two possible alternatives:

During development of test scripts switch the tested application (if possible) to HTTP (instead of HTTPS). Record the sequence of user browser actions using the standard procedure, debug the testing scripts if required. After developing the test scripts, switch the application back to HTTPS and set the Secure option in the test scripts to 1. Develop the scripts manually or within the IDE (refer to Manual - “Manual development of test scripts”).


Timeout

The maximum time of waiting for server response. Set in seconds! If you specify 0, a very big value (ca. 18 hours) will be used, but this will not completely switch off the check. Timeout=1 will most probably result in an error. When the response time exceeds the specified value of Timeout, the request is considered failed and is closed; the report will list "Ò/Î" instead of response time (if other symbols were not specified by the Timeouterrorsymbol setting).


Httpver

The version of HTTP protocol used by the virtual clients.

1 – Version 1.1;
0 – Version 1.0.


Method

Syntax example:

method=GET

The default method of HTTP request (GET or POST) used in the declared requests (see also $method).


Uri

Syntax example:

uri=/categories/category1.html

Default URI (see also $URI).


Keepalive

Determines whether to use keep-alive requests (see also $keepalive):

1 – make all requests keep-alive;
0 – make all requests non-keep-alive.


Delay

Delay after each request (milliseconds). Sets the delay inserted after each of requests within a sequence (InitSeq, FinalSeq, TestSeq).


Delaystart

Delay between starting times of different virtual clients. By default, if the number of virtual clients is big enough, the initial load on the server rises, but not instantly. E.g. if Clients=200, the chart of virtual clients' start times will look as follows,

potentially leading to poorer precision of the measured Requests per second value. If you set Delaystart to 0, all virtual clients will start exactly at the same time:


MaxRequests

Limits the maximum overall amount of requests made by all virtual clients. When this number is reached, all virtual clients will be stopped - disregarding the amount of requests made by each particular client.

This setting can be useful when measuring the precise value of Requests per second. Usually the chart of server load chart with multiple virtual clients looks as follows:

The reason of the stepwise character of the load rise at the initial stage was described earlier (see Delaystart). The reason of the stepwise character of the final stage is similar: it is both the fact that not all virtual clients start at the same time, and the various work times of particular clients (which can differ significantly).

Web Roller calculates the Requests per second value in the following manner:

Requests per second = Total requests made / Working period

where:

Working period - the overall time spent for processing all requests - from the time of sending the very first request till the end of processing the last one;
Total requests made – the overall number of requests made. Calculated as:

Clients x Requests x (Total number of requests in [TestSeq] section)

Thus the Requests per second value is measured incorrectly, since constant load is not provided during the Working period. You cannot provide accurate conclusions about application performance tested under a specific load (200 virtual clients in our case).

The method of avoiding the starting "steps" is described in Delaystart. The MaxRequests setting allows in turn to avoid the final "steps". The required actions are:

It also a good idea to consider also the running time of the script when setting the MaxRequests value - e.g. if the script successfully performs 10000 requests in 5 seconds, than the Requests per second will be calculated with a noticeable error).

Attention!

You need to remember that Requests sets the amount of runs for the [TestSeq] sequence which can include multiple HTTP requests, while MaxRequests sets the maximum amount of processed HTTP requests for ALL clients. The MaxRequests value should be significantly less than the value of Total requests made (see above for the calculation of this value), calculated for the case when MaxRequests equals to 0 (i.e. not used).


Softdelay

When this value is non-zero, a special Delay mode is used. This is not a value, but a switch! If the Softdelay mode is active, the full time spent on processing the previous request (Previous request response time) is subtracted from the Delay value, yielding the Resulting delay value. The delay equal to the Resulting delay value will be inserted after the next request. The Resulting delay value is calculated dynamically and is not a constant. This mode allows to provide a more even server load, since constant time between sending requests to the server will be maintained in no dependence, of the response times. The approximate value of the time period for sending the requests to the server is set by the Delay value. If the response time is greater than the value of Delay, then the Resulting delay value for the next request will be set to 0.


Rate

This setting was introduced for convenience. This value specifies the desired amount of requests per hour FOR EACH CLIENT! It controls only the frequency of requests. The test must not necessarily run for an hour - it can be 10 minutes, but if it works one hour, the required amount of requests will be made (of course, if the response time is not too slow to keep the specified rate). This command is equivalent to calculating the required Delay value and turning on the SoftDelay mode. Caution! The Delay value should not be changed after setting Rate - otherwise you will obtain unexpected results (the new Delay value will override the one calculated using the Rate setting).


Outfile

The name of the test run textual report file. Since MS Excel™ accepts data in Tab-Separated format used for the resulting report, the generated file has the xls extension, to avoid unneeded renaming.

Outfile can be set both in plain format and using the naming templates. The following macros can be used:

%s - the server setting of the General section;
%p - the port setting of the General section;
%r - the requests setting of the General section;
%c – the clients setting of the General section;
%f - the full name of the test script file;
%n - the name of the test script file (no extension);

Current date/time macros:

%y - year;
%m – month;
%d – day;
%t – hhmmss;

%% - the % character.

The default template for the file name is %n-%s-%cx%r.xls


Maxtime

The maximum time for running the test (seconds). The test will run until all requests will be processed, or until the run time exceeds Maxtime.

Caution!!!

For this setting to be active, the amount of requests should be big enough so that all requests are not processed before Maxtime elapses.


Abortseq

Determines whether to abort the current sequence [TestSeq] for all virtual clients if Maxtime is reached, or to wait for normal sequence termination.


Noredir

Handling of redirects:

0 – redirects are processed;
1 - redirects are ignored.

By default Web Roller handles redirects. Thus, if a redirect is obtained in response to a HTTP request, Web Roller will make an additional request to the location specified by the redirect. If yet another redirect is obtained, another request will be made, and so on. By default the total time of processing all redirects in a series is accounted as the response time for the current request. If some of the requests in this chain of redirects results in an error, the request is accounted as failed and the respective error code is written to the test report.


Sumredirecttime

Defines whether to add up request times for redirected requests:

0 – redirect times are not added, only the response time of the very first request is written to the report;
1 - redirect times are added, the total for all redirects is written in the report.


Locale

International settings for the report. E.g. to use “,” as the decimal separator you should set the locale to Russian. The default decimal separator is a period, setting locale to Russian switches to a comma.


Printdate, Reportinfo

Toggle whether to print supplementary test info in the beginning of the report file - including the run date, amount of clients and requests, the user and workstation names of the computer which ran the test. If Projectname and Description are specified, these will be listed as well.


Projectname

Information string. Used mainly for Reportinfo.


Description

Information string. Used mainly for Reportinfo.


Appendresult

Upon launching the test script the results are either appended to the existing report file, or the existing file is replaced with the new one each time the test script is launched.


Summarytimes

Defines whether to print Summary times in the report.

0 – do not print;
1 – print

You should not use Delay or other resource-consuming options (e.g. regular expressions) together with this option, otherwise the results will not be correct. Only the time period when all clients were launched and none of them has already terminated its run should be used for calculation of Summary times. If the amount of requests is not big enough, probably this requirement will not be met and the respective warning about results approximation will be added to the report.


Round

Rounding precision.

10 – round to 0.1;
0 – no rounding, minimal possible value is 0.01 ms.


Log

All the run results are written to the log file. Each virtual client produces a separate log file.

The comprehensive log of the Web Roller client run is created, including all requests, responses, redirects and the returned HTML code. If the response is obtained in chunked transfer encoding, it will be converted to normal presentation.


Proxy

All HTTP requests generated by Web Roller will be made through the specified proxy server. Provide host:port.


Proxyemulation

Toggles the use of additional fields of the HTTP header for imitating different users working through a proxy server. Useful to emulate requests coming from multiple computers.

0 – do not use emulation;
1 – use emulation.


Proxymask

IP address mask used in Proxyemulation:

$C1 – low byte of the client number;
$C2 – high byte of the client number;
$R1 – low byte of the request number;
$R2 - high byte of the request number.

All values span form 1 to 254. To emulate requests coming from unique IP's, set the mask to:

$C2.$C1.$R2.$R1

Obviously if the amount of requests exceeds 64262, the $Rx values will be repeated. You can use a static value instead of $C2 if the amount of clients is lower than 253.


Timestamp

Toggles whether to write the starting and ending times for each request cycle to the report:

0 – do not write;
1 – write.


Httperrorsymbol

A string (up to 8 characters) used in the report instead of response time if a HTTP error was detected (403, 404, 500 etc.). For instance, "HTTPERR".


Socketerrorsymbol

A string (up to 8 characters) used in the report instead of response time if a network (socket) error was detected.


Failerrorsymbol

A string (up to 8 characters) used in the report instead of response time if the $fail error condition was detected.


Timeouterrorsymbol

A string (up to 8 characters) used in the report instead of response time if the response time exceeded the Timeout value.


Showbadresults

Toggles whether to log the requests which lead to errors.

1 – write to the log;
0 – do not write to the log.


NoDetails

Toggles whether to write to the report the time of individual requests or only the summary statistics. It is a good idea to turn off the details when measuring overall performance, i.e. when a precise value of Requests per second is required.


NoWait

Turns off the timeout check. Allows to provide significant server load using a single workstation, thanks to saved resources. Consequently, can lead to hang-ups of virtual clients in case of a big amount of stuck requests (which would be normally closed using the timeout check).


ExactTimingInterval

Sets the value of minimum time period (milliseconds) for simultaneous work of all clients required to correctly estimate SummaryTimes.


IgnoreErrors

Defines whether to abort the sequence if an error was detected for a request. Useful for “negative” tests, i.e. when an error is an expected result.


TimingMode

The mode of time measurement:

0WebTransaction, the time period between the last request byte sent and the last response byte received;
1ResponseTime, the server response time. Works only under HTTP/1.1. Time period between the last request byte sent and the first response byte received;
2RequestTime, full request processing time. Time period between the first request byte sent and the last response byte received.


MaxSpeed

Sets the network connection bandwidth limit for virtual users (in kbps). Useful to emulate e.g. a modem connection.


Authorization

Syntax:

authorization=login:pass

Basic authorization. Substitutes the specified login:pass in requests requiring basic authorization.


Script

Toggles use of test scripting (inline or external). See details in Scripting.


Defining request parameters locally

Each HTTP request declared in the test uses the sending parameters set in the [General] section. But these parameters can be modified when required, and made unique for each request. The essence of these local settings is the same as for the respective [General] section ones. The parameter names are obtained by adding a $ character to the name of the respective parameter from the [General] section ($URI, $Server, $Port and so on). The locally defined parameters have a higher priority then the global ones, i.e. the global values will be used only if no local parameters were set. Examine the following test script:

Example 1

[General]

Method=GET
Server=myserver
Port=5000
URI=/servlet
[Request1]
param1=value1
param2=value2

 

In this case Request1 will use the values of Server, Port, URI set globally (in the [General] section), since these are not locally redefined. The resulting HTTP request will look as follows:

GET /servlet?param1=value1&param2=value2 HTTP/1.1
Host: myserver:5000

Now let us redefine locally some parameters, and we will obtain

Example 2

[General]

Method=GET
Server=myserver
Port=5000
URI=/servlet

[Request1]

$Server=server2
$Port=8000
$URI=/servlet/view
param1=value1
param2=value2

Here the Request1 will use the locally redefined values - $URI, $Port. The resulting HTTP request will be:

GET /servlet/view?param1=value1&param2=value2 HTTP/1.1
Host: myserver:8000

Not all of the settings from the [General] section can be redefined locally. The parameters that can be redefined are listed below.


$Server

Syntax:

$Server={servername}

Locally defined name (or IP) of the server to send the request to.


$Port

Syntax:

$Port={portnumber}

Locally defined server port to send the request to.


$URI

Syntax:

$URI={uri|$href(text)|$action|$var(name,default)}

Locally defined request URI. Unlike the globally defined one, it can use either a static value or one of the following functions:

$href(text) – searches for the link with the specified text in the HTML code of response to the previous request and takes the URI from this link;

Example:

The HTML code of the response contains:

<A href="/manual_3.html">preparing test scripts</a>

If we use in the subsequent request the $URI=$href(Preparing test scripts) function, we will finally obtain the actual URI of this link, which is equivalent to $URI=/manual_3.html.

$action – the first form in the respnonse to previous request is found and the URI is extracted from the form action.

Example:

The HTML code of the response contains:

<FORM NAME="search" METHOD="GET" ACTION="/cgi-bin/search.pl">
<INPUT TYPE="text" NAME="text" VALUE="" MAXLENGTH=80>
<INPUT TYPE="submit" VALUE="Start search">

If we use in the subsequent request the $URI=$action function, we will finally obtain the actual URI of this form action, which is equivalent to $URI=/cgi-bin/search.pl.

$var(name,default) – the URI value will be taken from the variable previously set by the $exportvar function. If the specified variable is undefined, the default value will be used.


$Method

Syntax:

$Method={GET|POST}

Request method.


$Keepalive

Syntax:

$Keepalive={0|1}

Toggles the use of keep-alive requests.


$Delay

Syntax:

$Delay={number}

Delay (in ms) after request execution.


$Fail

Syntax:

$Fail=[!]/regexp/

If the specified regular expression is matched in the HTML code of the response to the current request, the request will be accounted as failed. (see Failerrorsymbol). If you place the ! character (exclamation point) before the regular expression, the result of matching will be inverted, i.e. the request will be accounted as failed if the response HTML code DOES NOT contain the matching pattern.

Example:

The test script is:

[General]

requests=5
server=www.yahoo.com

[Request1]

$uri=/
$method=GET
$fail=/Yahoo/

The HTML code of the response contains:

<HTML><HEAD><TITLE>Yahoo!</TITLE>

If the request description contains $Fail=/Yahoo/, the request will be accounted as failed and the string specified by the Failerrorsymbol setting will be written to the report instead of request processing time. This setting is useful for functional testing.


$Exportvar

Syntax:

$Exportvar=(varname),/regexp/,{replacestring}

Defines a variable and its value (see also $Var, $RegExp). Extracts the pattern matching the regular expression from the current response HTML code and stores it in a variable named varname. $0-$9 metacharacters can be used in {replacestring} to specify the groups in the text matched by the regular expression.


Functions as request parameters

In this chapter the term "parameters" is used to describe the parameters specified to the right of the “?” sign in the HTTP request string. The functions are used to define the values of HTTP request parameters and provide the ability to dynamically set the required values for the Param=Value pairs. If a function is used, the declaration of a parameter looks like this: Param=$function. The functions substitute the resulting values, with one exception being the $rxname function (see below).

Now let us examine the available functions and the resulting values that can be generated.


$NRand

Syntax:

ParamName=$NRand({from},{to}[,{prefix}])

A random integer from the [from,to] range; precede by a {prefix} (if one specified).

Example:

The test script is:

[General]
clients=1
requests=3
server=myserver
uri=/view

[Request1]

param=$NRand(1,20,ID)

If you run this script , three HTTP requests will be sent to the server:

http://myserver/view?param=ID5
http://myserver/view?param=ID12
http://myserver/view?param=ID2

The part of the param value after ID is generated randomly. If there is nothing specified instead of ID in the script, i.e. the script is as follows:

[General]

clients=1
requests=3
server=myserver
uri=/view

[Request1]
param=$NRand(1,20)

then three different requests will be sent to the server upon the script run:

http://myserver/view?param=5
http://myserver/view?param=12
http://myserver/view?param=2

 


$ARand

Syntax:

ParamName=$ARand({from},{to})

Random alphanumeric sequence with length in the range [from, to].

Example:

Launching the test script below:

[General]
clients=1
requests=3
server=myserver
uri=/view
.. ..

[Request1]

param=$ARand(1,20)

results in issuing three HTTP requests:

http://myserver/view?param=g
http://myserver/view?param=fndy5lgs5pck
http://myserver/view?param=xfglfgjl4asd098ljk6c

Here all values of param are randomly generated.


$LRand

Syntax:

ParamName=$LRand(value1[,{value2},…])

Random value from the list of values {value}.

Example:

Launching this test script:

[General]

clients=1
requests=3
server=myserver
uri=/view
.. ..

[Request1]

param=$LRand(test4,bobby,alpha)

Results in sending three HTTP requests to the server:

http://myserver/view?param=alpha
http://myserver/view?param=test4
http://myserver/view?param=bobby

 


$List

Syntax:

ParamName=$List(value1[,{value2},…])

Similar to $LRand, but the elements are used in the order they are listed (not randomly). The 1st request uses the 1st value from the list, the 2nd one uses the second value and so on. After reaching the last value the list is looped, starting again from the first listed value.


$Seq

Syntax:

ParamName=$Seq({prefix}[,{startvalue}[,{clientshift}]])

Augmenting sequence. Increases by 1 with each run of [TestSeq]. The initial value is {startvalue} (if specified, otherwize 0). If the {clientshift} is specified, then the starting value for different virtual clients will be shifted using this value. You can also specify several leading zeroes for the startvalue, so that the numeric component always keeps the same length.

Example 1:

The following test script:

[General]
clients=1
requests=3
server=myserver
uri=/view
.. ..

[Request1]

param=$Seq(test,2)

Results in sending three HTTP requests to the server:

http://myserver/view?param=test2
http://myserver/view?param=test3
http://myserver/view?param=test4

Example 2:

If the following test script is launched:

[General]

clients=3
requests=3
server=myserver
uri=/view
.. ..

[Request1]

param=$Seq(test,2,5)

the virtual client #1 will send three HTTP requests to the server:

http://myserver/view?param=test2
http://myserver/view?param=test3
http://myserver/view?param=test4

virtual client #2 will send three different HTTP requests:

http://myserver/view?param=test7
http://myserver/view?param=test8
http://myserver/view?param=test9

and virtual client #3, again, three different HTTP requests:

http://myserver/view?param=test12
http://myserver/view?param=test13
http://myserver/view?param=test14


$UrlParam

Syntax:

ParamName=$UrlParam[({name})]

The response to the previous request will be searched for a GET request containing a parameter with name {name}. The value of this parameter will be substituted. If {name} is not explicitly set, then the name of the target parameter (ParamName) will be used - i.e., Myparam=$UrlParam is completely equivalent to Myparam=$UrlParam(Myparam).

Example 1:

The HTML code of the response contains:

<A href=”http://myserver/view?sessionid=fchhab02df&param=test111”</a>

Use of Myparam=$UrlParam(SessionID) in the next request will result in the actual value of Myparam=FCHHAB02DF.
This function is very useful when you need to use the parameter values generated by the server on-the-fly. This relates, e.g., to various Session variables.

Example 2:

You need to automate a simple scenario of navigating through a certain site using the respective HTTP requests:

1.User registration:

http://myserver/register?User=Alex&Pass=qwerty (a POST request)

2.After registration the user is redirected to the main category listing the links to all resources of the site. At this stage the SessionID session variable is generated. The user selected the Main category by clicking the respective link:

http://myserver/view?SessionID=FCHHAB02DF&Category=main

3.The user selects a link to a particular resource:

http://myserver/view?SessionID=FCHHAB02DF&Category=main&Item=67

If we record this sequence of HTTP request, it would result in the following test script:

[General]
clients=1
requests=1
server=myserver

[Req1]
$Uri=/register
$Method=POST
User=Alex
Pass=qwerty

[Req2]
$Uri=/view
$Method=GET
SessionID=FCHHAB02DF
Category=main

[Req3]
$Uri=/view
$Method=GET
SessionID=FCHHAB02DF
Category=main
Item=67

But we will not be able to use this script - after sending Req2 an error will occur. The reason is that we try to use the old value of SessionID. Consequently, we need that each time after user registration the test script operates with a correct (currently generated) value of SessionID. We can modify the script to achieve this:

[General]
clients=1
requests=1
server=myserver

[Req1]
$Uri=/register
$Method=POST
User=Alex
Pass=qwerty

[Req2]
$Uri=/view
$Method=GET
SessionID=$UrlParam(SessionID)
Category=main

[Req3]
$Uri=/view
$Method=GET
SessionID=$UrlParam(SessionID)
Category=main
Item=67

and thus solve the problems related to SessionID. Now before sending the Req2 request the $UrlParam function will parse the response (the returned HTML code) to request Req1, find the first link containing the required SessionID and substitute the actual value of SessionID into Req2 (in the Example this value is FCHHAB02DF).


$Hidden

Syntax:

ParamName=$Hidden[({name})]

The function searches for an element containing <.... name=”{name}” value=”{value}”> using the specified {name}. The found {value} will be used. If {name} is not specified explicitly, the name of the target parameter (ParamName) will be used. The syntax of this type is used typically to define input elements in HTML forms. The advantage of this function is that it allows to work e.g. with hidden form elements with dynamically generated values (again the Session variables).

Example:

The response to the previous HTTP request contains a form with a set of hidden elements:

<form action="/cgi-bin/select.pl" method="get">
<input type="text" name="index" size="25" maxlength="25">
<input type="submit" value="Search">
<input type="hidden" name="restrict1" value="yes">
<input type="hidden" name="restrict3" value="no">
....
....
</form>

and the values of these elements should be used in the subsequent HTTP request. Let us compose a test script for solving this problem:

[General]
clients=1
requests=1
server=myserver

[Req1]
$Uri=/

[Req2]
$Uri=/cgi-bin/select.pl
$Method=GET
index=AnyText
restrict1=$Hidden
restrict3=$Hidden

In the test script above, before sending the Req2 request the $Hidden function will parse the HTML code of response to request Req1. For the restrict1 parameter the function will find the fragment <….name="restrict1" value=……>, for restrict3 - respectively, <….name="restrict3" value=……>. Respective values will be returned and used as values of parameters restrict1 and restrict3 in Req2.


$RegExp

Syntax:

ParamName=$RegExp(/{regexp}/,{replacestring})

The response to the previous request will be scanned for a match to the specified regular expression {regexp}, and the resulting function value will be built according to the search results and the {replacestring} syntax. The syntax is similar to Perl syntax (see description in Syntax Reference ->Regular Expressions).

Example:

You need to automate the submitting of a form with comboboxes. Only the description of combobox values are static (in this case - minimal, middle and maximum), but the values are generated by the server and not initially known. The HTML code of such a form may look like this:

<form action=”send” method=”post”>
.....
.....
<select name="weight">
<option value="20">minimal</option>
<option value="40">middle</option>
<option value="60">maximum</option>
</select>
</form>

Let us compose a test script for solving this problem:

[General]
clients=1
requests=1
server=myserver

[Req1]
$Uri=/

[Req2]
$Uri=/send
$Method=POST
weight=$RegExp(/value=\"(\d)\">minimal</,$1)

Before sending the Req2 request $RegExp parses the HTML code of response to the Req1 request. For the weight parameter it will find the fragment <option value="20">minimal</option>, which matches the regular expression. The value 20 will be extracted and returned by the $RegExp function.


$RxName

Syntax:

$RxName(/{regexp}/,{replacestring},{value})

Modifies the parameter name according to the regular expression. No matter which parameter name was initially used in the script, it will be replaced by the result built according to {replacestring}. The rules are the same as for $regexp(). The {value} can be any correct value used in the request parameters, e.g. the ones listed above.

Example:

You need to submit a form for which the names of input elements are dynamically generated by the server, and thus initially unknown. The form is like follows:

<form action="send">
<input type="checkbox" name="Option124">Food
<input type="checkbox" name="Option78">Juice
<input type="checkbox" name="Option471">Biscuit
</form>

The test script for performing this task looks as follows:

[General]
clients=1
requests=1
server=myserver

[Req1]
$Uri=/

[Req2]
$Uri=/send
AnyText=$rxname(/name="(Option\d+)">Juice/i,$1),on

Before sending the Req2 request the $RxName function parses the HTML code returned in response to Req1 and finds the code fragment name="Option78">Juice matching the regular expression. The value Option78 is extracted and used as the parameter name, the value of this parameter is then set to on.


$Var

Syntax:

ParamName=$Var({name}[{,default}])

Uses a variable set earlier by the $exportvar function. If the variable with name {name} is not defined, the {default} value will be used instead (if present).

Example:

Let us examine once again the example for the $RegExp function and solve the problem in a different manner. You need to automate the submitting of a form with comboboxes. Only the description of combobox values are static (in this case - minimal, middle and maximum), but the values are generated by the server and not initially known. The HTML of such a form may look like this:

<form action=”send” method=”post”>
.....
.....
<select name="weight">
<option value="20">minimal</option>
<option value="40">middle</option>
<option value="60">maximum</option>
</select>
</form>

Let us compose a test script for solving this problem:

[General]
clients=1
requests=1
server=myserver

[Req1]
$Uri=/
$ExportVar=VarName,/value=\"(\d)\">minimal</,$1

[Req2]
$Uri=/send
$Method=POST
weight=$Var(VarName)

After sending the Req1 request the $ExportVar function parses the HTML code returned in response to Req1 and finds the code fragment <option value="20">minimal</option> which matches the regular expression. The value 20 is extracted and assigned to the VarName variable. Before sending the Req2 request the $Var function returns the value of the VarName variable (defined earlier), and assigns this value to the weight parameter.

All functions parsing the HTML code of the responses have one common limitation - they can operate only with the code of the previous request. Defining and using external variables with $ExportVar and $Var functions allows to overcome this limitation and transfer data between non-subsequent HTTP requests.


$File

Syntax:

$File({path filename}[{,mimetype}])

File upload. Multipart/form-data will be used. If {mimetype} is not explicitly specified, application/octet-stream will be used. Used to upload a file to the server, e.g. when sending e-mail messages with attachments via the Web interface of the mailing system.

Example:

Usually the code for file upload looks as follows:

<form method="post" action="upload" enctype="multipart/form-data">
<input type="file" name="filename">
</form>

If you need to upload the JPEG image image.jpg located in the C:\Graphics\Jpg\ folder, the test script will look as the one below:

[General]
clients=1
requests=1
server=myserver

[Req1]
$Uri=/upload
$Method=POST
filename=$File(C:\Graphics\Jpg\image.jpg,image/pjpeg)


$Clid

Syntax:

ParamName=$Clid({prefix})

{prefix} plus the ordinal number of the virtual client. Can be used to create values which should be different for different clients, but the same for a single client.


$GVar

Syntax:

Paramname=$gvar(OptionName, DefaultValue)

Defines the parameter value when running as a Win32 console application (see Console mode).

OptionName – the name of command line option,
DefaultValue
– default value of $gvar function.

Upon the launch of a test script woth a $gvar function the latter substitutes the value of the respective option from the command line (or the default value, if the requested OptionName was not found in the command line). Can be used for defining any parameters (or their components). CANNOT be used to define a parameter name (i.e., cannot stay to the left of the “=” sign).

Examples (correct use):

Server=$gvar(server, myserver)
$Server=$gvar(server, myserver)
param=$gvar(param, value1)
param=abc$gvar(param, value1)

Example (incorrect use):

$gvar(paramname, value)=value9


Passing request parameters from a sequence

When a HTTP request is listed in any of the sequences (InitSeq, FinalSeq, TestSeq), both the request sending parameters and the values of its local parameters can be set or redefined.

The syntax is as follows:

[TestSeq]
Request1 (param1=value, param2=value2, $uri=servlet)

Example:

Launching the following test script:

[General]
Method=GET
Server=myserver
Port=5000
URI=/servlet

[Request1]
param1=value1
param2=value2

will result in sending the following HTTP request:

GET /servlet?param1=value1&param2=value2 HTTP/1.1
Host: myserver:5000

Now let us redefine several parameters when calling Request1, and we will obtain the following example:

[General]
Method=GET
Server=myserver
Port=5000
URI=/servlet

[TestSeq]
Request1 (param1=value3, $URI=/servlet/view, port=5555)

[Request1]
param1=value1
param2=value2

Launching this script will result in sending the following HTTP request:

GET /servlet/view?param1=value3&param2=value2 HTTP/1.1
Host: myserver:5555

You can modify all request parameters (except for $exportvar and $fail) by setting request parameters within a sequence.


Scripting

The support of scripting in Web Roller allows to replace the sequence of requests with a program.

Web Roller scripting is implemented via the ActiveScriptingHost, i.e. any scripting language using the respective interface can be used. Currently JScript, VBScript and PerlScript (the latter requires installation of ActiveState ActivePerl) are supported. The name of the script file is set by the Script parameter of the [General] section. If this parameter is specified, the scripting is turned on automatically. It is also possible to include the script text in the body of a test script. In this case $inline.ext should be used as the name of script file.

Caution!: It is important to specify a correct extension - it is used to determine the script language:

Scripting imposes certain limitations:

The rest of the settings and functions described earlier remain functional. Web Roller transfers the control to the script program and does not issue any requests itself.

To provide operation with HTTP requests the wapt object is added to the script's namespace, offering the following methods:


AddRequest

int AddRequest(string RequestName, string Uri);

Declare a new HTTP request with name RequestName. Returns 1 upon success, 0 if a reauest with this name already exists.

Example:

$wapt->AddRequest("test","/");
$wapt->AddRequest("test","http://myserver");


AddParam

int AddParam(string ReqãuestName, string ParamName, string ParamValue);

Adds the Param=Value pair in the RequestName request.

Example:

$wapt->AddParam("test","param1","value4");


SetParam

int SetParam (string RequestName, string Param[, string Value]);

Sets the value of existing Param parameter in the existing RequestName request. If Value is not specified, then the parameter will be reset to the original value specified in the request declaration. Returns 1 upon success, otherwise 0.

Example:

$wapt->SetParam("test","param1","value9");


ResetRequest

int ResetRequest (string RequestName)

Restores the initial request parameters values modified by SetParam, AddParam for the RequestName request.

Example:

$wapt->AddParam("test","param1","value4");
$wapt->SetParam("test","param1","value9");
$wapt->ResetRequest("test");


Request

int Request (string RequestName);

Issue a RequestName request. Returns 0 if the request does not exist (or unsuccessful), 1 upon successful request termination.

Example:

$wapt->Request("test");


SetRequestOpt

int SetRequestOpt(string RequestName, string Options);

Sets the parameters ($uri, $server, $port, $delay, $method, $keepalive) for the RequestName request. The Options string should be in the same format as when redefining parameters from within a sequence.

Note: If Perl is used, the Options string should be in single quotes:

$wapt->SetRequestOpt("test",’$uri=/view’);


GetOpt

string GetOpt (string Option);

returns the value of the Option parameter from the [General] section. Additionallly returns clid – the ordinal number of the current virtual client.

Example:

$wapt->GetOpt("requests");


Response

string Response ();

Returns the response to the last HTTP request.


IsAborting

int IsAborting ();

Returns 1 if Ctrl-Break was pressed, otherwise 0. If you want the script to properly terminate on pressing, this value should be checked frequently.


Logit

void Logit (string logstr);

Writes the logstr string to the logfile, if logging is on ([General] section, Log=1).


Examples

Example1. Primitive Perl script:

[General]
script=$inline.pl

[script]
$wapt->AddRequest("test","http://myserver/");
$wapt->Request("test");
[/script]

This will result in sending the following HTTP request to the server:

GET / HTTP/1.1
Host: myserver

Example2. Using the Requests parameter

[General]
log=1
script=$inline.pl
requests=3

[script]
$wapt->AddRequest("test","http://myserver/");
for($i=0;$i<$wapt->GetOpt("requests");$i++)
{
$wapt->Request("test");
}
[/script]


Testing results and assessment

After the execution of a test script the textual and graphical execution reports are created.


Execution report

The textual report is generated as a file with .xls extension saved in the current folder; the concise version of the report is also presented on the Report->Report tab.

Let us examine a typical execution report. Henceforward all comments and explanations are in bold:

Report generated by Web Roller v3.5 build 939

 
   

Test started at:Sun Mar 31 20:41:26 2002

 

Test finished at:Sun Mar 31 20:41:26 2002

 

Project name: Test project

Project name (Projectname setting)

Description: Test

Project desctription (Description parameter)

Server name: kort

Server tested (Server parameter)

Clients: 3

Number of virtual clients

Requests: 5

Number of [TestSeq] runs

Test executed by: Igor

User name

Test executed at: kort

Workstation name

   

Clnt/Rqst Req1 Req2 Date and time of request execution (if timestamp=1)

C00R00000 5.07 51.06 2002/03/31 20:41:26 2002/03/31 20:41:26

C00R00000 – Client0, Request0. First run of [TestSeq] by virtual client #1, processing times for each request from [TestSeq] are listed in milliseconds.

C00R00001 0.22 125.02 2002/03/31 20:41:26 2002/03/31 20:41:26

C00R00002 61.18 89.67 2002/03/31 20:41:26 2002/03/31 20:41:26

C00R00003 42.41 112.37 2002/03/31 20:41:26 2002/03/31 20:41:26

C00R00004 6.81 69.35 2002/03/31 20:41:26 2002/03/31 20:41:26

C01R00000 4.57 89.27 2002/03/31 20:41:26 2002/03/31 20:41:26

C01R00001 32.19 90.65 2002/03/31 20:41:26 2002/03/31 20:41:26

C01R00002 22.72 79.29 2002/03/31 20:41:26 2002/03/31 20:41:26

C01R00003 41.30 114.60 2002/03/31 20:41:26 2002/03/31 20:41:26

C01R00004 26.73 81.37 2002/03/31 20:41:26 2002/03/31 20:41:26

C02R00000 16.14 68.92 2002/03/31 20:41:26 2002/03/31 20:41:26

C02R00001 6.52 80.96 2002/03/31 20:41:26 2002/03/31 20:41:26

C02R00002 29.99 99.36 2002/03/31 20:41:26 2002/03/31 20:41:26

C02R00003 26.81 61.01 2002/03/31 20:41:26 2002/03/31 20:41:26

C02R00004 18.99 62.73 2002/03/31 20:41:26 2002/03/31 20:41:26

   

--- Basic statistics ---

Minimum, average and maximum processing times for each of HTTP requests (Req1, Req2) - milliseconds.

 

Min: 0.22 51.06

Avg: 22.78 85.04

Max: 61.18 125.02

   

--- Network traffic details ---

 
   

Total bytes sent : 2604

Total amount of data sent and received.

Total bytes received : 25320

--- Summary times ---

 
   

Virtual Clients statistics:

Virtual clients statistics:

Count – amount of HTTP requests sent;

Time – virtual client operation time;

AR/S – average amount of HTTP requests per second;

AT/R – average time of HTTP request processing.

Count Time AR/S AT/R

10 601 16.64 60.10

10 601 16.64 60.10

10 500 20.00 50.00

   

Total work time: 631

Script operation time

Total requests made: 30

Total amount of HTTP requests made by all virtual clients

   

Total average time per request: 21.0333

Average HTTP request processing time (using all requests)

Total average requests per second: 47.5436

Average amount of HTTP requests per second

   

WARNING: approximation used!

The warning is displayed if the overall time of the test script operation is lower than required for precise measurement (and calculation) of the “Total average time per request “ and “Total average requests per second” values. In this case the results will be approximated.

Not enough requests per client for exact timing!

   

--- HTTP response codes details ---

Statistics on HTTP response codes:
Code – response code;
Count – amount of responses with this code.

 

Code Count

200 30

 


Execution report (if scripting is used)

If scripting is used, the textual execution report is simplified. Execution of the script below:

[General]
project=Test Project
Description=test
script=$inline.pl
requests=3
clients=2

[script]
$wapt->AddRequest("test1","http://myserver/");
$wapt->AddRequest("test2","http://myserver/");
for($i=0;$i<$wapt->GetOpt("requests");$i++){
$wapt->Request("test1");
$wapt->Request("test2");
}
[/script]

will result in the following report:

Report generated by Web Roller v3.5 build 957

 
   

Test started at:Fri Apr 19 16:28:42 2002

 

Test finished at:Fri Apr 19 16:28:43 2002

 

Project name: Test Project

Project name (Projectname setting)

Description: Test

Project desctription (Description parameter)

Server name: myserver

Server tested (Server parameter)

Clients: 2

Number of virtual clients

Requests: 3

Number of [TestSeq] runs

Test executed by: Igor

Computer user name

Test executed at: Kort

Computer workstation name

   

--- Client 0 ---

Statistics on processed requests for virtual client “Client 0”

test1 09.20

test2 12.10

test1 11.60

test2 09.70

test1 12.10

test2 11.60

   

--- Client 1 ---

Statistics on processed requests for virtual client “Client 1”

test1 09.20

test2 12.10

test1 11.60

test2 09.70

test1 12.10

test2 11.60

   

--- HTTP response codes details ---

Statistics on HTTP response codes:
Code – response code;
Count – amount of responses with this code.

 

Code Count

200 12

 


Execution Log

If in the [General] section Log=1, all the results of operation are logged to a file. Each virtual client generates a separate log file. The complete log of the Web Roller client operation is written to this file. This includes all requests, responses, redirects and the HTML code returned.

A test script like this:

[General]
clients=1
requests=2
server=kort
log=1

[TestSeq]
Req1
Req2

[Req1]
$uri=/
$method=GET

[Req2]
$uri=/view.pl
$method=GET

will result in the following log file:

----- [TestSeq]: Req1(0) -----

Separator line: pass #0 of [TestSeq], request Req1

Request:

Header of the HTTP request sent

GET / HTTP/1.1

Host: kort

Connection: close

   

Response:

Header of the server response.

HTTP/1.1 200 OK

Date: Sun, 21 Apr 2002 16:30:45 GMT

Server: Apache/1.3.12 (Win32)

Connection: close

Transfer-Encoding: chunked

Content-Type: text/html

   

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">

HTML code of response

 

<html>

</body>

..

..

</body>

</html>

   

----- [TestSeq]: Req2 (0) -----

Separator line: pass #0 of [TestSeq], request Req2

Request:

 

GET /view.pl HTTP/1.1

 

Host: kort

 

Referrer: http://kort/

 

Connection: close

 
   

Response:

 

HTTP/1.1 200 OK

 

Date: Sun, 21 Apr 2002 16:30:45 GMT

 

Server: Apache/1.3.12 (Win32)

 

Connection: close

 

Transfer-Encoding: chunked

 

Content-Type: text/plain

 
   

<html>

 

</body>

 

..

 

..

 

</body>

 

</html>

 

----- [TestSeq]: Req1(1) -----

Separator line: pass #1 of [TestSeq], request Req1

Request:

 

GET / HTTP/1.1

 

Host: kort

 

Referrer: http://kort/view.pl

 

Connection: close

 
   

Response:

 

HTTP/1.1 200 OK

 

Date: Sun, 21 Apr 2002 16:30:45 GMT

 

Server: Apache/1.3.12 (Win32)

 

Connection: close

 

Transfer-Encoding: chunked

 

Content-Type: text/html

 
   

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">

 
   

<html>

 

</body>

 

..

 

..

 

</body>

 

</html>

 
   

----- [TestSeq]: Req2(1) -----

Separator line: pass #1 of [TestSeq], request Req2

Request:

 

GET /view.pl HTTP/1.1

 

Host: kort

 

Referrer: http://kort/

 

Connection: close

 
   

Response:

 

HTTP/1.1 200 OK

 

Date: Sun, 21 Apr 2002 16:30:45 GMT

 

Server: Apache/1.3.12 (Win32)

 

Connection: close

 

Transfer-Encoding: chunked

 

Content-Type: text/plain

 
   

<html>

 

</body>

 

..

 

..

 

</body>

 

</html>

 

 


Launching test scripts and projects from Win32 console

The wr.exe executable can be used to launch test scripts and projects in console mode (from the command line). To launch a test project enter:

wr.exe projectname.wpj

To launch a single test script enter:

wr.exe testscriptname.tst [-option=value,., , ]

[Options] can be set if the $gvar function is used within the test script.

Example:

The server parameter in the test script testscript.tst is set as follows:

Server=$gvar(server, myserver)

If we launch the script this way:

wr.exe testscript.tst

the requests will be sent to myserver.

But if we launch the script as:

wr.exe testscript.tst – server=testserver

then the requests will be sent to testserver.


Test script examples

Performance measurements

To ensure correctness and sufficient precision of the measurements, the testing procedure should meet the following requirements:

[General]

 

delaystart=0

Provides simultaneous start of virtual clients (see description of delaystart)

maxrequests=50000

The maxrequests value should not exceed the product (clients x requests). Required to eliminate the “final steps” (see the description of maxrequests)

nowait=0

Do not use the timeout check. Allows to provide significant server load using a single workstation, thanks to saved resources. Consequently, can lead to hang-ups of virtual clients in case of a big amount of stuck requests (which would be normally closed using the timeout check)..

log=0

Logging is off - to reduce resource consumption of the workstation running the test.

nodetails=1

Turns off the printing of detailed statistics on separate requests for all virtual clients. Useful if measuring only the value of Requests per second.

summarytimes=1

Turns on printing of the summary statistics

Example: a test script for measuring the performance of loading the main page of a Web server:

[General]
clients=50
requests=10000
server=myserver
keepalive=1
delaystart=0
maxrequests=50000
nowait=0
log=0
nodetails=1
summarytimes=1

[TestSeq]
Req1

[Req1]
$uri=/


Working with session variables

A typical mistake when automating scripts using session variables is using the static value for the session variable (obtained by recording the sequence of HTTP requests with the Record function) while these session variables are unique for each session; thus an attempt of running such a test script will lead to errors (since other sessions will generate different values of session variables). To provide proper functioning of test scripts in cases like this you need to extract respective session variables from the responses to previous requests (see the description of $UrlParam, $Hidden, $RegExp).


Working with redirects

Usually when test scripts with redirects are recorded, the redirects are not included in Web Roller's proxy log. But if “meta http-equiv” redirects are used, then the situation is possible when the log will include both the initial pages (containing redirects) and the redirects themselves. In this cases the extra redirects should be removed from the log manually.