Power-Programmierung/CD2.mdf/c/library/dos/arrays/xarray/manual.txt

Text File | 1992-07-11 | 16.3 KB | 538 lines

    Introduction to Virtual Arrays in C

VIRTUAL AND HUGE ARRAYS Version 2.03
by Graham Robertson


    The Virtual Array manager in this package is designed
specifically to allow more memory than is normally available to
the C programmer by caching memory blocks to DISK or extended
memory. This is done by the use of a virtual memory manager
which transparently allows the programmer to access arrays of
any size. This not only makes programming of C programs with
large amounts of data easier but makes is portable to other
architecture's.

    Several example programs are included in the package which
quickly show the easy of use of the array manager. The array
manager is simplest to use when used with C++.

    The example programs are as follows

C++ programs
--------------------------------------------------------------
    ARRAYS.EXE         - Example of lots of Virtual
Arrays
    TARRAYS.EXE        - Same examples but for compilers
that have templates


C programs

--------------------------------------------------------------
    HUGEARR.EXE        - A huge array
    VIRARRAY.EXE        - Example of lots of Virtual
Arrays


--------------------------------------------------------------

This software is copyright British Software Licensing 1991-
1992. The usual disclaimers apply.

The number of virtual arrays is restricted to sixteen in the
unregistered version.

If you use this software please send 20 pounds to British
Software Licensing. If you require a version which allows
unrestricted virtual arrays please register your copy of this
software. A copy of the source code can be obtained by sending
50 pounds to British Software Licensing.

This price is for a single user license. A site license is
available at a very reasonable price.

If you use this software in any other Shareware or commercial
application please contact the author to make arrangements.



--------------------------------------------------------------
Order Form - Single User License only
---------------------------------------------------------------
To    British Software Licensing
    280 (T/L) West Princes Street,
    Woodlands,
    Glasgow
    United Kingdom
    G4 9EU

(prices are the same anywhere in the World)
please supply me with:

an unrestricted copy of the large array manager  [   ]20 pounds

or

a copy of the source for the virtual array manger[   ]50 pounds

                            -------------
Total payment
                            --------------


Payment type    ACCESS/VISA/MASTERCARD/CHECK(U.K. pounds)/POSTAL
ORDER

Name on Card


Card Number


Expiry date


Signature of Cardholder

Name
Address






--------------------------------------------------------------

You may also purchase this software from:
Graham Robertson on
                    041-339-8855x5021
or on an Answering Machine    041-339-7264
or via Email        [e-mail]
or by Fax on             041-334-1675
quoting the details required above.


--------------------------------------------------------------


 Files in the package.
VIRIMG.LIB        - Software to allow virtual and large arrays
VIRIMG.H            - Prototypes of procedures for creating
virtual arrays

VIRARRAY.C        - Source code of an example of using virtual
arrays
VIRARRAY.EXE        - Example of using virtual arrays
                  Compiled VIRARRAY.C linked with VIRIMG.LIB

HUGEARR.C        - Source code of an example of using a huge
array
HUGEARR.EXE        - Example of using a huge array

XARRAY.H            - C++ header for virtual arrays

ARRAYS.CPP        - Part One of the C++ example program
A2.CPP            - Part Two of the C++ example program

TXARRAY.H        - header with templates
TARRAYS.CPP        - part one with templates
TA2.CPP            - part two with templates


--------------------------------------------------------------
All the code here is compiled using the large data model
and Borland C or Turbo C compiler
--------------------------------------------------------------

This software is use by example, so the comments in this
document mainly explain the procedures that have been developed
to enable the use of huge and virtual arrays.




1.    Using virtual/extended arrays in C++

    This section describes using the Virtual Array manager in
C++, the next section describes its use with C, however when
using C much of the functionality and transparency of the C++
version is lost.

    The functions described can only be used if the file
xarray.h is included in any files using extended arrays.

1.1.    Declaring arrays

    Arrays can be declared and used with any type or with a
class or structure.

1.1.1.    Simple arrays

    In the header file xarray.h various variable types
have extended definitions. These types are as follows.

    float, double, int, char

    To create an extended array an 'x' must be prefixed
to the variable type and the size of the is in parenthesis
after the variable name.

    i.e.        x<type> <variable>(size of array);

    Therefore to declare an array of float called
'costing' with 87654 elements the declaration would be as
follows.

        xfloat costing(87654);

    This variable can then be used like any other array.

    e.g.        costing[5674]=5.334;

    To pass this array to a function you must retain the
prefix in the function deflation.

    e.g.        sum (xfloat values)
            {
                .
                .
            }

            main (..
            .
            .
            sum (costing);
            .
            .
            }

1.1.2.    Two dimensional arrays

    Two dimensional arrays are declared in a similar way
to one dimensional arrays except 'xx' is prefixed to the
variable type.

    i.e.        xx<type> <variable>(number of rows, number
of columns)

    Therefore to declare a 240x200 array of char called
image the declaration would be as follows.

        xxchar image(240,200);

    Again you can make assignments as usual

    e.g.        image[100][100]=10;

    A two dimensional array can be passed to functions as
for one dimensional arrays.

    Indexing a row of a two dimensional array return a
one dimensional array of the same type.

    e.g.        xchar imagerow;
            imagerow = image[10];    /* i.e. image[10]
returns a variable of type xchar */

1.1.3.    Arrays of undefined types

    Arrays of any type can be used but if the type is not
from the list of defined types you need to insert the
following after you include the file xarray.h.

        XARRAY (<type>)

            to declare one dimensional arrays and

        XXARRAY (<type>)

            to declare two dimensional array.

    Therefore to define extended arrays of type 'long'
the following would be inserted into the code.

        XARRAY (long)
        XXARRAY (long)

    Note: If two dimensional arrays of a type are
required then one dimensional arrays of that type need to
be declared first.

    If you want to declare a type with a space in it such
as 'unsigned char' then a typedef needs to be used first.

    e.g.        typedef byte unsigned char
            XARRAY (byte)
            XXARRAY (byte)

    These definitions allow the defined types to used as
described previously (i.e. xlong, xxlong, xbyte, xxbyte)


1.1.4.    Arrays of structures or classes

    An extended array of a struct or a class can be
defined in a similar way to other data types. An example
is given in the program ARRAYS.CPP with a struct called
'anything'.

First the struct is defined.

    struct anything
    {
        .
        .
    }

    Then the extended arrays are defined as before

    XARRAY (anything);
    XXARRAY (anything);

    This definition creates two types - xanything and
xxanything.

1.2.    Using templates

The description of declaring virtual arrays in section 1.1.
uses the file xarray.h that includes a couple of macros. Some
C++ compilers have class templates. Enclosed is another header
file (txarray.h) that uses templates instead of macros.

Templates are neated and easier to use. When using the template
header, to declare a virtual array of any type or class use the
following format (I have used square brackets here to indicate
the parameter because templates use the <> symbols):

xarray<[type or class]> [variable]([size])

to declare a two dimensional array use the following:

xxarray<[type or class]> [variable](no. of rows, no. of cols)

Use the xarray declaration in your subroutines as well.

e.g.

    sum (xarray<float> values)

1.3.    Controlling your memory

    With the C++ interface the use of extended memory or disk
caching is controlled in the background without intervention
from the program.

    The virtual array manager splits the virtual memory into a
number of blocks. A number of these blocks are held in
conventional memory and the rest is held in extended memory or
on disk. If a block is accessed which is not currently in
conventional memory then a block is swapped out and the
required block is swapped in. The size of a block, the number
of blocks and the number of blocks that can be held in
conventional memory all affect the speed and the size of
conventional memory required by your program.

1.3.1.    Declaring size and number of blocks

    When the first Extended array is declared the
extended memory manger is initialised or a swap file is
created. All the virtual memory that is needed by your
program must be defined before the first array is
declared. The amount of memory allocated is affected by
the following global variables.

    __block_size (default 32768)    -    Size of a block
    __blocks (default 16)    -    Total number of
blocks declared.
    __mem_buf (default 4)    -    Number of blocks
held in conventional
memory.



The amount of conventional memory used is
      block size*  mem buf.

The amount of virtual memory required is
      block size*(  blocks-  mem buf)

    By increasing blocks and mem_buf and decreasing
block_size higher performance may be achieved (And more
memory used).



1.4.    Other virtual array manager parameters

There is a whole host of parameters that can be adjusted. Most
of them have comments along with their declaration. You do not
need to adjust them to get the Virtual array manager to work.
But once you have got your program working you may want to play
with them. The variables that hold these parameters are found
in virimg.h. To change the default values of these variables
they must be set before any virtual arrays are declared. I.e.
the first thing in 'main()'.

___saftey
If the virtual array manager is using extended memory and
your program crashes then the array manager with not be
able to release its access to the extended memory. For
this reason when the array manager uses extended memory it
puts the extended memory handle in a file. When it
finishes with its extended memory it deletes that file.
However, if the program crashes then the next time you run
a program with the array manager it will find the file and
release the memory that had been taken. If you do not want
to use this safety utility include the following line in
your program : ___saftey=0;.
*___safe_filename
This is the name of the file where the extended memory
file handle is kept. The default is "c:\!saftey!.xms". If
you want to change the name of the file include the
following line in your program :
___safe_filename="<filename>";.

___msgs
By default various messages are output to the screen when
the array manager is set up and closed down. If you want
to stop these messages include the following line:
___msgs=0;.

*___xfilename
When the virtual array manager uses the disk to swap
memory the virtual arrays are put in the file
"virarray.swp" which is deleted when the virtual array
manager is closed down. To change the name of the swap
files include the following line:
___xfilename="<filename>";.
___xfile_buf
Your machine has a file buffer which is used when writing
to and reading files. The size of this buffer for the
array manager is 32768. To change the size of the buffer
include the following line: ___xfile_buf=<buffer size>;.

___xarrays
By default the array manager first tries to cache to
extended memory. If there is not enough extended memory
available it automatically tries to cache to disk. If you
want to force the array manager to cache to disk include
the following line: ___xarrays=XDISK;. There is no option
to cache part of the arrays to disk and the rest to
extended memory because much higher performance would be
achieved by using disk caching combined with a file disk
cache such as comes with windows, msdos5 and drdos. But if
you do chose this option allocate a lot of memory to the
file disk cache.
___auto_disk
If you do not want the array manager to try and cache to
disk if it fails to allocated extended memory then include
the following line: ___auto_disk=0;.



2.    Using the virtual array manager with C
If you do not have a C++ compiler the Array manager can still
be used but with slight modification to the code. The
underlying code of the virtual array manager is a C program
that creates a large number of virtual arrays of type void and
of a fixed size. The C++ code simulates the variable types and
array sizes by extensive use of operator overloading.

Because C does not have operator overloading available similar
routines need to be created by macros. A new macro is needs to
be created for each extended array declared. Not only this but
the program must directly interact with the array manager. The
rest of this section explains the routines available to the
programmer. Most of the variables described in section 1.3
still apply.

The virtual array routines create a specified number of arrays
in memory. If more arrays are used than can fit into the memory
allocated then the array that has not been accessed for the
longest is cached to disk or extended memory (The rest of this
section assumes you are caching to disk). All the virtual
arrays must be of the same size. The array memory must be
initialised as follows.

initialise virtual arrays (number of arrays to be stored in
conventional memory, size of each array, maximum number of
virtual arrays to be used, whether to cache to disk or
extended memory (XXMS or XDISK))

This routine initialises a specified number of virtual arrays.
Each array have identified by a number. The amount of memory
used depends on the size of the arrays and the number of array
buffers in conventional memory and virtual memory. The actual
arrays do not take up disk space or extended memory until they
are accessed.

To access an array the following procedure will return a
pointer to the array.

    void *img(<array number>)

(In the restricted version <array number> is between 0 and 15)

If the array is not in memory it will be created if it does not
exist or it will be loaded off disk if it does exist . This
pointer can be passed to a subroutine just like a normal array.

    e.g.
        process (char picture[50][50])
        {
            .
            .
            Some code processing the array normally
            .
            .
        }

        main ()
        {
            .
            .
            process (img(5));
            .
            .
        }

When you pass a virtual array to a subroutine it becomes
active. When the subroutine finishes the array becomes passive.
More than one array can be passed to a subroutine but there is
a limit to the number of arrays active arrays allowed at one
time. This limit is the first parameter in the
initialise_virtual_arrays routine.

Also note that if an array has been cached then when it is
restored it may be restored to a different memory location.
I.e. do not hold permanent links to and array after it has been
made passive.

2.1.    Simulating larger arrays

    Another useful procedure is

    void *large array (<array number>,<array index>)

This routine returns a pointer to the ith element of the <array
number>.

A macro can then be defined to access an element of a certain
array where an array number is assigned to an array name.


#define <array name>(x) *((<data type<)large array(<array
number>,x))

---------------------------------------------------------------

Hugh Arrays

    One hugh array can be created from a series of virtual
images to almost any size. An example is given in the file
HUGHARR.C