Power-Programmierung/CD1.mdf/pascal/library/dos/heap_7/mrkrls.pas

Pascal/Delphi Source File | 1993-02-19 | 14.2 KB | 403 lines

{

like many others, I am sure, I  had the need to port a real-mode
program to protected mode, and all memory management was done via
MARK-RELEASE. As we all know,  this is not supported in protected
mode, and  re-writting a large  program to use  getmem-freemem is
practically impossible.  Therefore, I came up  with this piece of
code to simulate the operation  of mark-release. I have tested it
on my program  , and it seems to  work. However, I had to  make a
lot  of assumptions  as to  how the  new memory management works,
without any real documentation. Therefore, I dont trust this code
much. The purpose of uploading it here is threefold:

1. I would like to get comments on the code and my rationale from
people that really understand protected mode

2. I would  like the code to be tested  with more programs than I
could ever write!

3. I want others with the same problem to benefit from it.

Therefore, please feel free to  use this code with your programs,
no royalties asked. BUT  there is one thing I DO ask  , and it is
this: PLEASE  let me know if  the code works for  you or not, and
any  problems  that  you  may   encounter  with  it.  This  is  a
developer's   forum,  and   we   would   all  benefit   from  any
comments-problems  you  would  report.  I  am  certain there is a
million  or programs  out there  just DYING  to port to protected
mode,  that are  facing the  same problem.  BE SOCIAL. Share your
opinions with others.

The preferred method of contact is E-MAIL, or messages posted in
BPASCAL, section 6, protected mode.

Mike Cariotoglou,

CIS ID : 100012,1767


DOCUMENTATION (sort of..)
-------------------------

The protected mode memory manager works somewhat like this :

The system maintains two variables, HEAPLIMIT and HEAPBLOCK
Any request  for memory >=HEAPLIMIT gets  a "normal" allocation ,
ie  a selector-offset  pointer that  is managed  by the  RTM. Any
request smaller than this, is handled as follows:
The  BP  heap  manager  allocates  a  "Large"  memory block, size
HEAPBLOCK,  and then  "carves" or  sub-allocates this  to smaller
chunks  that  it  passes  to  the  application.  Each large block
maintains its  own freelist of deallocated  pointers, and manages
free "small" blocks very much like T6 memory manager.
(By  large  blocks  I  mean  blocks  >=HEAPLIMIT,  that  are  not
sub-allocated. By  the way, a  program can distinguish  between a
large  and  small  block  by  the  fact  that  the  offset of the
corresponding  pointer is  0 for  a large  block, and  >=12 for a
small block).

It follows  from this, that  each large block  behaves like a  T6
heap, along with the 8-byte granularity used by the T6 manager to
reserve space for the free  list entries. However, I believe that
here  the granularity  is 4  bytes, not  8, since  only WORDS are
needed to manipulate a heap of  <=64k. This is not important, and
would  affect only  programs that  assumed and  used this  8-byte
granularity.  I am  sure, however,  that such  programs would  be
bound to do pointer arithmetic  also, meaning they would break in
many more places than this.

Each large block has an overhead of 12 bytes, which are used to
maintain the block and link to other large blocks.

The manual says that the default for heaplimit and heapblock are
1024  and   8192,  and  suggests  that   heapblock  is  at  least
4xHeaplimit.  I  dont  really  see  any  reason  for  that,  so I
experimented with the following values :

HEAPBLOCK=65532 ( I believe this has to be a multiple of 4)
HEAPLIMIT=heapblock-12; (the overhead)

This gives you  a large block that is similar  to a 64k T6 heap,
and its  performance speedwise is  very acceptable, so  these are
the values I used. It is  important that heaplimit is as large as
possible, since any programs using  the technique I will describe
CANNOT allocate a  block of more than HEAPLIMIT  bytes. The above
values give  a maximum block  of 65520, which  happens to be  the
limit in the old T6 heap manager as well.

Each "small heap" of 64k belongs to a circular linked list. The
system variable HEAPLIST holds a selector  that can be used as an
entry point to the circular list. This selector does NOT point to
the  head of  the list   (the first  large block  allocated), but
rather at the most recently used  "small" heap. I believe this is
done  to improve  performance, since  the heap  manager traverses
this list when  asked for memory, and this  ensures that the most
recently used block will be used.

Now, the basic idea behind my MARK-RELEASE implemenation:

Each time MARK is called,  the current HEAPLIST variable is saved
in a stack, and  then it set to 0. This has  the effect of taking
any  memory  allocated  before  the  MARK  "off-line",  making it
unaccesible to  the heap manager. The  first allocation after the
MARK will create a new 64k  "small" heap, that will be managed by
the heap manager normally. When this  is exausted, a new one will
be allocated, and so on. When RELEASE is called it will :

a. de-allocate all 64k heaps that belong to the current HEAPLIST
b. de-allocate all heaplists created up to but not including the
   corresponding MARK
c. restore the heaplist saved when MARK was called.

This has the  effect of restoring the memory  state exactly as it
was when the MARK was called.  Note that step (b) above allows us
to  use MARK-RELEASE  in nested  form, and  RELEASE to  any point
MARKED. This often happens in real programs. An example:

mark(p1);
...
...
...
mark(p2);
...
...
release(p1);

This would work with the old  heap manager, and it will also work
here, being the equivalent of :

mark(p1);
...
...
...
mark(p2);
...
...
release(p2);
release(p1);

(actually,  this  system  works  BETTER  than  the old, since any
memory in  the freelist before the  MARK is not lost,  as was the
case  with the  old manager,  which always  cleared the free list
upon a RELEASE)

This is the  basic idea, the comments in  the code should clarify
more.

Limitations and DONT'S
----------------------

1.  All   allocations  requested  by  the   application  MUST  be
less than HEAPLIMIT, so that they get sub-allocated. If any block
>= heaplimit is allocated, this memory will NOT belong to a small
heap, therefore will NOT be de-allocated by the release, and will
be  PERMANENTLY lost.  My GETMEM  dummy ensures  this ,by halting
the program in such a case. However, I cannot trap calls to NEW
like that, and therefore the potential for loss of memory exists.
I do not expect many programs to be bothered by it, since the
old real mode heap manager would  NOT allow an allocation of more
than 65520 anyway, so existing  programs should NEVER create this
situation. Still,I thought best to mention it.

2.  Do not  expect multiple  allocations to  give you consecutive
memory space. This was also the case with T6, therefore, again, a
program should not rely on  this. (it would involve pointer math,
anyway, which is a DONT in protected mode)

3. It is best that nested MARK-RELEASE calls are matched. Failing
this, it  is acceptable (as shown  above) to do a  RELEASE with a
pointer  that  was  MARKED  before  the  last  mark (any level of
nesting).  What should  NOT be  done, is  to leave dangling MARKS
around, that are not cleared by a RELEASE with this or a previous
MARK. This would lead the program to die soon, since the internal
stack of UHEAP would overflow. In other words:

mark(p1)
...
release(p1)

is best

OR

mark(p1)
...
mark(p2)
...
mark(p3)
...
release(p1)

is OK

BUT

mark(p1)
...
mark(p2)
...
release(p2);

(missing RELEASE(p1))

is NOT allowed.

Unfortunately, this  was allowed in  the old heap  manager, since
MARK did not do anything significant, (what it actually was, was
mark(p) :: p:=heapptr;
Therefore, a situation like this MAY exist in old programs.
you will find out by the fact  that the program will crash with a
runtime error if such a thing exists. A good way to test for this
would be  to take the  program through all  its steps, and  right
before exiting check the variable MARKN in uheap. It should be 0.
if not, there is a dangling MARK somewhere.

4. Freemem. Since programs based on mark-release will, sometimes,
   use  freemem  also,  I  have  included  an implementation that
   should do the job. It  should correctly handle the ill-behaved
   case where a program frees a pointer that was allocated before
   the MARK. I  cannot do the same for  DISPOSE, however (for the
   same reasons as NEW) therefore, be warned. It would be a very
   sick program that would do things like that, anyway. Note that
   DISPOSING a  variable that was alloocated  AFTER the innermost
   MARK, is OK. Note also that the implementation of FREEMEM is not
   very efficient, so avoid it if possible

5. Each  MARK that you call,  potentially loses you up  to 64k of
memory, since  it takes the  current heap block  off-line, and is
not accesible  to the heap manager.  This situation is temporary,
and gets fixed  when RELEASE is called. It  means , whoever, that
if  many  nested  MARKS  are  outstanding,  the  amount of memory
available  is less  than you   would expect.  Since PM  gives you
access to unlimited amounts of  memory (especially if you use the
swap  file  technique,  see  messages  section  6 on DPMI virtual
memory), this should not be a problem,  and in any case it is the
price to pay for getiing a MARK-RELEASE that works.

6. the old MARK gave you a real pointer. My MARK will actually
give you an INDEX to a stack. Any programs that actually USED the
value returned  by MARK (they  shouldn't, but...) will  DIE right
away with GP errors.

7.  Once set,  HEAPBLOCK and  HEAPLIMIT should  NEVER be  changed
during the life time of the program. Reasons are obvious


Using globalxxxx functions
--------------------------

Doing memory  management with the WINAPI  and GLOBALxxx functions
is fine, and will not interfere with this scheme


Ok, here is the code for all that. Please TEST it and let me know
if any problems arise..

Good luck!

Mike Cariotoglou,
100012,1767

19-2-93

--------------------------------------------------------------------------
}

unit uheap;

interface

procedure mark(var p);
procedure release(p:pointer);
procedure getmem(var p; bytes:word);
procedure Freemem(p:pointer; bytes:word);

implementation

type
     dword=record {for typecasting pointers}
            o:word;
            s:word
           end;

     theapheader=record {12 bytes, header of large block}
                  hsSignature:word;
                  hsReserved:word;
                  hsFreeList:pointer;
                  hsMemFree:word;
                  hsNextHeap:word;
                 end;

     pheapheader=^theapheader;

const HpOvrHead=sizeof(theapheader); {suballocator overhead}
      MaxMark=64;    {allows for 64 outstanding MARKS
                     which should be enough}

      markn:integer=0;  {stack pointer into array of HEAPLIST values}

var  markl:array[1..maxmark] of word;
{keeps the HEAPLIST values at the time of the MARK}

procedure mark(var p);
 begin
  if markn=maxmark then runerror(203);  {exceeded the limit for outstanding marks}
  inc(markn);
  markl[markn]:=heaplist;
  heaplist:=0; {remove the current heaplist from the heap manager}
  longint(p):=markn;   {uses the pointer passed to store the stack top}
 end;

{this routine will de-allocate a whole HEAPLIST, given an entry to any
point in it}
procedure killList(base:word);
 var p:word;
     pp:pheapheader;
 begin
  if base=0 then exit;
  p:=base;
  repeat
   pp:=ptr(p,0);
   p:=pp^.HsNextHeap;
   system.freemem(pp,heapBlock); {deallocates a LARGE block, since offset is 0}
  until p=base;   {circular list proccessing terminates when entry point is reached again}
 end;

procedure release(p:pointer);
 var i,j:integer;
 begin
  i:=dword(p).o;
  if i<=0 then runerror(203); {illegal stack index passed to release}
  killList(heaplist);         {kill the current heaplist}
  for j:=markn downto i+1 do killlist(markL[j]); {this is in case the release NOT for the last MARK}
  heaplist:=markl[i]; {restore the heaplist at the time of the MARK}
  markn:=i-1;         {pop values from the stack}
 end;

{this routine is used for FREEMEM support. It will try to find
which "small" heap a pointer belongs to, by searching a heaplist for
a selector that matches the pointer's. If you plan to use FREEMEM
a lot, this routine should be coded in assembly for performance}

function FindOnList(pp:pointer; base:word ):boolean;
 var p,bs:word;
 begin
  findonlist:=false;
  if base=0 then exit;
  bs:=dword(pp).s;    {selector of pointer}
  p:=base;            {used for list traversal}
  repeat
   if p=bs then
    begin
     findonlist:=true; {yes, this pointer was allocated from this small heap}
     exit
    end;
   p:=pheapheader(ptr(p,0))^.hsNextheap; {link to next heap}
  until p=base;
 end;

procedure getmem(var p; bytes:word);
 begin
  if bytes>=heaplimit then runerror(203); {kill programs that allocate blocks >=65520}
  system.getmem(pointer(p),bytes)
 end;

{this is the freemem implementation. it can handle pointers allocated
before the last MARK, but is not very efficient, so use with care}

procedure Freemem(p:pointer; bytes:word);
 var i:integer;
     tmp:word;
 begin
{if this is a large block (offset=0) or it belongs to the current HEAPLIST,
just do normal freemem}
  if (dword(p).o=0) or findonlist(p,heaplist) then
   begin
    system.freemem(p,bytes);
    exit
   end;
{search the stacked heaplists to find which one it belongs to}
  for i:=markn downto 1 do if FindOnList(p,markL[i]) then
   begin
{if found, then :
 restore the heaplist.
 do the freemem
 put back the current heaplist.
 this has the effect of returning the memory to the correct heaplist,
 so it is not lost}
    tmp:=heaplist;
    heaplist:=markl[i];
    system.freemem(p,bytes);
    markL[i]:=heaplist;
    heaplist:=tmp;
    exit;
   end;
  runerror(203); {should never get here}
 end;

begin
 heapblock:=65532; {I saw somewhere that this must be a multiple of 4}
 heaplimit:=heapblock-HpOvrHead; {max block allowed}
end.