NetNews Usenet Archive 1992 #31/NN_1992_31.iso/spool/comp/os/vms/19997

Text File | 1992-12-29 | 5.2 KB | 105 lines

Path: sparky!uunet!zaphod.mps.ohio-state.edu!saimiri.primate.wisc.edu!ames!agate!ucbvax!lrw.com!leichter
From: [e-mail] (Jerry Leichter)
Newsgroups: comp.os.vms
Subject: re: How do I use more memory?
Message-ID: <[e-mail]>
Date: 28 Dec 92 19:31:28 GMT
Sender: [e-mail]
Organization: The Internet
Lines: 94


    I've recently acquired a VAXstation with 80 Mbytes of physical memory,
    and I would like to use the memory almost exclusively to run really
    big (large arrays) scientific computational code. I'm more than
    willing  to sacrifice other management concerns.

    Unfortunately, my limited knowledge of VMS os management has prevented
    me from taking full advatage of the memory; whenever I run large
    executables the CPU spends about half its time in kernel mode, and the
    system also page faults an extraordinary amount considering there
    appears to be plenty of free  memory as indicated by $show mem.  I'd
    like my code to use all the memory available  without resorting to
    faulting.

    Currently I've set:
         MIN_WSMAX=119531
         MIN_WSINC=1000

(This one is going to have to be an "is it plugged in?" question, since what
you've done basically makes sense.)

The MIN_ values are not system parameters - they are inputs to AUTOGEN in
MODPARAMS.DAT.  Did you run AUTOGEN after adding these lines?  What value is
WSMAX set to?  (BTW, how did you come up with 119531?  Was it 80 meg less
SHOW MEMORY's "pages permanently in use by VMS", but any chance?  There's no
point in trying to calibrate things so finely - the number of pages that VMS
uses will vary with almost any parameter change, and in any case there's no
hope of a single process getting ALL the rest of memory; it's used for too
many different things.  The difference between 119531 and 120000 is of no
importance.)

I'm not sure why you are setting WSINC so high - the cost of growing your
memory in smaller chunks is pretty minor, since you only pay it once, at
image startup.  Making it this large may decrease the overall stability of
the system, as working sets for processes oscillate widely.  I'd leave this
one alone.

    and boosted up my user authorizations to:

       WSdef:          150
       WSquo:       163840
       WSextent:    163840

This will work, but I would recomment making WSQUOTA smaller - perhaps 40000
or so at most.  When the system has plenty of free memory (as it should for
what you are trying to do to make any sense) WSQUOTA will be pretty much
irrelevant, as a process that needs memory will be limited by WSEXTENT.  But
there are going to be times when you want to run a couple of big, but not
really huge processes, and it's nice to have the WSQUOTA control in effect.

       Pgflquo:     163840

You've clearly chosen the last three values from the total amount of physical
memory in the system.  That makes some sense for the working set values -
the number of physical pages is certainly an upper bound on how big a working
set can get, though there is even less point in trying to calculate this one
than in trying to calculate WSMAX:  It costs absolutely nothing if WSEXTENT is
larger than WSMAX, and setting it to something huge - like 500000 - means that
if you ever DO get more physical memory, you only have to change WSMAX, not
individual process limits.

As for PGFLQUOTA, though:  This places a limit on the maximum VIRTUAL size
(indirectly), and has nothing to do with the number of physical pages of
memory.  PGFLQUOTA can, in theory, be much smaller, much larger, or the same
as the amount of physical memory.  Since most process memory in a large
program is likely to be backed by the page file, it makes little sense for it
to be smaller than the working set limits, and on most systems it will be
quite a bit larger.

But here may be the crux of your problem:  It's fine to give a process a lot
of PGFLQUOTA, but do you have the page file space to back it up?  VMS is able
to overcommit the page file - with recent versions, you don't really need a
page of pagefile space for every page of real memory you are using - but there
are limits to this.  If your pagefile isn't large enough, your process's
ability to expand its virtual address space will be limited.  (It might also
be limited by the SYSGEN VIRTUALPAGECNT parameter.)

Here, however, we run into a lack of information.  "Too little working set"
is invisible to an application; it just gets more page faults.  "Too little
virtual memory" IS visible, as memory allocation attempts fail.  Most programs
do not even attempt to recover from running out of virtual memory.  However,
it is quite possible to write a program that allocates as much VM as it can,
then runs with that.  There's no way of knowing if your application is in that
catagory.

Finally, we don't have any idea how big your "large arrays" really are.  Sure,
120,000-odd pages of memory SOUNDS like a great deal, but it's amazing how
quickly something like a 3-d mesh of double-precision values can chew that up,
and more.  If the algorithms you are running aren't configured properly for a
virtual memory environment - (e.g., running through arrays along the "wrong"
dimension") you only have to exceed available physical memory by one page
before you begin thrashing.
                            -- Jerry