malloc(3C)


malloc, calloc, free, realloc, memalign, valloc, mallinfo -- checking memory allocator

Synopsis

   #include <stdlib.h> 
   

void *calloc(size_t nelem, size_t elsize); void free(void *ptr); void *malloc(size_t size); void *realloc(void *ptr, size_t size);

void *memalign(size_t alignment, size_t size); void *valloc(size_t size);

struct mallinfo mallinfo(void);

Description

calloc- checking memory allocator

free- checking memory allocator

realloc- checking memory allocator

memalign- checking memory allocator

valloc- checking memory allocator

mallinfo- checking memory allocator

These functions provide a simple general-purpose memory allocation package. An allocation request is made by a call to one of the functions that return void *; the pointer values returned are the start (lowest byte address) of disjoint objects, suitably aligned to serve as an array of one or more objects totalling the requested size. Except for calloc, the initial contents of these objects are indeterminate. The ptr argument (free and realloc) must be either a null pointer or a pointer value returned by a previous allocation request which has not been deallocated by an intervening call to free or realloc.

calloc allocates space for an array of nelem elements each of elsize bytes. All bytes of this space are initialized to zero.

free deallocates (makes available for further allocation) the space pointed to by ptr. If ptr is a null pointer, no action occurs.

malloc allocates space for an object of size bytes.

realloc changes the size of the object pointed to by ptr to size bytes. Note that the object may have been moved, in which case the old version of the object has been freed. The object's contents are unchanged up to the lesser of the old and new sizes. realloc(NULL,size) is a convenient equivalent to malloc(size).

memalign allocates space for an object of size bytes with the specified alignment, which must be a power of two. (The address returned is a multiple of alignment.)

valloc(size) is equivalent to memalign(sysconf(_SC_PAGESIZE),size).

mallinfo returns a structure that describes the current status of the allocation arena. The structure has at least the following members:

   	size_t arena;    /* total arena size in bytes */ 
   	size_t ordblks;  /* number of full-sized allocations */ 
   	size_t smblks;   /* number of small-sized allocations */ 
   	size_t hblks;    /* number of small-sized allocation containers */ 
   	size_t hblkhd;   /* overhead for these containers, in bytes */ 
   	size_t usmblks;  /* total bytes of in-use small-sized allocations */ 
   	size_t fsmblks;  /* total bytes of available small-sized allocations */ 
   	size_t uordblks; /* total bytes of in-use full-sized allocations */ 
   	size_t fordblks; /* total bytes of available full-sized allocations */ 

Errors

If an allocation request cannot be satisfied, a null pointer is returned and errno is set. When realloc returns a null pointer, the object pointed to by ptr remains intact. Note that zero-sized requests are not taken to be errors; an attempt will be made to return a pointer to a distinct zero-sized object.

Besides a lack of available space from the system, both realloc and memalign also can return null pointers due to invalid arguments.

Usage

This memory allocation package can be built to include valuable application checking support. This support comes in three parts: usage information/statistics, application diagnostic checking, and internal assertions. The primary effect of all three is to emit single-line messages to standard error or a specified file descriptor. Inclusion of the internal assertions is completely independent of the other two parts, but the diagnostic checking code requires the inclusion of the usage statistics code.

If the environment variable MALLOC_FILENO exists with a decimal integer value, messages are written to that file descriptor instead of standard error (or 2). This allows these messages to be separated from any of the application's error messages, for example.

The usage statistics and checking code, when included, are enabled only through separate environment variables, respectively, MALLOC_STATS and MALLOC_CHECKS. Both variables are expected to be decimal integers; a zero value disables, a positive value enables, and generally a larger value enables more detail.

Definitions

The allocation arena is a set of blocks, each of which consists of a data area (whose addresses are returned by the allocation functions) immediately preceded by a header which describes the block's size and status. In this implementation, a block's size is the distance in bytes from its header to the next block's header, and each size is normalized to be a multiple of a minimum internal alignment. This can easily result in blocks with spare bytes at the end of the data area, from an application's viewpoint.

Another cause of spare bytes is an enforced minimum block size. To implement a quick best-fit allocation policy, the initial portion of the data area in free blocks is used for internal data structures. The amount of space this requires determines the size of the smallest full-sized block. But, since a minimum block size can cause a disproportionally large amount of space to be wasted for small allocation requests, the implementation also provides for small-sized blocks with some restrictions, such as no coalescing with neighboring free blocks.

A limited view of the allocation arena's current status with respect to full- and small-sized blocks is available from mallinfo.

Assertions

When included, the internal assertions are always enabled. If an assertion fails, a line of the form
alloc assert failure: line num: expr
is emitted, where expr is the source test expression that failed which occurred on the source line num. These diagnostics should only happen after some allocation arena corruption, and are not generally very useful in tracking down any application error(s).

After an assertion failure is reported, the package usually continues processing as if the internal assertions had not been included (no corrective action takes place) but, if either (or both) of the statistics or checking code has been enabled, it will attempt to terminate the process through a call to abort.

Statistics

When MALLOC_STATS is set to a positive value, lines of the general form
function(args)@caller{fact}->return
are printed, where function is one of calloc, free, malloc, memalign, or realloc and the presence of some of the other parts varys. When one of these functions calls one of the others (such as calloc using malloc), the first function's information occurs as a label:
function1(args)@caller:function2(args)...
where some redundant args to function1 may not be present.

Numeric values are printed in hexadecimal (base 16) for pointers, while sizes are printed in decimal. Also, size arguments are presented in a before and after form:
incoming=>normalized-header
where normalized is the incoming value after including the block header overhead and rounding up to the minimum internal alignment.

``@caller'' represents the ``return address'' that will be jumped to when the function returns. With help from a debugger, for example, this address can be associated with a calling function. It is present when MALLOC_STATS is at least two.

Except for free, a ``->return'' pointer value will be shown.

Each ``{fact}'' gives some implementation internal detail--they are not intended to be generally useful--none of which will be present unless MALLOC_STATS is at least three. However, one typically useful exception is the true block size:
{max=size-header}
which displays the block's actual size less its header. The true block size is printed just after both the return value and an incoming ptr argument.

Checking

When MALLOC_CHECKS is set to a positive value, the package enables runtime checking of calls to these functions and the whole allocation arena. If a problem is found, a diagnostic line of the form
alloc error (ptr/owner @addr): problem description
is printed, where ptr is the address of the data area for the block (the same as would be returned by an allocation function), and ``/owner @addr'' is present only if MALLOC_CHECKS is at least three and ptr is a valid block with a recorded last-allocated owner (see ``@caller'' above).

There are three checking levels: basic-fill, safe-copy, and added-space, which correspond to MALLOC_CHECKS values of at least one, three, and five, respectively. Fundamental to this package's approach to checking is that it can be fully enabled without affecting the allocation behavior, at least until a diagnostic line is printed. This means that whatever presumed allocation arena corruption is occurring within an application with checking disabled should be reproduced after enabling checking. Only at the added-space checking level will the allocation behavior be changed, which is its purpose.

When checking is enabled, a call to mallinfo will also cause an examination of the whole allocation arena. For each of the checking levels, the next higher MALLOC_CHECKS value (2, 4, and 6, respectively) also cause a similar walk of the entire arena at the start of each of the main functions.

At the basic-fill checking level, shape checks on the ptr argument to free and realloc are performed -- it must be correctly aligned, fall somewhere within the allocation arena, and it should be an allocated (in-use) block. These checks produce the following problem descriptions for free and realloc:

   	free() of invalid block 
   	block multiply free()d 
   

realloc() of invalid block realloc() of free()d block realloc() of just free()d block

The distinction between the last two reflects the implementation's ``temporary holding slot'' behavior in which an attempt is made not to immediately modify a block's data area when deallocated. (This is primarily a hold-over from much older implementations which guaranteed this behavior!) An attempt to realloc a deallocated block will fail unless the block happens to be found in the holding slot.

Also at the basic-fill level, all available (deallocated) or freshly allocated data area bytes are filled with a ``noise'' value of ``0xCA''. When an available block is handled by the implementation (such as just before it is allocated), its data area is examined for bytes other than ``0xCA''. If one is found, a diagnostic is printed with the problem description free()d block was modified.

At the safe-copy level, the implementation also creates and manages a separate and parallel section of memory (via mmap) into which safe copies of the allocation arena's block headers are maintained along with a record of at least the block's last-allocation owner (see ``@caller'' and ``/owner @addr'' above) and usually the number of spare bytes present in the block's data area. At this level, when a block is handled, its header is checked against the corresponding copy -- which can result in a problem description of invalid block address if the block is not aligned, or not an allocated block is there is no corresponding copy, or block header was modified if the copy differs -- and any spare data area bytes (for blocks passed to free and realloc or other in-use blocks) are examined for bytes other than ``0xCA'' -- which can result in a problem description of ``spare block space was modified''.

At the added-space level, the implementation also increases each allocation request by an additional minimum alignment to force the presence of spare bytes in every in-use block. These spare bytes provide a little more protection from typical one-too-many misuse as well as better tracking of the corruption owner.

It is common to enable these diagnostic checks while running an application under the control of a symbolic debugger. With this in mind, the implementation calls a local (internal linkage) function, modified, just before the three ``... was modified'' diagnostics so that the modified contents can be examined with the debugger before the corrupted header or data area is reset. modified is called with three arguments, a void * that points to the start of the detected corruption, a size_t that specifies the number of bytes (remaining) to be checked including the one pointed to, and a const char * that points to a short string that denotes which kind of modification was found: spare, header, or free()d.

Two other local functions are useful when debugging: allocassert, which is called with the assertion expr string and num, and checkmsg, which is called with a block's data area pointer and the problem description string. Finally, two local integers, announcing and checking hold, respectively, the value of the MALLOC_STATS and MALLOC_CHECKS variables, so modifying these two can permit finer-grained control over these facilities. Check with your preferred symbolic debugger about how one refers to local identifiers.

Implementation approach

Note that the following assumes the reader has access to the source code. It freely uses terms and names that are likely to be meaningless out of this context.

The C library internal header inc/mallint.h describes the tunable constants and data structures for gen/malloc.c. The main data structure Tree declares just enough information for blocks in the top-down splay tree that is used for the free blocks. This structure is 20 bytes big, but the smallest full-sized block is 24 bytes because sizes are rounded up to multiples of eight. For small-blocks, only the block header and next pointers are assumed to be present.

Unlike earlier implementations, this approach does not force the same base alignment requirements on block headers as it does for the user pointers. A block header occurs in the HEADERSIZE bytes that come just before the aligned user pointer. Also, the basic alignment is eight, not four, so that three low-order bits are available for block status. Note that this means that the block size cannot be the number of data area bytes since the size must be a multiple of eight -- HEADERSIZE need not be a multiple of eight; this code uses a block size that is the distance from this block's header to the next.

The third status bit signifies whether a freed block is in the free tree or on a quick list. It is the quick lists that give this implementation its speed since by far the greatest number of calls are for smallish allocations. These lists cover the small-sized block pools without needing (much) special case code. Generally, the implementation tries to keep blocks at or below the MAXQUICKSIZE line on these lists; only when the implementation would otherwise have to request more space from the system does it flush the quick lists into the free tree.

This implementation also differs from older ones in that it can return memory to the system. This turns out to be quite important for long-lived processes, such as daemons, that have allocation peaks, but do not need nearly as much memory while waiting for requests.

References

brk(2), mmap(2).
30 January 1998
© 1998 The Santa Cruz Operation, Inc. All rights reserved.