Difference between revisions of "CPGuide - Memory Management"

From EDM2
Jump to: navigation, search
(Created page with "This chapter describes the memory management features and functions of OS/2. The key features of OS/2 memory management are paged virtual memory and a 32-bit linear (flat) add...")
(No difference)

Revision as of 02:32, 27 March 2020

This chapter describes the memory management features and functions of OS/2. The key features of OS/2 memory management are paged virtual memory and a 32-bit linear (flat) address space that is mapped through page tables to physical memory. An OS/2 application can allocate memory for its own use or to be shared with other applications.

The following topics are related to the information in this chapter:

  • Exception handling
  • Program execution and control
  • Semaphores
  • Queues

About Memory Management

OS/2 offers developers a 32-bit, linear (flat) memory address space. OS/2 uses a paged memory structure. OS/2 allocates, protects, and manipulates memory in terms of pages.

Process Address Space

The OS/2 memory allocation functions return a 32-bit pointer to the allocated memory object. While a 32-bit pointer is sufficient to address the entire 4 gigabyte global address space, applications can access only the first 512MB of linear memory, called the process address space. Of this 512MB process address space, a minimum of 64MB is reserved for shared memory regions, leaving 448MB. Of this 448MB, some will be used by the application itself and a small amount will be taken by operating system overhead. The remainder is available for allocation. The amount of memory that can actually be committed and used is, of course, determined by the physical memory and hard disk space available on the machine.

Keep in mind that the amount of memory that can be committed for actual use is limited by the amount of physical memory and free hard disk space that is available on the computer on which the application is executing.

Memory Objects

Applications allocate and manipulate memory in terms of memory objects. A memory object consists of one or more pages of memory. An OS/2 application can allocate any number of memory objects, within the following limits:

  • the physical memory in the system
  • the free hard disk space on the hard disk containing the swap file
  • the 512MB process address space limit (see Process Address Space).

When requesting memory, the size of the memory object is rounded up to the next higher multiple of 4KB. An application can suballocate a memory object into memory blocks whose size can range from 1 byte to the size of the memory object.

Memory objects have the following characteristics:

  • They are not relocatable.
  • They are allocated in units of 4KB. One 4KB unit is called a page.
  • They can be larger than 64KB in size.

Memory Pages

OS/2 allocates and commits memory objects in pages. A memory page is a 4KB (4096 bytes) piece of memory. Memory access protection is also done on a page-basis, rather than the segment-based protection used in previous versions of OS/2.

A page range is a linearly contiguous group of pages within a memory object. A page range can be any of the following:

  • The entire memory object
  • Part of the memory object
  • A single page within a memory object

If an application requests 512 bytes of memory, it will receive a 32-bit pointer to a 4KB page. All 4096 bytes are available to the application, even though the request specified only 512 bytes. If an application requests 62000 bytes, it will receive a pointer to a 65536-byte (64KB, 16-page) object. Again, all 65536 bytes are available for use.

Each page in the virtual address space of the process is either free (unallocated), private (available only to the process that allocated it), or shared (memory that is shared between processes).

Each page within a memory object can be in one of two states, either uncommitted (that is, the linear address range has been reserved, but is not yet backed by physical storage) or committed (physical storage has been allotted for the logical address range).

Access to a committed page is controlled by the page's access protection attribute. These protection attributes are read access, write access, execute access (on the 80386, this is the same as read access), and guard page access.

An uncommitted page is not accessible.

Memory Overcommitment and Swapping

Memory overcommitment occurs when applications allocate and commit more memory than is actually available in the computer. OS/2 handles memory overcommitment by copying memory to the system swap file, SWAPPER.DAT, on the hard disk then reusing the memory for another allocation. OS/2 copies as many pages of memory as are necessary to make room for the new allocation. The swapped memory can be retrieved the next time it is accessed; at that time, some other memory might be written to the swap file.

OS/2 selects the memory to swap based on when it was last used. The page that is least-recently-used, that is, the page that has gone the longest since its last access, is the page chosen to swap to disk.

Swapping is transparent to an application, although excessive swapping can cause an application to run slowly.

Through swapping, OS/2 enables applications to allocate more memory than actually exists in the computer, bounded only by the amount of free space on the hard disk that contains the swap file.

User Configuration of Memory Swapping

Although an application cannot control swapping, the user can specify whether the system can swap memory by including the MEMMAN command in the CONFIG.SYS file.

If the MEMMAN command specifies SWAP, OS/2 writes selected memory pages to the SWAPPER.DAT file whenever insufficient physical memory exists to satisfy an allocation request. This is the default choice. If the MEMMAN command specifies NOSWAP, OS/2 does not swap memory.

Be aware that disabling swapping will severely limit the number of applications that the user will be able to run concurrently; if there is not enough physical memory present, OS/2 might not even boot.

The exact amount of memory available to an application depends on the amount of physical memory in the machine and the amount of free disk space on the partition that contains the SWAPPER.DAT file. The location of the SWAPPER.DAT file can be specified by including the SWAPPATH command in the CONFIG.SYS file. With the SWAPPATH command, one can specify the location of the SWAPPER.DAT file, the amount of free disk space to reserve in the partition which will contain SWAPPER.DAT, and the initial size of SWAPPER.DAT. OS/2 adjusts the size of SWAPPER.DAT as necessary, leaving other files on the drive and the requested free space untouched.

Memory Allocation and Commitment

When an application asks OS/2 to allocate memory, a linear address range is reserved. The range is not backed by physical memory until the memory is committed. Commitment assigns physical memory to the linear address range.

A memory object that is allocated, but not committed is called a sparse memory object. A sparse memory object must be committed before it can be used. An attempt to read from or write to uncommitted memory will cause an access violation.

An application can ask OS/2 to commit the memory at the same time it is allocated, thus making it immediately usable, or the memory can be committed at a later time. If the application commits the memory at the same time the memory is allocated, the entire memory object is committed. If the application commits the memory at a later time, it can commit the entire sparse memory object or only commit a portion of it.

When multiple pages are committed at the same time (a page range), the pages will have sequential linear addresses.

Managing Memory Allocation and Commitment

The recommended way to manage memory is to make a large memory allocation early in program execution, then to commit or suballocate memory as the need occurs.

The initial allocation should be for as much space as you expect to use during program execution. Allocation without commitment does not actually use any physical memory, so there is no waste involved in allocating several megabytes.

After the memory object is allocated, the application uses one of two ways to manage the memory object:

  • Commit and decommit the memory as it is required
  • Set up the memory object as a heap and suballocate memory from the heap.

Committing and decommitting memory gives the application more control over the process, but the application will have to keep track of which pages are committed and which pages are not. When suballocating memory from a heap, the application can have OS/2 track commitment and decommitment of physical memory pages, so the application does not have to. If you want DosSubAllocMem to manage the commitment of the pages spanned by the heap, all of the pages spanned by the memory object must be uncommitted initially.

Remember, no matter how much memory is originally allocated, the amount that an application will ultimately be able to commit and use is limited by the amount of physical memory and free disk space available on the machine.

Applications are not limited to a single large allocation of memory-other memory allocations can be made as necessary during execution-but large allocations and small commitments or suballocations are the most efficient way to manage memory.

Memory Resizing and Reallocation

In earlier versions of OS/2, an application could increase or decrease the size of an allocated memory segment by reallocating the segment.

Memory objects cannot be resized. Instead, an application should allocate a sparse memory object of whatever size might be necessary, then commit or decommit portions of the object.

If the amount of memory required cannot be determined at the time the memory is allocated, the application should allocate a sparse memory object large enough to meet the largest memory requirement. The application can then change the amount of committed memory as necessary.

For example, if you anticipate your application will use around 512KB of memory for most purposes, but might use 5MB under certain circumstances, you might take the following steps:

  • During program initialization, use DosAllocMem to allocate 5MB.
  • Commit the first 512KB (or some part of it) using DosSetMem.
  • Proceed with normal processing.
  • If extra memory is required occasionally, commit it and decommit it using DosSetMem.
  • When the situation arises that the application requires the full 5MB, commit it at that time, using DosSetMem, then decommit it after you are finished with it, also using DosSetMem.
  • When the application is finished with the memory, use DosFreeMem to release the memory back to the system.

Memory Protection

When an application allocates a memory object, it can specify the type of access to allow to the object. Memory access protection provides a program with control over the type of access that its threads have to a page of memory.

Access protection can only be defined for committed pages of memory and is initially set at the time the memory is committed. Different pages within the same memory object can have different access attributes and access attributes can be changed on a page-by-page basis at any time.

An application can request any combination of the following access protection attributes:

Memory Access Protection Attributes
│Access              │Defined Constant    │Description         │
│Read Access         │PAG_READ            │The object can be   │
│                    │                    │read from, but not  │
│                    │                    │written to.         │
│Write Access        │PAG_WRITE           │The object can be   │
│                    │                    │written to.  On the │
│                    │                    │80386               │
│                    │                    │microprocessor,     │
│                    │                    │write access implies│
│                    │                    │both read and       │
│                    │                    │execute access.     │
│Execute Access      │PAG_EXECUTE         │This is equivalent  │
│                    │                    │to read access on   │
│                    │                    │the 80386.          │
│Guard Page Access   │PAG_GUARD           │Causes a            │
│                    │                    │guard-page-entered  │
│                    │                    │exception to be     │
│                    │                    │raised in a process │
│                    │                    │that attempts to    │
│                    │                    │access the memory.  │
│                    │                    │This exception can  │
│                    │                    │be ignored or       │
│                    │                    │handled by the      │
│                    │                    │application's       │
│                    │                    │exception handler,  │
│                    │                    │if one is           │
│                    │                    │registered.         │

The guard page attribute is intended to provide automatic stack growth and stack limit checking. An application can also use it in other data structures, such as arrays. For example, if an application is using an array of 4096 bytes (one page), the application can allocate and commit two pages, one with read and write access and one designated as a guard page. If the application tries to write past the end of the array a page guard exception will be generated.

Any reference-read, write, or execute-to a guard page causes an access violation (page fault) to be generated. This fault causes a Guard Page Entered exception to occur for the thread that referred to the guard page. The exception can be handled by the exception handler of the process, if one is registered. If the process does not have an exception handler registered, OS/2's default exception handler will handle the exception. The default action by the system exception handler is to convert the page from a guard page to a committed page, then try to mark the next page in memory as a guard page. If the system is not successful in marking the next page as a guard page, an Unable-To-Grow-Stack exception occurs. The thread is allowed to continue execution, but must be aware that it has at most 4KB of stack remaining.

Obtaining Information about a Page Range

DosQueryMem is used to obtain information about a range of pages within the virtual address space of the current process.

Each page in the virtual address space of a process is either free, private, or shared.

Each page within a memory object can be in one of two states, either committed or uncommitted.

A committed page has its access controlled by an access protection attribute. These protection attributes are read protection (PAG_READ), write protection (PAG_WRITE), execute protection (PAG_EXECUTE), and guard page protection (PAG_GUARD).

Protection Violations

OS/2 fully utilizes the memory protection capabilities of the 80386 microprocessor. OS/2 grants an application access to a memory object only if the object has been explicitly allocated by the application or made available for use by the application.

If an application attempts to access memory that it is not assigned, the system interrupts the application and executes the exception handling routine for protection violations. Protection violations can be handled by the application (in its own exception handling routines) or by OS/2. If the protection violation is handled by the operating system, the system exception handling routine determines the cause of the exception, displays an error message, and then terminates the application.

It is usually not possible for an application to recover from a protection violation. Therefore, programmers should ensure that all pointers are valid. Because protection violations commonly occur during application debugging, each message displayed for a protection violation includes the contents of the registers when the violation occurred. If the violation occurred as a result of passing an invalid pointer to a memory function, an error code is returned by the memory function.

In earlier versions of OS/2, protection violations could be used to find bugs when an application accessed memory that was not allocated to the application. This approach will no longer work because memory objects can be larger than the size requested by the application because the memory objects are allocated on 4KB page boundaries. For example, a pointer to the 513th byte of a 512 byte memory object is valid and does not cause a protection violation. This means that programmers cannot always rely on protection violations to spot memory addressing errors.

Memory Suballocation and Using Heaps

There are times when a process requires only small amounts of memory rather than an entire memory object. It would be wasteful to allocate an entire page of memory when only a few bytes are necessary, so a mechanism is provided for applications to allocate a large block of memory and then suballocate portions of the memory as necessary to fulfill small requests from an application. This is done by creating a heap.

A heap is a region of storage within a memory object from which an application can allocate blocks of memory. A memory block is a piece of memory within a heap. The size of the memory block is rounded up to the next higher multiple of 8 bytes.

Because OS/2 allocates a 4KB page for each memory allocation, using a heap to suballocate amounts of memory smaller than 4KB is more efficient than using DosAllocMem.

When an application creates a heap, it can have OS/2 track the committing and decommitting of memory within the heap. When the application commits and decommits memory itself, it has to keep track of the access state of each page as they are accessed.

Applications use DosSubSetMem to initialize a memory object for suballocation, then use DosSubAllocMem and DosSubFreeMem to allocate and free the memory.

Memory is still committed in pages when an application uses suballocation. If the application suballocates 512 bytes, 4096 bytes will be committed. Accessing the 513th byte will not cause a protection violation, but you could be accessing memory that was suballocated by another thread in the process.

Shared Memory

Shared memory is memory that two or more applications can read from and write to. Shared memory is prepared in such a way that any application can receive a pointer to the memory and access the data. Applications must explicitly request access to shared memory; the shared memory is protected from applications that are not granted access.

There are two kinds of shared memory: named and unnamed. For named shared memory, any application that knows the name of the shared memory can access it. For unnamed shared memory, a pointer to the shared memory must be passed from the process that created the shared memory to the process being given access. Access can be granted to any application; it is not necessary that the process being granted access be related (parent-child) to the application that created the shared memory.

Since memory sharing is done by sharing linear addresses, the linear address range of the shared memory object is reserved in all process address spaces.

There are two basic methods of managing shared memory:

  • In one method, two or more applications share the same memory at the same time. These applications read from and write to the memory object, usually controlling access to the memory by using a semaphore.
  • In the other method of managing shared memory, one application prepares data in memory, then passes that memory to another application for further processing. The first application releases the memory after passing it along, so that only one application accesses the memory at a time.

Thread Local Memory

Thread local memory is a 128-byte region of thread-specific memory that OS/2 assigns to a thread when the thread is created. Thread local memory for each thread is always at the same virtual address, but each thread's local memory region is backed by different physical memory, so the memory is specific to each thread. This memory region is divided into 32 DWORDS (4 bytes per DWORD), and is normally used to store a small number of pointers.

A thread can allocate thread local memory by calling DosAllocThreadLocalMemory. Thread local memory is freed by calling DosFreeThreadLocalMemory.

Note that thread local memory should be used sparingly. You should not use the entire 128 byte region.

Using Memory Management

This section describes how to use the OS/2 memory management functions and configuration commands to control the use of memory for OS/2 applications.

In the example code fragments that follow, error checking was left out to conserve space. Applications should always check the return code that the functions return. Control Program functions return an APIRET value. A return code of 0 indicates success. If a non-zero value is returned, an error occurred.

Allocating Private Memory

An application can allocate regions of storage within the virtual address space of the process. Such an allocated region is called a memory object.

DosAllocMem is used to allocate a memory object. You specify a variable to receive the pointer that will address the new object, the amount of memory needed, and the allocation attributes and access protection attributes of the new memory object. When choosing the size of the memory object to allocate, remember that the maximum size of the memory object is defined when it is allocated, and memory objects cannot be resized.

When applications call DosAllocMem, the operating system reserves a range of private pages large enough to fulfill the specified allocation request from the private virtual address space of the subject process.

DosAllocMem will reserve this linear space and return zero if the allocation was successful. If it was unsuccessful, DosAllocMem will return an error code. An application should always test the return value before attempting to use the memory.

The following code fragment requests an allocation of 512 bytes. Remember that 4096 bytes will actually be allocated for this request:

    #define  INCL_DOSMEMMGR   /* Memory Manager values */
    #include <os2.h>

    PBYTE   pb;
    APIRET  ulrc;

    ulrc = DosAllocMem((PVOID *) &pb,
                       fALLOC); /* pb receives the base address of the */
                                /* 4KB memory object                   */

    if (!ulrc) {           /* If the allocation was successful, ulrc == 0 */
        *pb = 3000;        /* Use the allocated memory                    */

In this example, DosAllocMem returns a 32-bit pointer to a 4096 byte committed memory object and allows the application to write to and read from the memory. This pointer is valid only for the 4096 bytes allocated by the system. An attempt to use the pointer outside the allocated memory will cause a protection violation.

Committing and Decommitting Page Ranges

If an application allocates a sparse memory object, no physical memory location is committed for the object. Memory in a sparse object must be committed before it can be used. DosSetMem is used to commit or decommit a range of previously allocated pages in a private or shared memory object. Applications can make specific address ranges within a memory object valid or invalid. Commitment and decommitment always take place in multiples of one or more pages.

Applications can also use DosSetMem to change the access protection attributes of a range of pages within a memory object.

The following code fragment requests allocation of 2MB of uncommitted memory and then commits 4096 bytes of the memory:

    #define  INCL_DOSMEMMGR   /* Memory Manager values */
    #include <os2.h>

    APIRET  ulrc;
    PBYTE   pb;

    /* Allocate 16KB object */
    ulrc = DosAllocMem((PVOID *) &pb,
                       PAG_READ |

    /* Commit 4KB           */
    ulrc = DosSetMem(pb,
                     PAG_COMMIT |

An application can also allocate a large committed object and then decommit portions of it as they are no longer needed. Decommitment, like commitment, is done on page boundaries; an application can decommit no less than a 4096 byte page.

The following code fragment allocates 16384 bytes of committed memory and then decommits the first 4096 bytes of the memory:

    #define  INCL_DOSMEMMGR   /* Memory Manager values */
    #include <os2.h>

    APIRET  ulrc;
    PBYTE   pb;

    ulrc = DosAllocMem((PVOID *) &pb,
                       fALLOC);    /* Allocate 16 K object */

    ulrc = DosSetMem(pb,
                     PAG_DECOMMIT);            /* Decommit 4KB         */

After memory is decommitted, an attempt to access the decommitted memory will cause a protection violation.

You cannot pass an argument that crosses multiple memory objects. The function will return an error.

Establishing Access Protection

When an OS/2 application commits a memory object, it specifies the types of access permitted for the memory object. This can be done at the same time the memory object is allocated, with DosAllocMem, or at a later time, using DosSetMem.

Any combination of read, write, execute, or guard-page access can be set, but at least read or write access must be specified when the memory object is committed; it is not possible to commit an object with no access protection attributes.

The application can also use DosSetMem to change the access permission of pages within a previously committed memory object. An application can permit read access to one page of the memory object and write access to the rest.

When using DosSetMem, all the pages in the range being changed must be either committed or decommitted.

The following code fragment commits a region of two pages within a previously allocated memory object, and sets read-only access rights for the region.

    #define  INCL_DOSMEMMGR   /* Memory Manager values */
    #include <os2.h>
    #include <stdio.h>

    PVOID   pBaseAddress;       /* Pointer to the range of pages to be changed */
    ULONG   ulRegionSize;       /* Size, in bytes, of the region to be changed */
    ULONG   ulAttributeFlags;   /* Flag describing the page range              */
    APIRET  ulrc;               /* Return code                                 */

    ulRegionSize = 8192;        /* Specify a two-page region                   */

    /* Obtain the base address  */
    ulrc = DosAllocMem((PVOID *) &pBaseAddress,

    ulAttributeFlags = PAG_COMMIT |

    ulrc = DosSetMem(pBaseAddress,

    if (ulrc != 0) {
        printf("DosSetMem error: return code = %ld", ulrc);

Querying Memory Object Information

DosQueryMem is used to determine the allocation state and access protection for a specified memory object. The application can query an entire memory object or a range of pages within an object.

The following code fragment uses DosQueryMem to ensure that memory is committed before the application attempts to use the memory:

    #define  INCL_DOSMEMMGR   /* Memory Manager values  */
    #include <os2.h>
    #define HF_STDOUT 1       /* Standard output handle */

    PBYTE   pb;         /* Base address of an allocated object */
    ULONG   ulSize,
    APIRET  ulrc;       /* Return Code                         */

    ulSize = 4096;

    ulrc = DosAllocMem((PVOID *)&pb,
                       PAG_COMMIT |

    ulrc = DosQueryMem(pb,
                       &ulFlags);     /* Queries first 4096 bytes */

    if (ulFlags & PAG_COMMIT) {     /* If memory is committed, use it */
        ulrc = DosWrite(HF_STDOUT,
                        "\r\n 4KB is committed.\r\n",

Freeing Memory

When memory object is no longer needed, the application uses the DosFreeMem function to release the memory.

If applications do not release memory, the operating system either swaps the memory to the hard disk (if swapping is enabled) or uses memory that could be used by other applications. If OS/2 swaps the memory to the hard disk, the SWAPPER.DAT swap file could become very large. Therefore, applications should release memory as soon as it is no longer needed. Any memory that is still allocated when the application ends is released by OS/2.

DosFreeMem frees a private or shared memory object from the virtual address space of the process. The released pages are returned to the system.

The following code fragment allocates 8192 bytes of committed memory and then releases the memory:

    #define  INCL_DOSMEMMGR   /* Memory Manager values */
    #include <os2.h>

    PBYTE   pb;
    APIRET  ulrc;

    ulrc = DosAllocMem((PVOID *) &pb,
                       fALLOC);    /* Allocate 8KB object */
    ulrc = DosFreeMem(pb);         /* Free the object     */

Using Suballocation and Heaps

Suballocating Memory

Increasing the Size of a Heap

Allocating a Block of Memory from a Heap

Freeing Memory Blocks

Ending the Use of the Heap

Allocating and Freeing Thread Local Memory

Using Shared Memory

Using Named Shared Memory

Using Unnamed Giveable Shared Memory

Using Unnamed Gettable Shared Memory