Programmer 7500

home *** CD-ROM | disk | FTP | other *** search

/ Programmer 7500 / MAX_PROGRAMMERS.iso / PROGRAMS / UTILS / MEMORIA / ABOUTEMS.ZIP / ABOUTEMS.TXT

Wrap

Text File | 1989-09-26 | 26.7 KB | 485 lines

This document was exported from WP 5.0 and some of the formatting codes were lost (especially footnotery). Please pardon the occasional anomalies especially out-of-place "1"s which were originally superscripted (footnote). MEMORY EXPANSION IN 80x86-BASED COMPUTERS UNDER MS-DOS prepared by John Wilson, Hyperdyne, Inc. Annandale, VA IN THE BEGINNING ... In the beginning, the was the 8080 microprocessor. The 8080 was, defendably, the "first" microprocessor.1 When the 8080 was invented, memory for computers was very expensive. The 8080 could directly address 64 thousand bytes of information, which was a very great deal in those days. Few systems (or more correctly, few system owners) could afford to actually place a full 64 K of memory in their systems. But things changed quickly. The Silicon Valley guys learned to make transistors smaller and better and it became much more economically feasible to talk about fully populated 64K machines. On the software front, things like spreadsheets and word processors were being invented. These programs were hungry for memory and the microcomputer user rapidly outgrew the 64K box. Before going on with the story, a little digression into the origins of the 64K limit. This limit was the direct result of the fact that the CPU chip had only 16 address lines. The address lines are a set of wires coming out of the CPU which allow the CPU to indicate what item of memory it wants to read or write. In most computers, the size of the "pieces" are bytes, or 8-bit characters. These 16 wires are called the Address Bus. The voltages on the 16 address lines are interpreted as a binary number (with the first pin representing the 1's place, the second pin representing the 2's place, the third pin representing the 4's place, etc.) the resultant number is the Address being addressed by the CPU. The number of distinct patterns of 16 things, each of which can have two values, is 2 raised to the 16th power, or 65536. This number is 64 times the quantity "1K" which is 1024. (Early on, programmers engineers decided to use the "K" as a unit of counting things like memory because it was a nice power of two and figured it was close enough to a "real" thousand that nobody would notice). The point of this story is that the original microprocessors could only address 64K because of the simple reason that they only had 16 address lines. Why didn't they just build one with more address lines? Well its not that simple. Those address lines had to have circuitry driving them, and other circuitry to actually generate the 16-bit addresses, and other circuitry to decode the new instructions that would use more than 16 bits, and so on, and so on. And this circuitry was made of transistors. And, unfortunately, the state of the art of chip manufacturing did not allow chips to be built with many more transistors. So, mankind waited for the silicon boys to learn how to fit more transistors on one raisin-sized chip. Meanwhile, back the ranch, The Intel Corporation, in a stroke of electromarketing genius took the basic 8080 architecture and doubled up a small portion of the internal workings of the CPU's address circuitry. They basically duplicated the address register (the transistors that hold the pattern of bits to place on the address bus), slid it left 4 bits, and added some simple circuitry to add it arithmetically to the "old" address register. An so was born the infamous Intel Segment Register. By making this simple kluge to the 8080, Intel created the 8086 and 8088 microprocessors which now effectively had TWENTY address lines and could therefore address 2 to the 20th locations, a little over one million bytes. This new unit was called a Megabyte even those it was further away from a million than a "K" was from a thousand, but then again you bought the "K", didn't you? The 20 bit address bus allowed the new amazing spreadsheets to really do some amazing things. This, in turn allowed Intel to beat competitor Motorola to the marketplace (who were expanding the address bus the "right" way in their 68000 family). This in turn led IBM to select the Intel family over Motorola's as the basis of their new PC, and the rest is history. Using the power of the 8088, the Microsoft corporation adapted the CP/M operating system to the new chip, and, using its new power, created MS-DOS. Because of a lot of reasons, the 8088 and MS-DOS took over the world. And everyone was happy. Except ..... the programmers kept getting more bold in the ways they found to use memory, and the users wanted still BIGGER spreadsheets, and started playing with things like CAD/CAM, DBMSs, Artificial intelligence, desktop publishing, etc, all of which had insatiable appetites for memory. The silicon boys kept up with the hunger by developing bigger and better CPUs, the 80286, 80386, 80486 etc.). These 80286 had a 24-bit address bus and could therefore address 16 MB of memory directly. No one could possibly want to put that much actual memory in a PC, right? In a pre-emptive first strike, they also created 80386 which had a 32-bit address bus and could therefore address 2 to the 32 bytes or over 4 billion bytes! (Pow! Bam! Take THAT, Power User). The day had finally come when the CPUs sitting on desktops could finally address more memory than anyone could afford to buy. ----------- 1 Yes, there was an 8008, and a 4004 before that, but their only, surviving significance today is that they were the predecessors of the 8080. ---------- End of problem, right? Wrong. Unfortunately, there were millions and millions of the 20-bit machines out there now (in the mid-1980's). Probably more significantly there were hundreds of millions of dollars invested in MS-DOS software that did not know how to use the new 32-bit instructions and capabilities. Especially MS-DOS. (Unix and OS/2 and a number of other less well- known operating environments do use the 32-bit mode, but MS-DOS was still king). Because Intel wanted to sell more than 3 of these new chips, they wisely decided to build "modes" into the new chips to make them compatible with MS-DOS. A Mode is essentially a switch, inside the CPU that turns it into another chip, insofar a all logical functionality is concerned. So when you're running MS-DOS on your shiny new 386 or 486, you're still running with only 20 bits of address and consequently a 1 MB address space. This mode is called real mode and the lower 1 MB of addressable memory in real mode is called conventional memory. The solution? EMS or Expanded Memory Specification. EXPANDED MEMORY Expanded memory is a way to allow more than 1 MB of memory to be used by MS-DOS applications. How can this be? The CPU can only address 1 million different addresses; how can I have more than one MB in my PC? The answer is that the CPU re-uses the same address to get to more than one byte of data. It does this by allowing any one address to actually be used to reference several, distinct, physically different storage locations - but only one at a time, of course. EMS memory usually resides on special "EMS cards", like the AST "RAMPage". (I say usually because there are some clever ways of implementing EMS on 80286 and 80386 machines that don't use "special" EMS cards; this is discussed later in the section entitled "Software Approaches to EMS"). The EMS card has memory chips on it just like regular system memory cards. The difference is that the memory chips on the card are not connected directly to the CPUs address bus. These chips are actually wired to another address bus, totally contained on the card, that has more than 20 address bits, usually 24 or so. Where do these extra address bits come from? Well some of them are just passed-through systems address bits. The rest come from special storage locations onboard the EMS card, called page registers. How are these page registers loaded? These registers are themselves directly addressable by the CPU. To access a byte of data in EMS memory, the CPU first loads the page register (itself simply another location accessible by the CPU) and then makes a normal memory reference. Some of the bits come from the address bus, and the rest come from the bits previously squirreled away in the page registers. Thus, it can be seen that a given address on the regular CPUs address bus forms only part of the address needed to select a particular byte of memory on the EMS card. To uniquely identify a byte, you need to specify the regular address plus the contents of the page register. It follows that one CPU address can correspond to several EMS memory locations, each of which differs only by the contents of the page register. The CPU can thus access more than 1 MB of memory on the EMS card by using its normal address bus augmented by the page registers. HOW IT ALL FITS TOGETHER In the early versions of EMS, all EMS memory was mapped to appear to be in a special address range in the range of all possible addresses accessible by the CPU. This was usually at addresses between D0000 and DFFFF (in hexadecimal notation). This includes exactly 64KB possible addresses. This area of address space is called the Page Frame, an analogy to a frame around a picture. The page frame is logically divided into 4 16KB "pages".1 When we say that the EMS memory is "mapped" into this range, what we mean is that the EMS card does not respond to any addresses outside this range. When an address is placed on the address bus ---------- 1 For the sake of simplicity in the following discussion, the multi-page nature of the page frame will not be mentioned further. The explanations apply to each page within the page frame individually. It should also be noted that, strictly speaking, the 4-page, 64 KB page frame applies only to EMS versions 3.2 and earlier. In EMS 4.0+, the page frame is not limited to just 4 pages and can, in fact, be all or partially located within the 640 KB address range normally occupied by conventional RAM. This feature is used by various multitasking overlays to DOS, such as Desqview, which actually allow program code to be paged in and out. ---------- outside this range, the EMS board remains totally passive, just like it was not plugged in at all. When the CPU asserts an address within this range, the EMS board comes to life and responds like regular memory. When the CPU references the EMS address space, the CPUs address bits are used along with the page register bits to actually specify which EMS byte to access. The net effect is to make the EMS card memory appear as a series of "banks" which can be made (one at a time) to masquerade as "real" system memory at a certain address in the range D0000-DFFFF hex. These banks are called EMS pages. Within each page, the practically any byte addressed is selected by the system address bus bits that were passed through by the EMS circuitry. The particular page selected is determined by what value was previously written into the EMS page registers. All the physical EMS memory locations that respond to a common value in the page registers are said to be in the same page. That is, once that special value is loaded into the page register, any of those locations can be accessed by the CPU using normal memory read/write instructions. If another value is loaded into the page registers, a totally different set of EMS memory bytes (i.e, physically different transistors) are made to respond to the same CPU addresses as before. The model that this behavior suggests is that the EMS memory, divided up into 16K pages, exists out in limbo somewhere, and is unable to be addressed by the CPU in any way. The CPU can however invoke the right magic to instantaneously plug any one of these disembodied pages right into its addressable memory space. The magic consists of loading the Holy Page Register. The CPU can, with equal ease, banish that same page back into limbo, by putting a different value in the page registers. The thing that makes this magic useful is that, any data stored in an EMS page is faultlessly remembered even after it has been banished to zombie land. And, furthermore, that data can be read by the CPU just by remapping it into the address space. This means that an MS-DOS program can juggle megabytes of memory resident data using just 20 address bits in good old 8088 real mode. Of course at any one given instant, most of that data is in zombie land, but no matter, it can be called back from the netherworld with a simple, hardware-assisted incantation in microseconds. EMS MEMORY MANAGERS OR DRIVERS Each manufacturer of an EMS board is free to actually design the actual circuitry of his EMS board to suit his whim, his engineers, and his marketing plan. Most boards are different in a real, physical way from one another. The magic incantations necessary to shuffle EMS pages between here and Hoboken is different for each one. Does each application program need to know which particular brand/model of board is plugged in and what its religion is? Fortunately not. Enter the Enhanced Memory Specification. EMS is a specification of a standardized way that applications interact with EMS hardware. This interaction is via the software interrupt feature of the 8088/MS-DOS. All applications that wish to use EMS memory call interrupt 67H the same way with the same arguments, no matter who made the board. When the interrupt is issued, control passes to the memory resident EMS management software, usually called either an EMS memory manager or EMS driver (same thing). This piece of software is unique to each brand of board and is normally supplied by the boards vendor. It is the express purpose of this piece of software to turn the standard EMS invocation arguments into the particular set of hardware incantations understood by the board. Beware mixing boards and drivers from different sources! This may work in rare circumstances but will eventually lead to consumption of excessive amounts of alcoholic beverages. EXTENDED MEMORY OK. Now we know how EMS works: it expands a selected 64KB- sized range of addresses in the CPUs address space to several megabytes by paging-in one chunk at a time. But what about "Extended Memory"?. Actually extended memory is a much simpler concept. Remember those unused address lines in the 80286 and 80386? (MS-DOS and other real-mode applications only use the first 20 of the 80286's 24 and the 80386's 32). They were not put there for decoration. The CPU can be put in Protected Mode and can then use those extra address lines to address megabytes and megabytes of memory without needing the help of the special EMS hardware like page registers and private (EMS-card) address busses. In protected mode, the CPU can address all physical memory in the same, natural way. In fact, the one megabyte boundary loses all significance, except for the painful memory of what it used to be like back in that awful 20-bit real mode. Extended memory is thus just like conventional memory, just extended up to higher addresses. The down side is, of course, that MS-DOS does not know how to switch into protected mode, and wouldn't know what to do there if it did. Rectifying this shortcoming, and all its implications, is the sole reason for OS/2. SOFTWARE APPROACHES TO EMS IMPLEMENTATION The discussion of EMS so far has talked exclusively about hardware approaches to EMS. In the 8088, hardware must be employed to supplement the deficiencies of that chip. In the 80286 and 80386, however, there are software-only methods to give the same functionality as EMS hardware. Both approaches use extended memory for the storage of EMS page data. In the 80286, EMS memory contents are brought into the 1 MB, conventionally-addressable range by physically copying 16 KB blocks of memory to and from extended memory. The EMS "page" that the application program sees is actually a block of conventional memory that has been filled with the contents of a block of extended memory. The copying is done by a piece of software known as an EMS Emulator (driver) which is usually loaded like other DOS device drivers in CONFIG.SYS. Note that to access the extended memory, the EMS Driver must switch into protected mode, copy the data, then hightail it back into real-mode to keep DOS happy. The extended memory blocks, in this scheme, are emulating a block of memory that would normally be physically resident on the EMS card. Note that these are not really "paged"-in in the same sense as true EMS pages, nor is there any "mapping" going on. All physical memory contents retain their actual addresses as far as the CPU is concerned, only there contents are copied back and forth. The advantage of this scheme is that EMS functionality can be achieved in machines without actual EMS hardware. A disadvantage of this scheme is its performance. Whole 16K blocks must be moved to access a new page (which takes milliseconds), in contrast to "true" EMS where just a page register must be loaded (which takes microseconds). Another disadvantage is the fact that some precious conventional memory is consumed by the emulated page frame. In the 80386, the solution is much more elegant. In true EMS, the contents of the page registers can be thought of as a memory- mapping table. That is, the contents of the page register, in essence, point to a particular block of EMS-card-resident memory - change the page register contents and a new physical page shows up in the page frame. The 386 was designed for operating systems much more sophisticated than MS-DOS; these operating systems take for granted the presence of memory mapping capability. The 80386 has, in fact, an internal memory mapping facility much more sophisticated than the crude, bank-oriented page registers of an EMS card. The 386's memory management unit allows any arbitrary- sized chunk of physical memory to be mapped to anywhere in the address space, including the lower 1 MB. And, most importantly, to the address were an EMS-aware application expects to find the page frame and the EMS pages contained therein. With the 80386, hardware within the CPU performs the mapping previously done by EMS hardware. Programming of the CPUs mapping registers is performed by a device driver usually known as an Expanded Memory Manager. Memory managers are written to run on the (standard) 80386 and not some particular vendor's EMS hardware. This allows third-party vendors to produce EMS emulators for any 80386. Examples are "QEMM-386" by Quarterdeck Systems and "386-to-the-Max" by Qualitas. Finally, there is one more software approach to EMS that can be used with any machine. That approach is called Virtual EMS and employs a system's hard disk as storage for EMS pages. A device driver intercepts EMS calls in much the same way as the approach described above for the 80286, except that copying is done between a page frame in conventional memory, and sectors of your hard disk. This is a clever approach, and allows EMS memory to be much greater than the amount of memory in your machine, but, because disk is thousands of times slower than semiconductor memory, this approach should only be used by the terminally desperate. APPENDIX - SUMMARY FOR USERS EMS is the specification of a software technique for making more than 640 KB (the normal DOS limit) available to your programs. Put simply, EMS reserves a block of memory space in your PC and allows a block of RAM (usually resident on an EMS card) to be switched into that address range. There are generally many identical blocks of RAM present on the EMS card, each and any one of which can be "plugged" in -- only one at a time. Your CPU can use one of these blocks to store data in, and then switch in another block, store data in that, switch in yet another block, and so on, and so on. Later, the CPU can recall these blocks in the same or different order and read back the original data. In many ways, this performs the same function as your system's disk -- except that it's all done in solid-state memory and is thousands of times faster. There are four approaches to actually implementing EMS, depending to some extent, on what type of machine you have. These approaches are: - an EMS memory card (like the AST "Rampage") [any DOS computer, but usually 8088s or 80286s] - an EMS emulator [80286 or 80386, but usually only on 80286's] - an Extended Memory Manager [386 only] (for example Quarterdeck System's "QEMM-386 or Qualitas's "386- to-the-Max" ) - a Virtual Memory Manager [any DOS machine] An EMS memory card is more than just a memory expansion card: it contains special circuitry to perform the bank-switching operation discussed above. To use an EMS card, you will have to perform two steps: (1) Configure the EMS card hardware to match your computer's configuration and (2) install a special EMS card driver for the board in your CONFIG.SYS file. Details and procedures differ for different makes and models of EMS cards. Consult your EMS card's users manual for instructions. Note that drivers are usually card- specific; you cannot, in general, use Vendor A's driver with Vendor B's card. Not all "memory" expansion cards are EMS cards. There is another type of memory called extended memory which is used by other operating systems such as OS/2 and Unix. It is also used by a few DOS utilities, most notably IBM's VDISK RAM disk emulator. If your computer is advertised as having more than 640 KB of memory installed, it's a good bet that it's extended memory and not EMS memory. Few applications can use extended memory, although by using a software technique which will be discussed in a moment, extended memory can be made to serve as EMS memory. Before deciding on an EMS strategy, determine the type of memory your computer already has installed. Be forewarned: IBM, as usual, has a different name for extended memory (like everything else). They call it (you guessed it) expanded memory. So, if you bought it from IBM, and it's called expanded memory, it's extended memory. Everyone else pretty much sticks to standard nomenclature but to be sure, look for the phrase "EMS x.x compliant" in the documentation, where x.x is usually a number like 3.2 or 4.0. ---------- 1 The 'S' in EMS stands for "Specification". EMS is not a particular way to build EMS memory, rather, it is the specification of a software interface to it. Different vendors can, and do, implement EMS differently. What is the same, however, is the way that applications programs interface to this memory. ---------- Many EMS cards allow the memory contained on them to be configured as all EMS memory, all extended memory, or a mixture. If you use VDISK or any other special programs that use extended memory, you may wish to reserve part of the board's memory for use as extended memory. Otherwise, there's really no good reason for not configuring all of your memory as EMS. (Note that EMS boards are generally more expensive than "plain" extended memory boards because of the additional circuitry required). Consult your board's users manual for the proper switch settings or software settings to give the mix you desire. EMS cards can be used in any machine, but are usually found in 8088s and 80286s because there is a better and cheaper way to go in 80386s as will be discussed below. EMS EMULATORS A less common approach to implementing EMS in your computer is a EMS Emulator. This is a software-only approach that requires no special hardware to use. It essentially turns extended memory into expanded memory. Unfortunately, there is a price for this magic - performance. Because extended memory lacks the special hardware to switch its address like the blocks of memory on an EMS card, this software must copy whole blocks of data (16 KB's worth) back and forth between your program and extended memory every time a new block is required -- even if it's just to read a single byte. Depending on the nature of the program, this can be a few times slower or hundreds of times slower than "true" EMS. This is not a recommended solution for that reason, however, if necessary, it can be used. This approach can only be used on 80286 and 80386 machines which have extended memory. Machines based on the 8088 (like the original PC and XT) cannot accommodate extended memory. On the 80386, a much better solution is described below. EXTENDED MEMORY MANAGERS Built into every 80386 is a special capability that can be used to do an excellent job of providing EMS memory without the use of EMS hardware. This facility is called the paging unit or Memory Management Unit (MMU). The MMU consists of circuitry very much like the switching circuitry onboard EMS cards, except much more sophisticated. It was actually included for use by advanced operating systems but can be used quite nicely to emulate EMS in 80386-based DOS machines. The MMU, like the EMS emulators, can turn extended memory into expanded memory through software-only means. Unlike those emulators, the MMU, in conjunction with a piece of software known as an Extended Memory Manager (EMM), does not suffer any performance penalty. In fact, it is usually faster than true EMS cards because: the circuitry is onboard the CPU chip; the 80386 is faster than lower-class machines that usually use EMS cards; and the extended memory used is often fast, 32-bit system memory rather than card-based memory which is slowed down by the relatively slow I/O bus. On 80386 systems, this is definitely the way to go. Excellent EMMs include "386-to-the-Max" by Qualitas, Inc., and "QEMM-386" by Quarterdeck systems, Inc.. To use these EMMs, you need to install them in your CONFIG.SYS file. Like the EMS cards, you will have to configure them to partition your available system memory between extended and expanded memory. Consult the users manual for the package you are using. VIRTUAL MEMORY MANAGERS Virtual Memory Managers are another software-only approach to EMS. These function almost identically to the EMS emulators discussed above, except that they use the system disk rather than extended memory as the storage medium for blocks of memory copied out of your program. As you can imagine, this is excruciatingly s-l-o-w. Use this approach only as a last resort.