Programmer 7500

home *** CD-ROM | disk | FTP | other *** search

/ Programmer 7500 / MAX_PROGRAMMERS.iso / CLIPPER / MISC / TSR.ZIP / TSR.TXT

Wrap

Text File | 1989-06-09 | 20.7 KB | 364 lines

July 1987 The History and Technology of TSR Terminate and Stay Resident software by Steve Gibson InfoWorld TechTalk Columnist and President of Gibson Research Corp. HISTORY BEGINS ---------------------------------------------------------------- Once upon a time a Big Blue company named IBM decided that it wanted to make personal computers too. For reasons known only to the Gods of Boca Raton Florida, IBM decided to use the 8088 microcomputer from Intel rather than Motorola's vastly "cleaner" 68000 (as is used in Apple's macintosh). Viewed in retrospect, this has turned out to be a ridiculously expensive, i.e. WRONG, decision. But anyway... IBM journeyed to the West to get its software for this new machine. IBM first visited the company who was then the acknowledged king of operating systems, Digital Research, Inc. DRI had developed CP/M, the defacto standard operating system for the very popular 8080 and Z80 microcomputer systems. IBM explained that it needed an operating system for the new 8088 microprocessor in their new, blue, personal computer and asked if DRI would please provide them with one. DRI said, quite truthfully, that they didn't have one. Big blue must have then asked if they would like to make one, to which DRI must have again said: "No thanks." IBM shrugged and headed North. . . . . Though IBM may have only planned to ask Microsoft for an 8088 BASIC language interpreter for their new machine, they found themselves in Washington without an operating system. Microsoft said they would be pleased to provide IBM with both an 8088 operating system and the BASIC language interpreter. IBM said "Thank you" and said they needed it very soon please. But Microsoft didn't have an 8088 operating system either, and being so very much smaller then than now, didn't have the resources to build one from scratch by IBM's deadline. So they found an even tinier company called Seattle Computer who had built a little unpretentious operating system for the 8088 and 8086 micros called 86-DOS. It was also known by some as QDOS. QDOS was said to stand for Quick and Dirty Operating System. So Microsoft, then able to meet IBM's 8088 machine timetable, delivered both an operating system (albeit one of unassuming parentage and questionable virtue) and a BASIC interpreter. Everything seemed fine until we all began really using the new operating system. We wanted additional and rather reasonable things from MS-DOS and PC-DOS. Things which had not been designed into 86-DOS, like printing in the background while the user was doing other things. Or supporting RS-232 serial printers, which required routing the printer data out through a serial port instead of the normal parallel printer port. To meet these reasonable demands Microsoft made a series of small changes and additions to the guts of MS-DOS which allowed extra "add-on" code to be "hung" onto the outside of MS-DOS and also provided for limited communication between DOS and the add- on code. Microsoft's idea was that these MS-DOS limitations would be "fixed" by writing some new DOS commands (in the above examples PRINT and MODE) which would make use of special undocumented aspects of their latest version operating system. The problem was that MS/PC-DOS, still really old 86-DOS (QDOS) in disguise, was never built to support multitasking. This means that it was never meant to serve more than a single master at a time. A command like PRINT, which can continue to operate even AFTER the user has begun doing something else, could very easily confuse it, since now there could be service requests coming into MS-DOS from two quite different places at once. Since this began making the entire operating system rather fragile and weird, Microsoft's grand plan was to retain all these special hooks and communications paths as a company secret. They figured that as long as they were the only ones to use these new MS-DOS secrets things wouldn't get out of hand. But Microsoft must have gravely underestimated the cleverness and determination of the development community. When the MODE command proudly stated: "Resident portion installed" developer's eyebrows shot up! RESIDENT portion? Hmmmmmm. And if the PRINT command could access the disk for file printing when we were messing around with a spreadsheet, why couldn't we build a little resident notepad or two, too? But EVEN after these new resident goodies began appearing in increasing numbers, Microsoft refused to "disclose" anything whatsoever. They said they "couldn't" document these things or they'd be forced to make all FUTURE versions of the operating system support them in upwardly compatible fashion. Since these were awkward "kludge" solutions to the real need for a true multitasking operating system, Microsoft's delicate condition can be readily understood. However, developers sensing massive market opportunities were not to be stopped. Debugging software which allowed programmers to peer inside the still working machine sold like wildfire. Using these debuggers (like PERISCOPE, my personal favorite), developers probed into the very heart of Microsoft's own PRINT and MODE commands, unscrambling and deciphering the undocumented secrets of their operation. Using the same secrets Microsoft's own developers had used, they learned how to give their OWN programs similar capabilities. In an old episode of Star Trek, Mr. Spock said, after capturing the Romulan's cloaking device, "...military secrets are the most fleeting of all, Commander." This applies equally well to technological secrets in general, and as we've seen, to Resident Software in particular! It's unfortunate that Microsoft did not have time back then in the beginning to build their own DOS from scratch. They might have done it right. But we're stuck with what we have today, and making the most of it is certainly our best strategy. It remains unfortunate, though certainly understandable, that Microsoft isn't willing to disclose the details of MS-DOS's resident software hooks and techniques. This policy keeps such knowledge quite secret, since even the third-party developers are reluctant to disclose their own hard-won "insider's" knowledge and proprietary techniques for forcing reliable multitasking behavior from good ol' single tasking DOS. It's quite interesting to speculate that perhaps today even Microsoft does not KNOW how to write MS-DOS multitasking applications as well and robustly as several of the larger commercial resident software publishers! The tricks required to make DOS multitask are real nasty stomach-turners. So... what are these tricks and techniques? How DO resident programs "pop up over" applications at the simple press of a "hot key"? THE TECHNOLOGY OF RESIDENT SOFTWARE --------------------------------------------------------------- Since we've now know how we got to wherever it is we are, let's answer the those questions posed above.... Two things are required for the functioning of TSR technology: First, the TSR program needs to somehow "hang around" well after any other program has started to run. Secondly, it has to "know" what's going on in the system to be able to jump in and help out when called upon for action. It also needs some way of re-assuming control of the machine when the time comes for it to take action. The requirement of continuing RAM residency is satisfied by a service provided by the operating system from which TSR's get their name: Terminate and Stay Resident (TSR). The main operating RAM memory of an IBM compatible PC is a contiguous block beginning at address zero (called the bottom of RAM memory) and extending "upward" for a maximum of 640K bytes (called the top of memory). Note that 640K is actually 655,360 bytes since "Ks" in computerdom are really 1024 bytes, not 1000. Since the very bottom of this large contiguous block of RAM is reserved for system housekeeping purposes, DOS is "booted" by copying it from disk into RAM just "above" this low-RAM region. Once DOS has been loaded, all the space "above" it, to the top of RAM, is available for application program loading and use. We'll call this location just above DOS the LOAD POINT for application software. So, when a program is started, it is copied from disk into RAM starting at this LOAD POINT, continuing upward until the entire program has been loaded. The memory available, and allocated, to the program for its own data storage and manipulation extends from the end of itself to the physical top of the RAM memory. Most standard programs word processors, spreadsheets, or databases have "control" of the computer until the user tells the program to terminate. After performing whatever cleanup and file saving might be required, the program simply informs DOS that it is all finished by saying the equivalent of "I'm all finished now, terminate me please". Responding to this "Terminate" request from a program, DOS simply "forgets" all about the program and whatever data it might have had in RAM. It displays the command line prompt and awaits another command. The next program to run is loaded at the same LOAD POINT as before, thus overwriting whatever terminated program was just running, and implicitly making the same amount of memory available to this next program. Now consider the special case of the RESIDENT PROGRAM... It begins innocently, just like any other application program, when its name is typed at the DOS prompt. DOS copies it from disk into the user's RAM memory starting at the same LOAD POINT as any prior application software, implicitly leaving the rest of RAM memory for the program's own use until done, then hands control of the computer system over to this program. Now things start getting interesting! The resident program will generally say hello to the user announcing its intention to hang around for the duration. Then it does a few "non-standard things" to the system. These are the very things MICROSOFT wanted to keep all to itself. These special "things" keep the program from being ex-communicated from system activities after it terminates. Then, rather than simply saying "terminate me" to DOS, it says: "I'm xxx bytes long. Now terminate me but DON'T OVERWRITE ME." Which is to say: Terminate me, but let me Stay Resident. Upon receiving this request, DOS takes control from the program, moves the software LOAD POINT UPWARD by those xxx bytes, and only THEN returns the DOS prompt to the user. Since the LOAD POINT has been moved up by "xxx" bytes, all subsequent programs will be loaded AFTER the programs which ASKED to remain resident, thus leaving it undisturbed in the system's memory! Additionally, since it's the resident program which tells DOS how big it is, it is able to declare itself to be ANY LENGTH IT CHOOSES! This is how programs like SideKick, which are only 39K bytes long while on the disk, can have a 50K notepad file and take up 89K bytes while in RAM! It effectively LIES to DOS about how big it is, so DOS obligingly leaves it space for itself and for however much working memory it wants! This is the way resident programs are able to eat up so much of our main RAM memory! When this process is repeated many time (as is frequently the case) a resident program "stack" is created in main RAM memory. So, we've now seen how programs are able to remain in memory after being "officially" terminated. The final question is: How are these "terminated" programs able to remain "aware" of the activities around them?... and how are they able to regain control of the computer whenever they see fit to do so? In other words, how can a notepad or other utility "pop up" ?? To answer these last questions let's step back for a moment, reviewing a bit about how computer's are controlled: A computer's actions are directed by its reading of instructions which are stored in its main memory. These instructions may be temporary, as in the case of programs loaded from disk into RAM, or permanent, as in the case of instructions permanently wired into ROM chips. In either case, each instruction has a unique address which is just the number of the byte it occupies in the machine's memory. We cause the computer to perform a given task by having it read a series of instructions which detail the task. In a sense we CALL upon those instructions whenever we need to exert their intended function. Such a "CALLED" instruction sequence generally has a clever instruction called RETURN which is able to send the computer back to wherever it was called from! This notion of FUNCTIONS which are being performed by instruction sequences called upon in memory, and which will return to us when finished, is very powerful. IBM compatible computers come equipped with a BIOS ROM which contains a host of very useful general purpose functions which return control to their caller as soon as they have performed their designated duty. This is EXACTLY what that often mentioned BIOS ROM is all about! B.I.O.S. stands for "Basic Input Output System", and consists of a collection of standard functions for reading and writing to the system keyboard, screen, mass storage, and many other standard devices. Naturally, we must know the exact address of these "functional" services in order to CALL them into service. By convention, the starting address of every built-in function in the IBM PC is located in a predefined table of addresses located at the very start of the computer's RAM memory. Every function is numbered simply by the location of its address in this table. This means that in order to call for any built-in IBM PC function we only need to know its standard function number - from this we're able to calculate the location in the table where that function's starting address can be found. There's even a clever computer instruction which lets you directly call upon a function whose ADDRESS is stored at a specific location in this table. So in effect you can simply ask the computer to call "standard function number 7". In response to this function request the computer will look into the 7th slot in the table for the address of function number 7, and go there directly, returning to you when the function has been performed. In computer lingo, a place which contains the address of something else is a POINTER or a VECTOR. Even the electrical hardware of the computer is able to call upon these standard functions. In fact many of the standard functions in the ROM BIOS are designed to operate hand-in-hand with the PC's hardware. For example, every time a keyboard key is either pressed or released the standard function number 9, whose address is naturally in location 9 of the Address Table, is called upon by the hardware. This function is called EVEN IF THE SOFTWARE WAS BUSY DOING SOMETHING ELSE AT THE TIME! This ability of the hardware to INTERRUPT whatever the software might be doing at the time is called a "hardware interrupt". Essentially the keyboard hardware says to the processor: "Hey there CPU, hold on a second, remember where you are and what you are doing, now go perform the function whose address is in slot number 9, and when that function returns to you, go back to whatever is was you were doing when I bugged you just like nothing ever happened." This is EXACTLY how we are able to type ahead of our computers. Even if our spreadsheet is lost in thought recalculating our tax refund, we're able to type ahead of it! Each depression and release of a key steals a fraction of a moment, a slice of time, away from the spreadsheet's recalc. The function invoked by the keyboard's hardware interrupt places the key into a type-ahead buffer ... which the spreadsheet will read from as soon as its good and ready ... then returns control to the recalc as if nothing at all had happened. Now imagine what would happen if some program (like our resident one) were to point the POINTER stored in the 9th table location to somewhere else, such as to ITSELF, prior to issuing the Terminate and Stay Resident (TSR) request to DOS! THEN the hardware interrupt occurring at every keyboard action would turn control over to THIS TSR'd RESIDENT PROGRAM instead of to the normal built-in keyboard handling function! If the resident program wasn't "interested" in that particular keyboard action or keystroke, it could simply pass control along to the place where the POINTER HAD BEEN POINTING BEFORE just as if it hadn't been evesdropping at all! However, if the key action happened to be THIS resident program's assigned "HOT KEY", it could retain control of the system as long as it wished, before passing it along as if nothing at all had happened! That's all there is to it! As we've seen, the clever application of these various tricks and techniques can significantly expand the power and flexibility of MS-DOS and PC-DOS based personal computers. There are those who feel that resident software is a makeshift temporary solution which is not long for this world. They say that TSR programs will be eliminated by the presence of full multitasking operating systems. This author could not disagree more. I believe TSR programs will be here for the duration as necessary and viable add-ons to otherwise incomplete solutions. So there you have it. The history and technology of TSR Resident Software in a nutshell. I hope this gives you a better feel for what's going on under the hood of your machine! Thanks for listening ... and REMEMBER, if you would like to receive notification of other free information, software, and our future TSR and utility products, simply pickup the phone or drop us a note or a card. We'd be pleased to keep you up to date. Steve Gibson. <<< END OF FILE >>>