Black Box 4

home *** CD-ROM | disk | FTP | other *** search

/ Black Box 4 / BlackBox.cdr / database / datamage.arj / CODE.ZIP / ISAM.TXT < prev next >

Wrap

Text File | 1990-08-20 | 15.7 KB | 303 lines

D I S C L A I M E R The code included is provided AS IS, with NO WARRANTY WHATEVER. If you chose to place it into your programs YOU are responsible for verifying that the overall program, including code furnished by HCWP, works. I will bear no liability OF ANY KIND for this code. And I will MOST CERTAINLY NOT guarantee it to work in YOUR program! All I can say is: It works here, in mine. DATAMAGE The draw_border function included with your sample places the DATAMAGE banner atop the screen. DATAMAGE is a registered trademark of Monte Ward and HCWP. You may NOT use it in your programs! THE CORE LIBRARY The module of code called HCWP_LIB.C is the core library, and has it's own docs in the file: HCWP_LIB.TXT. THE ISAM LIBRARY The module of code called MAGEISAM.C (and MAGEISAM.H) are the C language interface to DATAMAGE files. So far. This module was compiled with the QUICK C compiler (2.5) and, save a few of those dern long/short mis-matches, compiled perfectly. If you try to use another compiler, GOOD LUCK. MIGHT work, but you'll find dieeetomsbin and dmsbintoieee functions, which turn BASIC formatted double-floats into ieee formatted double-floats don't exist in many compilers, so be prepared to do it yourself. The module called MAGEDEMO.C uses SOME of the functions to create a simple program. It's purpose is to provide a practical demonstration of the use of the ISAM, and to provide a platform for your programs. It is NOT a slick, finished product. Merely a template. WHAT IS AN I.S.A.M. ? What we have, here, folks, is the GUTS of a database. ISAM stands for Indexed Sequential Access Method/Manager. In this module is the code to read/write records, find records, load DATAMAGE files - the basics. The DATAMAGE ISAM was designed for maximum speed, low disk/memory overhead, to make optimum use of MS-DOS and, most importantly, to be easy to program with. This is FAR easier/simpler than dBase methodology, for instance. DATAMAGE ISAM PARTICULARS The files are flat and assessable to the BASIC interpreter. So, you CAN write hybrid programs with it, if you must. But, PLEASE do yourself a favor and get QUICK BASIC if BASIC is your thing. See POWRMAIL.BAS. For those of us who prefer C, the format of the files on disk makes little difference. Whatever it is, we can handle it. But simplicity of design/operation and the aforementioned speed and LOW OVERHEAD are still valuable. And, of course, the accessibility of your files to the DATAMAGE system adds many features to your programs. The ISAM is NOT designed to be state-of-the-art, just to WORK (fast!) and to be used by ordinary human beings. DEFINITION FILES One BIG difference between DATAMAGE files and ordinary files produced by BASIC is that they have definition files. In a "dedicated" program definition files aren't necessary - we know the details of the file and hard-code them into our program. This necessary with some of the big-name ISAMs, like B-TRIEVE. Other files, like dBase, have the file definition "built in" to the main datafile. This makes it MURDER to process the file and almost completely dis- allows BASIC from processing the files. But, it DOES make it possible to transport the file more easily. To transport a DATAMAGE file you simply copy all the files in the sub-dir. That's harder. DATAMAGE files have their definition in two files: HEADINGS.SAD and KEY.SAD. The headings file has strings, maximum 25 chars, and the key file has three numbers in it's 33 byte records, ascii format. These files are read by the open_file function, and the DATAMAGE file structure is filled. From then on, the information about a particular field in a particular file is at hand in the structure. To find out if field four in file four is numeric you might go: if (fi [4].key [4] -> type > 1). fi is the file-info structure, subscript gets us file four (the FIFTH file loaded), .key gets us into the array of far pointers to structures defining the fields, -> type gets us the field type, which is defined in the code. So, the DATAMAGE isam provides you with the power to load and process ANY file created by/accessible to DATAMAGE. It's loaded with PRACTICAL features the big boys just don't have, like read/writing ANY individual file/field, instead of having to move seven thousand bytes to/from disk just to change one. And a window that allows the user to select a field in a file, etc, etc, etc. DATAMAGE ISAM - FORMAT AND PARTICULARS SPECS RECORDS IN FILE: 32,000 FIELDS IN FILE: 200 MAX STRING FIELD LENGTH: 35 ALL NUMERICS DOUBLE PRECISION, MBF FORMATTED 8-BYTE STRINGS MAX INDEXES: 5 INDEX MODES: UNIQUE, NON-UNIQUE, CROSS RECORD NUMBERS: MANDATORY, UNIQUE MAX FILES OPEN: 8 RECORD NUMBERS Each record in a DATAMAGE file has a number. It may be 1 - 32,727. The number is stored in the file CTRLFILE.RAD. If the number is zero, the record is a "hole" and is available for use. No record number can be duplicated. FILE INDEXES DATAMAGE indexes are simple NUMBERS. The targeted string is turned into a signed word and recorded in the file: FILEINDX.RAD. The indexes have the capacity to hold five two-byte integers, so the records in the file are ten bytes long. This SIMPLE method offers INCREDIBLE speed in searching and updating, and is the easiest to understand BY FAR. SEQUENCING The first record in YOURDATA.RAD, the main datafile, is numbered by the first record in CTRLFILE.RAD and indexed by the first record in FILEINDX.RAD. So, if we were searching in FILEINDX and found something we liked, we would know that the record to read in YOURDATA.RAD would be the SAME sequence as the index record we now had. We could get a field out of it to compare, or even the whole record if we needed it. Likewise, if we went searching in the CTRLFILE.RAD and found a record number that struck our fancy, we would know that it's record was the SAME sequence in YOURDATA.RAD as the record number we now had from CTRLFILE.RAD. CUSTOMIZATION The ISAM is provided as SOURCE CODE. If you have a program to do that requires only two files open you may find it convenient to open the three associated files in each DATAMAGE file and just LEAVE them open during the program. Feel free to do so. The ISAM is set up to process eight files, and thirty-two files open is more than most folks allot for in their config.sys file, and would waste memory most of the time. In a program you have you want it to process one file, and ONLY one file. So, you won't need the show_datafiles function. You will pass a constant to the open_file routine. Feel free to REMOVE unused functions from your final version. There's no sense having un-used code. LAN OPERATION You will need to add your own record locking. May I suggest you read the file's record from the CRTLFILE.RAD file, multiply it by -1, and write it back? That way, the main record could be "read-only." Locked, but still readable. This bypasses the locking on the lan, and a LOT of grief! PROGRAMMING EXPERTISE I can but assume that you are a better C programmer than my humble self. I am just a hacker from Indiana who likes to punch these keys. Anyhow, you SHOULD understand C (if anyone REALLY does?) and have written random-file programs of your own, possibly incorporating an ISAM. If you are just learning C (aren't we all?) this AINT the place to start. If you are wanting to learn random-file processing and already know C you will find the code invaluable. Whatever your level of digital dexterity you are going to have to THINK to get this. There are remarks in the code, but they are "slim." You will need to read the function's code to figure out what global variables it takes as input, and changes as output. If I listed this you wouldn't read the code. NOTHING NEW After programming for a number of years I realized that the really BEAUTIFUL solutions all had only one common ingredient: SIMPLICITY. And the DATAMAGE ISAM, pretty or not, has a liberal dose of that. This is the way it's been done for YEARS, folks, on many a multi-million dollar mainframe computer. All I've done is to simplify it, adapt it to a PC, and add my indexing scheme which I thought about FOR YEARS while using convoluted and overhead-intensive alternatives. My way may be described as brute-force computing, but I, for one, can do without the errors that "finesse" often generates. DATAMAGE INDEXES - THE BRUTE FORCE THEORY As the hardware progresses BRUTE FORCE works better and better. Modern machines running at 20 Mhz with RLL controllers on FAST hard drives, possibly cacheted with smartdrv etc, might as well be SLOW ram. While other ISAMs are half-way up their B-trees I will have checked a thousand records with brute force. They say: "I'll find the last record as quickly as the first!" Isn't that like saying: The first record takes as long to find as the last? You may have noticed that the RAM_BASE program within DATAMAGE will find a record in a hurry. That's because it can load up to eight thousand record's indexes into 80K of ram. Then, it does index searches without bothering the disk drive. In this case, I will have checked FIVE thousand records with brute force before the B-tree is clumb. In either case, all I need to do to update an index is to decode the string, turn it into a number (routines supplied) then write it into the index file in the proper sequence. The write_MAGE_record function does this for you. And, from it, you can easily see how to do it yourself. BRUTE FORCE programming provides both the programmer and the computer a rare opportunity to run flat out. You don't spend a lot of time reasoning; that's not what a computer does well, and reasoning introduces the opportunity to err. Just target 10 bytes, gotten off disk or in ram, compare one or more of then to search values, repeat if not equal and not end of file. THAT's what a computer does: many FAST repetitions of some MINDLESS operation. BRUTE FORCE, when well-directed, often results in the lowest overhead to do the job. DATAMAGE indexes are just ten bytes per record, and can cover five fields; any/all of which can be unique or cross. I have often seen data "sets" (consisting on one or more files) in which the indexes took up FAR more disk space than the data. And the PROGRAMMING overhead often equaled/surpassed the actual procedure of the program and entailed linking in object files for which I had no code or using code I didn't understand and couldn't have fixed if my life depended on it. And, for what? GETTING STARTED In the first couple screens of the MAGEISAM file the structures: fi (defining DATAMAGE files) and finfo (defining fields within DATAMAGE files) are defined. Copy these definitions to a piece of paper, then load the MAGEISAM.C file into your compiler. Search for open_file. Read the function, while referring to the paper. Dig it. All the information on the loaded file is there, but YOU will have to write the if statements to check it, and your understanding of the ISAM depends on "C-ing" these structures. Once you "C" the open_file function the rest should become crystal clear. The open_file function uses two of it's own FILE *s. Such is not the case with the majority of ISAM functions. In any environment that has up to eight files open, requiring the use of three FILE *s each, there's going to be a shortage of IOCBs. And, it's FASTER to test a file to see if it's open than to (attempt to) open it. LOTS faster. So, then, HOW to test a file for open status? LOTS of ways to do that, too, Many of which involve linking in and calling yet another function. Got lots of THEM, now. I decided to set the file pointers in the fi structures to NULL during the load, then set them to NULL every time I closed them. That's EVERY TIME, hear? The first time you don't, you'll be SORRY! The ISAM functions test the files, then open/close them if they were closed. If you don't set the FILE * to NULL after closing a file the next ISAM function you call on that file will "think" it's open. NOT good. Not to mention trying to close it, when it's not open. A "cheapie" way around this is to use your own FILE *s. NOT recommended. Use mine. Like I said, they're in short supply. Let's say you have the main file open and you call write_MAGE_rcd. Stipulate further that there are no "holes" in the file, and a record is ADDED. But, it was added "to" the FILE * in the fi structure, which was closed. YOUR file pointer doesn't "know" it's there. What will actually happen is dependent on your DOS version, your controller, etc, etc. It's a chance you DON'T need to take. I have seen it corrupt the fat, (yeah, I make mistakes, too) and maybe it could do worse? THE files_loaded VARIABLE Every time you load a file you MUST increment files_loaded. And decrement it when you close it. It's defined in the first screen of code. If you wish to add this to the open_file function, by all means. But I sometimes read it off disk, then use it as comparitor for a for loop when loading files. If the files_loaded BYTE is set to a value not equalling the number of files open, expect trouble. The files are opened/closed IN ORDER. You COULD open file #8 first, but WHY? I never have, so I don't know if it will work. COMPILING The ISAM is contained in a module of code. Most all of it's functions are FAR, so that they can be called from other modules. This means you will need to use the medium memory module when compiling. NO, huge won't work. The ISAM calls many of it's own functions. These functions are in the same module as the calling function, but their definition requires them to be far calls. Far calls take longer than near calls, and occupy more code space. To rectify the catche-22 above, the linker has a /F option. This converts all far calls to functions that occur in the same module to near calls. Saves time and memory. To use it, type /F after link/qlink. WRAP-UP Yeah, I could have compiled it all to object files and simply made a list of functions, their parameters and returns. I always like to be different. YOU may have to do a little work, but you will actually understand the ISAM after you do. You can even change it, if you DARE. It's true that only a real programmer can understand the code. And, in my opinion, just as well. If you can't read and understand the code you've no business writing this kind of program in C. If you think you might, but avoid dis-appointment by not playing - you LOSE. If you're just plain too lazy to read the code and, thereby, understand it and be, therefore, able to modify it - you LOSE, too. But there are PLENTY of high-tech, object-file (and even tsr) ISAMs to fill your "need." And, all you have to do is to link them in, call them like THIS, and they return like THAT. You won't be able to access their files with DATAMAGE, though. It's fine, until you need to "call" them a way they don't call, or get a return they don't return. If you are satisfied with limiting your programs, in both scope and performance, to their capabilities then your standards are lower than mine. I like to take PRIDE in MY work.