Programmer 7500

home *** CD-ROM | disk | FTP | other *** search

/ Programmer 7500 / MAX_PROGRAMMERS.iso / PROGRAMS / UTILS / COMPRESS / IDC40.ZIP / NARC.DOC < prev next >

Wrap

Text File | 1990-04-27 | 85.2 KB | 2,165 lines

NARC (tm) zip - A STAND-ALONE DE-ZIP UTILITY (no other files required) NETWORK COMPATIBLE Documentation for NARC.EXE Written by Gary Conway and Shawn Merrick Infinity Design Concepts, Inc. Louisville, Kentucky Copyright (c) 1987-90, All rights reserved Version 4.0 NARC.DOC Copyright (c) 1987-90 Infinity Design Concepts, Inc. NARC.EXE is a shareware program. NARC.EXE is and will remain the property of the author. This program may not be used in any connection with commercial ventures, nor as a sales aid, without the expressed written consent of the author. All rights are reserved. Infinity Design Concepts, Inc. 1052 Parkway Drive Louisville, Kentucky 40217-2333 Voice: (502) 636-1234 BBS: (502) 635-5471 CIS: 72657,3006 Member IEEE AOPA PCCL WINGS KKUG NSPE A very special thanks to Curt Edwards, for running the SoftStone BBS system for many years. Curt has decided to retire from SYSOPing after a long time of helping others, we at IDC would like to thank Curt for his endurance, his patience, his help and most of all, for the Stone.. NARC is a trademark of Infinity Design Concepts Inc. NARC.DOC Copyright (c) 1987-90 Infinity Design Concepts, Inc. REGISTRATION If you find yourself using NARC, please take the time to do the right thing and that is register your copy. You have been provided the opportunity to freely test the program before even thinking about registering. This is only fair, so, in fairness, you should reciprocate and register your copy, if you continue using the program. Why register ? 1) You get the NARCCFG.EXE program for customizing NARC. 2) You get notification of updates to all IDC software. 3) You get patch table information. 4) You get online phone support. Customers residing outside of the United States of America should send a check or money order drawn on a U.S. bank or in U.S. funds or use a credit card. You will find the registration form in the archive with this document under the name REGISTER.FRM. Please use this form for registration. Infinity Design Concepts, Inc. 1052 Parkway Drive Louisville, Kentucky 40217-2333 (502)-636-1234 The user is granted the right to make unlimited copies of the SHAREWARE versions of the program (except NARCCFG.EXE and NARCCFG.DOC) and to distribute these copies as desired without profit, EXCEPT that Infinity Design Concepts, Inc. reserves the SOLE right to distribute the program(s) for profit. NOTE: The word ZIP is used throughout this document and almost all functions that apply to ZIP files, can be applied to ARC and ARK files as well. NOTE: NARCCFG.EXE is NOT a shareware program and may not be released to the general public in any form under any circumstances whatsoever. NARC.DOC Copyright (c) 1987-90 Infinity Design Concepts, Inc. ............................................................................. ........................ ............................ ........................ TABLE OF CONTENTS ............................ ............................................................................. Page FEATURES UNIQUE TO NARC..............................1 WHAT IS SHAREWARE.................................. 2 NARC ? WHAT IS IT ANYWAY........................... 3 ACKNOWLEDGEMENTS................................... 3 ABOUT ZIP FILES.................................... 4 COMPATIBILITY...................................... 5 Archive Storage Methods Supported By NARC...... 5 Packing Squeezing Crunching Squashing ZIP Storage Methods Supported By NARC.......... 5 Storing Reducing Shrinking Imploding OVERVIEW........................................... 6 Video Modes ................................... 6 Mouse Support ................................. 6 Sound ......................................... 7 Environment Variables.......................... 7 HELP............................................... 9 COMMANDS........................................... 9 Extract Command................................ 9 View Command................................... 10 Print Command.................................. 11 ZIP-wind Command............................... 12 DRV-wind Command............................... 12 DirTree Command............................... 12 Quit Command................................... 12 ALTERNATE COMMANDS................................. 13 Function keys.................................. 13 Find Command................................... 14 Kill File Command.............................. 14 Page UP, DOWN, HOME,END........................ 14 Using extensions besides ZIP and ARC........... 14 Calling IDCshell.EXE........................... 14 Command Line Processor..(new).................. 15 Trivia status screen........................... 15 OPERATING HINTS AND SHORTCUTS...................... 16 ERROR MESSAGES..................................... 17 ARCHIVE FILE FORMATS AND GENERAL INFORMATION....... 19 Packing (RLE).................................. 19 Imploding...................................... 20 Huffman coding (SQUEEZING)..................... 21 LZW Compression.......... ..................... 25 DETAILS OF STORAGE VERSIONS (ARC).................. 27 Archive file Header Structure.................. 29 HASHING............................................ 30 CRC - calculations............................. 31 NARC.DOC Copyright (c) 1987-90 Infinity Design Concepts, Inc. ═══════════════════════ FEATURES UNIQUE TO NARC ═══════════════════════ 1) allows random access to ZIPfiles. ALL of the other archive utilities allow ONLY serial access. 2) allows viewing DOC files before extraction. This feature allows you to determine which, if any of the subfiles that you want to extract. This can save considerable time and disk clutter when you extract only the necessary files. 3) has a view feature that allows page up, page down, home end and arrow key movement through a file. Binary files can also be viewed in HEX mode. Also includes a text search function. 4) allows printing files from an archive without extracting the file first. Supports many printing options. 5) allows users with floppy systems to load NARC once and examine many archives from all floppy drives. Other archive utilities must reload (and reread the arc) for each command. 6) directory tree allows one to easily move from directory to directory or drive to drive. 7) large archives are much easier to deal with in NARC since you may page through the subfiles. 8) mouse support for all functions. 10) First and still the ONLY menu driven decompression package with all data compression routines built in ! NARC.DOC Copyright (c) 1987-90 Infinity Design Concepts, Inc. Page 1 ═════════════════ WHAT IS SHAREWARE ═════════════════ Shareware is a concept for marketing software, whereby the author takes the only risks. The purchaser is allowed to take the software home without spending a dime and trying it out. If the software is found to be useful and is used, then the author expects the end user to buy it. From the author's standpoint, he is so confident that you will like his package, that he is willing to GIVE you a copy on the chance that you will take it home and find it useful and want to buy it. There are many advantages to registering shareware software. First and foremost is the fact that unless shareware authors are supported, they cannot maintain the high standards in software that they do, and the prices cannot be maintained at low levels. It takes financial support for any company to continue to develop quality software. The benefit to the author is that he can distribute his software very economically, thus allowing low prices on the software. Another benefit of shareware, is that shareware authors are much more accessible than commercial authors and that means better, more informative support to you. When was the last time you called Micropro and got to speak to the author of WordStar ? How many times has Microsoft sent a nice piece of software to your local BBS and asked you to try it out ? Shareware is a vital and benefical concept, but it requires your support. Look at it this way, when you send in a shareware registration for $20, ask yourself just how many hours of that authors time are you buying with that money, and where else could you get such a bargain ? NARC.DOC Copyright (c) 1987-90 Infinity Design Concepts, Inc. Page 2 ═══════════════════ WHAT IS IT ANYWAY ? ═══════════════════ NARC is a menu driven de-archive facility, written entirely in assembler. NARC allows you to easily move from ZIP file to ZIP file, with the option of viewing, printing, extracting or deleting the subfiles from the archive. The program may be operated from the mouse or the keyboard. Menus are of the musical popup variety to add a little "TechNoFlash" to the proceedings. NARC is the culmination of about 18 months of frustrating effort and 13000 + lines of 8088 source code. NARC was developed using the Norton Editor and the Microsoft Macro assembler on a Kaypro PC and an ALR 386/20. Why.... Because I use a lot of ZIP files and PKzip and the clones are reminiscent of the early Ward Christensen CP/M days in user interface etiquette,I wanted something a little more flexible and friendly to use. I would like to pause here for a second and give a little credit to Mr. Christensen ( the Don Garlits of CP/M ) for the fine FREE utilities he has given to ALL of us over the years. The next time you do a modem transfer, you can thank him and Keith Petersen for the original XMODEM from which all others have transpired. Why NARC... It seemed like a good idea. Short for uN-ARC. The idea was originally Bob Freed's. Acknowledgments.. I would like to thank my parents for not killing me at any early age and my wife for not killing me at a later age... I would like to thank Bob Freed for his allowing me to examine his Z80 code before writing NARC. Bob wrote UNARC for the CP/M world. A special thanks to the following kind folks. Curt Edwards Jerry Taylor Chuck Crumpton Frank Roemer Paul Bowling Dr. Matthew Witten Paul Clarke Sherman Able Carlisle Crutcher H.W. Straley John R. Gernert Dudley Hoffman Dr. Melvin Maron Bob Fergus NARC.DOC Copyright (c) 1987-90 Infinity Design Concepts, Inc. Page 3 ═══════════════ ABOUT ZIP FILES ═══════════════ The ZIP file format was developed as the natural extension of the older LBR and ARC files. This new format allows much greater data integrity than we have ever had before with archive utilities. The 32 bit CRC implemented plus the use of both a central directory and a distributed directory, offers the very best archive integrity. NARC ZIP 3.0 AND ABOVE SUPPORTS BOTH ARC AND ZIP FILES The compression algorithms used in ZIP files are shrinking and reducing. The shrinking algorithm is basically the same as squashing and crunching used in ARC files, except, that the algorithm has been fine tuned to eliminate the wasted bytes in earlier implementations. The code length responsibility has been shifted from the decompressor to the compressor, which again offers an increase in compression ratios. The total string table clearing encountered in ARC files, has been replaced with a much more intelligent partial clear, which, instead of eliminating all strings that have been learned in the table, only parts of strings are eliminated and used again. The reducing algorithm is really a combination of two distinct algorithms, statistical compression and run length encoding (packing), just as crunching was a combination of LZW and packing. The statistical compression method takes advantage of certain statistical character pair occurences common in the English language, but the algorithm also fairs well with other languages and binary files. The ZIP file format was developed by IDC and PKware and several other individuals after much discussion. The LZW algorithm improvements were readily visualized by all parties, but the reducing and imploding algorithms were developed by Phil Katz and PK-friends and has shown themselves to be very powerful, as we have seen files compressed as much as 99%. NARC.DOC Copyright (c) 1987-90 Infinity Design Concepts, Inc. Page 4 ═════════════ COMPATIBILITY ═════════════ NARC is compatible with all known "skrunching" algorithms, that is up to and including Squashing. NARC is compatible with PKarc, PKpak and PKzip. NARC supports both .ARC and .ZIP files and recognizes the .ARK extension soon to be prevalent in the CP/M world via Bob Freed's CP/M archive facility, NOAH. ═════════════════════════════════════════ ARCHIVE STORAGE METHODS SUPPORTED BY NARC ═════════════════════════════════════════ Packing - all versions Squeezing - Huffman Coding Crunching - all versions (LZW encoding) Squashing - one version ═════════════════════════════════════ ZIP STORAGE METHODS SUPPORTED BY NARC ═════════════════════════════════════ Reducing - statistical compression + packing Shrinking - very intelligent LZW algorithm Imploding - Shannon-Fano coding of sliding dictionary length and distance pairs Note: LZW stands for Lempel-Ziv-Welch NARC.DOC Copyright (c) 1987-90 Infinity Design Concepts, Inc. Page 5 OverView... NARC first searches the default path for ZIP, ARC and ARK files and if any are found they are displayed in the ZIP- window on the left side of the screen. The arrow keys (or the mouse), PgUP,PgDN, Home and End may be used to move the cursor bar up and down the window, there are two ways to select the highlighted ZIP file. (1) Hit the ENTER key (2) Press the left mouse button After selecting the sub-file of interest, NARC displays all of the ZIP sub-files and their statistics on the screen. You are also given a menu bar at the bottom of the screen. You may use the arrow keys or the mouse to move the cursor bar to the desired selection and then select with the ENTER key or the left mouse button. As the cursor bar is moved, you are also given a brief description of the highlighted command. A file may be located in the list by the now common ALT-letter key combination. This applies to all menus in NARC. The commands will now be discussed individually. Note: You may also select any option from the command bar by entering the first letter of the command. The ESCape key will abort any operation or window. Video Modes NARC will determine whether a monochrome or color video adapter is being used and act accordingly. If a CGA or EGA card is detected, then NARC will select 80 x 25 color mode, unless color has been turned off with NARCCFG. The users video mode and cursor shape is saved when NARC is invoked and restored upon exit or shell. On EGA/VGA equipped systems, the F2 key may be used to toggle NARC into and out of 25 and 43 line modes. Mouse Support NARC will determine if a mouse driver is present. If so, then NARC's mouse support routines are enabled and working at all times within the program. The left mouse button ALWAYS emulates the ENTER key (for selecting highlighted items) and the right mouse button ALWAYS emulated the ESC key (for aborting operations). Highlight bar movement can always be accomplished with mouse motion. The sensitivity of mouse motion can be changed with NARCCFG. The mouse routines were written with the Microsoft mouse in mind and any mouse that is compatible with the Microsoft mouse should work. NARC.DOC Copyright (c) 1987-90 Infinity Design Concepts, Inc. Page 6 Sound NARC makes use of the 8253 programmable interval timer chip and speaker port to create some gurgles,bleeps,whistles and burps in general, that some folks like and some don't. The chip is programmed for square wave output in mode 3. The sound may be permanently disabled with NARCCFG. NARC determines microprocessor speed and adjusts the sound routines accordingly so that the sounds are pretty much the same on all machines. ENVIRONMENT VARIABLES If you don't know what an environment variable is, then continue reading. If you DO know, then skip to "The Goods" section below. Note that all of the environment information can be configured directly into NARC zip 3.0 and above with NARCCFG. However, this information can be overridden with the environment variables. The environment variables are given precedence so that if you take your NARCCFG'd version to another machine, you can do a quick temporary set up with the environment variables. What are they ? An environment variable is a convenient way for programs to communicate with DOS or with each other through a commonly shared spot(s) in memory. If you type "SET" <ENTER> at the DOS prompt, DOS will respond by showing you the current spot of shared memory which we call the environment. These text strings can be created,read, modified and deleted at any time by any program. Each variable has a certain format that must be adhered to. The format is as follows, variable_name = string An example would be IDCED=C:\UTILITY\WS Spaces are allowed before and after the equal sign. This variable will mean absolutely nothing to DOS, but, to NARC, it will show the way to your editor. The variable name here is IDCED and the string is C:\UTILITY\WS which is the pathname for finding WordStar on the disk. You can change this string to anything you want by simply typing --- SET IDCED= (and type your pathname for your editor). NARC uses several of these environment variables to communicate with you and you should set them up in your AUTOEXEC.BAT file. What do I do if DOS says, "Out of Environment Space" ? DOS reserves only so much space for the environment strings, and if you have a long search path or a lot of strings, then NARC.DOC Copyright (c) 1987-90 Infinity Design Concepts, Inc. Page 7 you may run out of space. I can offer two suggestions, one for DOS 3.1 and one for DOS 3.2. There is a "switch" that you can add to your CONFIG.SYS file (If you don't have one, then you should !). One of the lines in your config.sys file should read Shell=C:\COMMAND.COM or something similar. You can add the following to the end of this line to increase your environment space. DOS 3.1 - /E:nn where nn = 10 up to 62. This number times 16 is the length of your environment. example: Shell=c:\command.com c:\ /P/E:10 This would reserve 160 bytes. DOS 3.2 - add /E:nnnn where nnnn is the actual number of bytes to reserve. example: Shell=c:\command.com c:\ /P /E:160 This would reserve 160 bytes. Environment variables can be set up in your Autoexec.bat file as follows. Set IDCARCPATH=c:\utility\idcshell.exe Set IDCED=c:\wordproc\editor.ext The Goods .. IDCED=c:\editors\ws.exe Use this variable to tell NARC where your editor is located, so that when the F9 key is pressed to invoke the editor, NARC will know where to locate it. The drive and path and editors name must be given (in upper or lower case). The file extension (.COM or .EXE) MUST be given. IDCARCPATH=c:\utility\idcshell.exe Use this variable to tell IDCshell where to find IDCshell when the "I" key is pressed. As with IDCED, the drive and path and filename and extension must be given. NARCHELP=c:\utility\narchelp.hlp Use this variable to tell NARC where to find the NARC help file when the F1 key is pressed. NARC.DOC Copyright (c) 1987-90 Infinity Design Concepts, Inc. Page 8 IDCTEMP=e:\ramdisk Use this variable to tell NARC where it can place temporary work files. If you have a ramdisk set up, then this would be a good use for it since all file operations will be MUCH faster from the ramdisk. THE HELP SCREEN From the files screen, it is always possible to get a context sensitive help screen. Hitting the F1 or F10 key will cause a full screen of help to pop up that has information specific to the command that the cursor bar was pointing to. Once the help window is opened up, it is then possible to get help on any of the commands. The Home key will take you to the help index (as will the "I" key). The window may be put at any time with the ESC key. NOTE: There are several commands that are listed under miscellaneous commands in the INDEX section of the help screens, so be sure to check them out. ════════ COMMANDS ════════ ═══════════════ Extract Command ═══════════════ Selecting this option will cause another prompt to be displayed, asking whether you wish to extract the highlighted file or tagged files. (Files are tagged with the space bar). "Point and shoot" here again as before. The ESC key or the right mouse button will abort the operation. If the disk becomes full, you will be informed and have the option of aborting or continuing. Highlighted File When EXTRACT is selected, you will be asked for a drive/path to extract the file to. If you simply hit ENTER or the left mouse button, the file will be extracted to the default drive/path. You may also enter any valid DOS drive/path. The ESC key or the right mouse button will abort the operation. NOTE: Control-R can be used to recall the last path that files were extracted to. NARC.DOC Copyright (c) 1987-90 Infinity Design Concepts, Inc. Page 9 Tagged Files The Space bar (or F3 key) is used to TAG the current file. When a file is tagged, a diamond will be displayed on the line with the current file in column 80. The space bar will also unTAG a file, thus it is a "toggle". When the space bar is pressed, a diamond will appear as described above and the cursor bar will move to the next file. When the "TAGGED" option is selected from the command line, all files that have been tagged, will be extracted to the SAME drive/path. After the file is extracted, it's date and time are set to those contained in the ZIP file. The file is also checked for size and CRC, if both of these do not match exactly what was contained in the ZIP file header, then an error has occurred and the user is notified. The files will also remain tagged after the extraction. If a zipfile contains stored subdirectory names, each file will be displayed with the INFINITY symbol (sideways 8) next to it. When you wish to extract a file that has this "tag", NARC will ask you how you wish to handle the stored subdirectory names. You have several options.. (A)bort - aborts the extraction (P)rompt - prompt and ask before creating any subdirectories (C)reate - create subdirectories as needed (I)gnore - ignore the stored pathnames If the (C) or (P) options are chosen, then directories that are created, will be created starting with the extract path that you have selected as the top directory. ════════════ View Command ════════════ This option will display the currently highlighted file on the screen. The PgUP, PGDN, Home and End keys, as well as the cursor keys allow movement through the file. The file to be viewed is first extracted to the default drive (or to whereever the IDCTEMP variable points) to a file called NARC.TMP. This file is deleted when the view is ended. The extraction is performed due to the sequential nature of ZIP files, which makes it nearly impossible to perform the page up,down operations on the compressed file itself. If NARC finds that there is not enough disk space or directory space to create NARC.TMP, you will be asked for another drive where the temporary file can be created. NARC.DOC Copyright (c) 1987-90 Infinity Design Concepts, Inc. Page 10 The newest addition to the view command in NARC 2.1 is the text search command. Simply hit "S" and you will be prompted for a string to find. When located, the text will be highlighted on the screen, otherwise a message is issued stating that the text was not found. The search is case insensitive, that is you may enter upper or lower case letters and NARC will find a match. The mouse is also supported now in the view function. ═════════════ Print Command ═════════════ The print option will print the currently highlighted file. After selecting the print option, you will be asked which character pitch you want to print in. Enter the number that you wish (or 0 for the default pitch) and the printer will be set to that pitch. NOTE: The printer strings that come installed with NARC are compatible with EPSON printer strings. If you wish to install NARC with your own strings, see NARCCFG.DOC for complete instructions. After selecting selected the printer pitch, you will be shown three more options for formatting the printout. Use the arrow keys to move left and right. The space bar is used to toggle the options ON or OFF. When you have finished and are ready to print, hit ENTER (or left mouse button). The ESC key or the right mouse button will abort at any time. The options are; Format - YES - This option causes NARC to format the output with page breaks and page numbers. NO - NARC does not format the file. The following two options work independently of the Format option. Strip High - YES - NARC will strip the high bit off each character before it is sent to the printer. Some word processors set this high bit on some characters as a means of text formatting. These characters will print as garbage usually. NO - NARC will not strip the high bit. Strip Control - YES - NARC will strip all control characters from the file before it is printed. This is useful on files that have embedded formatting characters, and you wish to have NARC provide the formatting. NO - NARC will not strip the control chars. NOTE: On all of the following windows, the PG UP, PG DN, HOME and END keys in addition to the cursor keys allow movement through the window. NARC.DOC Copyright (c) 1987-90 Infinity Design Concepts, Inc. Page 11 ════════════════ ZIP-wind Command ════════════════ This option will display the window containing all of the ZIP, ARC or ARK files in the current sub-directory. Move the cursor bar up and down with the mouse or arrow keys and select with the ENTER key or left mouse button. ════════════════ DRV-wind Command ════════════════ This option will pop up a window that contains all of the logical drives that DOS reports to NARC. Select as before and the ZIP-window will be popped up so that you can then choose a ZIP file to examine. ═══════════════ DirTree Command ═══════════════ This command is totally new with release 2.1. The old window approach won a lot of criticism and rightfully so. I had a tree routine that I wrote some time ago, that I just wasn't happy with speed-wise and didn't want to stick that thing in NARC. I did have some nice ideas for a new routine that would be much faster and slicker all around and finally found the time to write it and NARC release 2.1 has it, you may judge for yourself. A visual tree is presented that displays all subdirectories on the disk. Once the tree has been read, NARC will not read it again, unless you have changed drives. From the tree, you can create and delete subdirectories, select a new drive or subdirectory or return with no action at all. The ESC key will abort the tree operations. ════════════ Quit Command ════════════ The exit options have been changed so that now you will be prompted for where you wish to exit the program. You have the choice of exiting to the current directory or the original directory where NARC was invoked from. This can be made totally automatic with the NARCCFG program in the registered version. NARC.DOC Copyright (c) 1987-90 Infinity Design Concepts, Inc. Page 12 ══════════════════ ALTERNATE COMMANDS ══════════════════ The extra commands can be located on the help screens, which are invoked by the F1,F10 or ? keys. F1 - key Invokes the context sensitive help system. F2 - key This key allows toggling between 25 and 43 line modes on EGA/VGA systems. On CGA/MONO systems, it will have no effect. F3 - key This key will tag a subfile and move the cursor bar on to the next subfile. This key also has other functions, and they are also shown on the screen when necessary. F4 - key The F4 key will print an image of the screen less the advertisement and command lines. F5 - key Invokes the NARC-DOS command processor. You may then enter any valid DOS command. When finished, simply hit ENTER by itself and you will be returned to NARC. You may also enter "COMMAND" which will invoke a second copy of COMMAND.COM, if the file COMMAND.COM is in your search path. To return to NARC, you would then type "EXIT". F6 - key This key will tag all of the subfiles in the archive for subsequent extraction. F7 - key This key will invert all of the tags on the subfiles, that is all files that were tagged will become untagged and vice-versa. F9 - key This key will call your editor. Note that you MUST have first set up the environment variable IDCED before this function can be used -OR- in NARC 3.0 and above, you may also set the path to the editor permanently into NARC with NARCCFG in the registered version. See the section on environment variables. F10 - key This key will call up the context sensitive help system. NARC.DOC Copyright (c) 1987-90 Infinity Design Concepts, Inc. Page 13 ALT-F10 - key This sequence displays the trivia screen, where your serial number and other information is located. (F)ind command. Will prompt for a wildcard filename to find in the sub-file list. Any number of characters may be used, for example, you may enter a single character and NARC will find the first file whose name begins with that character. Alternatively, you may enter a complete wildcard specification and NARC will attempt to find a match. (K)ill file command. Will remove the currently highlighted sub-file from the archive. No additional disk space is required for temporary files. PgUP,PgDN,Home and End These keys do what you might expect. These functions work in all windows. (S)et search spec command. When NARC searches the current directory for archive files, it normally searches for ZIP, ARC or ARK files. The .ARK may be changed to another extension (or wildcard extension) with this command. All subsequent searches will use the .ZIP and .ARC extension and the newly defined mask. This command may be used from the ZIP file window or from the subfile screen. The extension may also be passed from the command line. Enter NARC /h for complete information. (I)DCshell command. This command will run IDCshell.EXE. Note that the environment variable IDCARCPATH must be set up before this command will work, unless IDCshell is in the current directory. IDCshell is the companion program to NARC and was be released in Aug - 88. This command provides a simple link between the arcing functions of IDCshell (and its copy / rename / delete/print/view etc...) functions. Note that NARC can also be called from IDCshell via its "N" command along with a filename. Note that in NARC zip 3.0 and above, the path for IDCshell can be permanently configured into NARC with NARCCFG in the registered version instead of using the environment variables. NARC.DOC Copyright (c) 1987-90 Infinity Design Concepts, Inc. Page 14 Command line syntax: NARC [drive:\path\subfilemask] If the drive:path\subfilemask is given, then NARC will display all files that match the given specification in the zipfile window when NARC is first invoked. If no subfilemask is given, then NARC defaults to displaying all ZIP/ARC/ARK files if no subfilemask is not given. If the drive and/or path is given, then NARC will search the given drive or path for files that match the subfilemask. TRIVIA STATUS SCREEN This screen shows some useful information about the current NARC environment. It is invoked with the ALT-F10 command sequence from the subfiles screen. NARC.DOC Copyright (c) 1987-90 Infinity Design Concepts, Inc. Page 15 Operating Hints and Shortcuts When NARC is first invoked, it pops up the ZIP-window showing all of the ZIP/ARC files in the current directory. NOTE: When the ZIP file window is onscreen, you may also choose another file extension beside .ARK with the "S" command. See the section entitled "Using extensions besides .ZIP and .ARC". When there are no windows popped up, the "A" key or the "Z" key will pop up the ZIP file window, regardless of where the command line cursor bar is. This is handy when you have a lot of ZIP's that you want to thumb through, with just the 2 mouse buttons or ENTER and F2 keys, you can look through a whole directory of ZIP files in nothing flat ! Also along these lines, when the ZIP window is on the screen, the right mouse button or the ESCape key will exit the program. NARC buffers 64k of the input file at one time, thus speeding up file operations. The output buffer is 32k and should be larger in the next version. NARC requires about 220K of RAM to operate. (This prime advertising space for rent) NARC.DOC Copyright (c) 1987-90 Infinity Design Concepts, Inc. Page 16 Note that the arrows keys may be used to move the error message box up and down the screen. This feature is provided so that if the message covers up something on the screen that you would like to see, you may move the box. ══════════════════════ Error Messages. ══════════════════════ Memory Allocation Error. - NARC allocates memory when it is invoked, this says that DOS told NARC that there was not enough memory left to run the program ERROR: Extraction Failed due to CRC error, Hit ENTER - After a file is extracted, the CRC contained in the ZIP header is compared to the CRC that NARC calculates, this message says that the two were different. This is the CRC-16 polynomial in ARC/ARK files and CRC-32 in ZIP files. ERROR: Extraction Failed due to FileSize error - Same as above, except with filesize ERROR: Disk File Inconsistency. Hit ENTER - This will usually mean that the user has swapped disks just after telling NARC to View,Print or Extract and NARC does not recognize the file, or it may also be displayed as the result of reading/extracting a damaged ZIP file. ERROR: Incompatible Crunch Format - Says that either the stowage code for this file is not supported by NARC -OR- there is an error in the ZIP header ERROR: Extraction Failed due to Lack of Disk Space - Abort will stop tagged extraction, continue will try to fit next file. Squeezed File Has a Diseased Decode Tree. - When unsqueezing a file, NARC has found a bad entry in the decode table. ERROR: No directory space on destination! - Self explanatory Bad Path Name, Hit ENTER - The destination path that the user entered for extraction is not a valid DOS pathname, re-enter. NARC.DOC Copyright (c) 1987-90 Infinity Design Concepts, Inc. Page 17 Requires DOS version 2.0 or above. - NARC requires DOS 2.0 or above to operate. Invalid archive file format - NARC could not find any ZIP headers, this is probably not a ZIP or ARC file. Warning: Bad archive file header, bytes skipped = xxxxx - If an entry has a bad header, NARC will examine the next 64k bytes looking for a good header. This is to maintain compatibility with self-unpacking ARC files. Unexpected end of archive file - Says that NARC couldn't find the last ZIP header No matching file(s) in archive - ZIP file is empty Cannot create work file, enter drive for temporary file -there was not enough directory space or disk space to create NARC.TMP for viewing. Enter a drive letter where NARC can create the necessary work file. The file will be deleted when the view has ended. NARC.DOC Copyright (c) 1987-90 Infinity Design Concepts, Inc. Page 18 ════════════════════════════════════════════ ARCHIVE FILE FORMATS AND GENERAL INFORMATION ════════════════════════════════════════════ For Those With a Little More Curiosity... The following are the currently supported stowage methods. 1 unpacked (obsolete) 2 unpacked 3 packed 4 squeezed (after packing) 5 crunched (obsolete) 6 crunched (after packing) (obsolete) 7 crunched (after packing, using faster hash algorithm) 8 crunched (after packing, using dynamic LZW variations) 9 Squashed c/o Phil Katz (no packing) (var. on crunching) NOTE: LZW is Lempel-Ziv-Welch compression algorithm A little about the stowage methods. ══════════════════════════════════ Packing (or run length encoding) ══════════════════════════════════ This is the simplest of the storage methods. Suppose that you have a line of text and at the end of the line, you have 40 spaces. These 40 spaces are compressed into 3 bytes in the ZIP file by the "packer". The first byte is the actual character to be expanded (in our case a space). The second byte is a special "flag" byte that indicates that we need to expand these bytes when we extract the file. The third byte is the count byte (in our case it would be 40). So you can see that any time the compressor finds repeated bytes like this, it can compress them into 3 bytes. The anomalous case to watch out for here is when the count byte is the same character as the "flag" byte, this proved to be a difficult roach to kill ! NARC.DOC Copyright (c) 1987-90 Infinity Design Concepts, Inc. Page 19 ═════════ IMPLODING ═════════ The imploding algorithm scans the input file for string matches. As the algorithm proceeds through the input file, it looks "backward" toward the start of the file to see if any character strings can be found to match the string it is currently examining. If a match is found, then the LENGTH of the match, as well as the DISTANCE "backwards" in the file can be sent to the output file, rather than the string itself. For example, if our input file looked like this; "This is the input file. This is the last sentence." We note there are 50 characters in our input file. As the algorithm scans this file, it notices that the phrase "This is the " is repeated, so we need not send this string to the output file, but instead we send the LENGTH and DISTANCE values. When the algorithm is looking at the second "This is the " string, it finds a match at 24 characters backwards in the file (e.g. the first string), this is the DISTANCE value; and the match LENGTH is 12. The output file would now look like this; "This is the input file. LENGTH,DISTANCE "last sentence." The imploding algorithm, however, takes this "sliding dictionary" output (LENGTH,DISTANCE pairs) and further encodes them using a Shannon-Fano tree. Just as the Huffman algorithm translates one character set into another based upon frequency of occurrence, so the imploding algorithm does with the LENGTH,DISTANCE pairs. A table of values is built determined by the frequency of occurrence of the LENGTH,DISTANCE pairs, that is the number of times each LENGTH,DISTANCE pair is used in the encoding process. The table is then sorted by frequency of occurrence and unique variable bit length codes are assigned to the table entries. The net result is that the more frequently used pairs will be assigned the shortest codes (measured in bits). There are up to three tables used in the encoding process, one for LENGTH, one for DISTANCE and one for literal characters. These tables are stored in the zipfile in BCD compressed format. For a complete description of Shannon-Fano coding, see "DATA COMPRESSION", Held, Wiley Pub. NARC.DOC Copyright (c) 1987-90 Infinity Design Concepts, Inc. Page 20 ══════════════════════════ HUFFMAN CODING (SQUEEZING) ══════════════════════════ It does, at first, seem that making a file smaller would be an impossible task. I will make an attempt here to shed a little light on this subject since that is a question that I hear pretty frequently and it is not a two minute discussion question. It does require some thought. To compress a file with the Huffman algorithm, commonly called squeezing, the first thing that must be done is to read the file completely and count the occurrences of each character. That is you count the "A" 's and the "B" 's and so forth. There are 256 characters in the extended ASCII character set, of which approximately 90 are "printable", that is you can see them on the screen. The IBM set has more "printables", but that is of no consequence, since the squeezer deals only with the numbers and doesn't care whether or not the file is an ASCII text file or an EXE file. Once the squeezer has counted the occurrences of each character, thus the frequency of occurrence, it scans the table for the characters that appear the least number of times and forms an imaginary link between them, called a node. Somewhere else in the tree, we will later develop a pointer that points to this node. When you start putting all of these things together, you will form a binary tree in memory. Confused enough ? Let us try an example. We have a file that is 100 bytes long and has 6 different characters in it. We have counted the occurrence of each of the characters and found the following. quantity character 5 - D 10 - A 10 - F 20 - B 25 - E 30 - C The spelling in the file wasn't very good, but we don't care. Now we take these numbers and will call them the frequency of each character. We then arrange the table as below. This is an arbitrary arrangement, but it is useful here so as to make our tree readable on the screen. The arrangement makes no difference. NARC.DOC Copyright (c) 1987-90 Infinity Design Concepts, Inc. Page 21 Frequency 20 10 5 10 30 25 Character B A D F C E We then examine the table to find the two characters with the smallest frequency of occurrence. In our case, it is obvious that one of them is 5,but which 10 do we choose. As it turns out, it doesn't matter which one you choose, we will arbitrarily choose the F. We draw lines from the D and the F to form our node (the box below). Frequency 30 10 5 10 20 25 Character C A D F B E \ / \ / ╔══╗ ║15║ = 5 + 10 ╚══╝ The number in the box is the sum of the frequencies of the D and F characters. Now we again look for the lowest two frequencies, except, this time we do not consider the D and F characters individually, we instead consider the node. The lowest two now are the A and the node, that is 10 and 15. We again do some artwork. Frequency 30 10 5 10 20 25 Character C A D F B E \ \ / \ \ / \ ╔══╗ \ ║15║ = 5 + 10 \ ╚══╝ \ / \ / ╔══╗ ║25║ = 10 + 15 ╚══╝ We look at the table again for the next two lowest frequencies and now find B and E. We continue in this fashion until the entire "tree" is built, that is until it all condenses to one node. The leaves are the actual characters at the top of the tree and the nodes represent branch joints with the root at the bottom. NARC.DOC Copyright (c) 1987-90 Infinity Design Concepts, Inc. Page 22 Frequency 30 10 5 10 20 25 Character C A D F B E \ \ \ / \ / \ \ \ / \ / \ \ ╔══╗ ╔══╗ \ \ ║15║ ║45║ \ \ ╚══╝ ╚══╝ \ \ / / \ \ / / \ ╔══╗ / \ ║25║ / \ ╚══╝ / \ / / \ / / ╔══╗ / ║55║ / ╚══╝ / \ / \ / ╔════╗ ║ROOT║ ╚════╝ Now that our tree is made up, we can encode the file. We start at the root (always). To encode the first character (leaf) of the tree (the letter C), we trace up the tree until we hit the letter C at the top. Along our journey, if we make a left turn, we record a 0 bit, and a 1 for a right turn. So for the C, we would go left to 55 (and record a 0) and then left again to the letter C (and record another 0),so the Huffman code for our letter C is 00. For A we go left to 55, right to 25 and left to A and it becomes 010. By doing all of the letters this way, we find the following. C = 00 ( 2 bits ) A = 010 ( 3 bits ) D = 0110 ( 4 bits ) F = 0111 ( 4 bits ) B = 10 ( 2 bits ) E = 11 ( 2 bits ) Mind that the zeroes and ones above are bits and not bytes. Each character was represented in the original file by 8 bits (one byte) and since we have reduced the number of bits needed to represent each character, we therefore reduce the size of the file. The savings add up as follows, NARC.DOC Copyright (c) 1987-90 Infinity Design Concepts, Inc. Page 23 character frequency original bits squeezed bits savings C 30 30 x 8 = 240 30 x 2 = 60 180 A 10 10 x 8 = 80 10 x 3 = 30 50 D 5 5 x 8 = 40 5 x 4 = 20 20 F 10 10 x 8 = 80 10 x 4 = 40 40 B 20 20 x 8 = 160 20 x 2 = 40 120 E 25 25 x 8 = 200 25 x 2 = 50 150 ══════════ ══════ ═════ ═════ Totals 100 800 240 560 │ │ original file size ──────┘ │ squeezed file size ───────────────────────┘ 240 is 30% of 800, so we have compressed this file by 70%. Golly Wally, that seems pretty good. The rub lies in the fact that in order to reconstruct the original file, we must have access to the decode tree and since each tree will be different for each file, we must therefore save the tree with the file. It turns out that the tree can have only 256 nodes in a bytewise compression technique and each node will hold 4 bytes as pointers,a full table will be about 1k long. The table in our example has 5 nodes plus the 6 leaf nodes (where our characters are), totaling 11. 4 times 11 is 44 and if we add a few bytes for storing the node count and some other statistics, our table is about 50 bytes long. If we look at the 240 in the above table this gives the total number of bits that it will take to encode the file, divide 240 by 8 to get the number of bytes (30) and add it to our 50, we get a compressed file size of 80 bytes. Since our original file was 100 bytes, we have achieved a 20% reduction in file size. Not bad. What we have really accomplished is a translation of character sets, with our new set requiring less space than the original ASCII set. How far can we go ? If we look at the maximums that we can obtain for the different bit combinations in a optimally skewed tree, that is a tree that is not exactly symmetrical, we find that we can have only 4 - 2 bit codes, 8 - 3 bit codes, 16 - 4 bit codes, 32 - 5 bit codes, 64 - 6 bit codes, 128 - 7 bit codes, the remaining 4 will be 8 bit codes. 2 - 1 bit codes 4 - 2 bit codes 8 - 3 bit codes 16 - 4 bit codes 32 - 5 bit codes 64 - 6 bit codes 128 - 7 bit codes -------- 254 NARC.DOC Copyright (c) 1987-90 Infinity Design Concepts, Inc. Page 24 And since we have a total of 256 different bytes to encode, the remaining 2 characters must have 8 bit codes. If we add the number of bits that this represents,we find a total of 1554 bits or 195 bytes. So at maximum, we have compressed the 256 bytes to 195 or 33%, thus the idealistic maximum that can be achieved with the Huffman algorithm is 33% when using a byte level implementation. One final note; The Huffman scheme requires the input file to be read twice, once to count characters and frequencies and then again to do the actual encoding. The major differences in Huffman coding and crunching lie in the fact that crunching is a one pass operation and does not require the table to be stored with the file. Both, however, are extremely vulnerable to errors, for example, imagine what would happen if you skipped one bit when squeezing the file, all of the remaining characters in the file would become the proverbial garbage, since we are looking at the file on a bit level. NARC uses the method described in K. & R. pp. 130 for setting up the binary tree with several modifications. The simple binary tree is acceptable for this, since the tree never grows and therefore will never become unbalanced. If you followed that, now go back and read the second paragraph again, maybe it will make sense this time. ═════════════════════════════ CRUNCHING/SQUASHING/SHRINKING ═════════════════════════════ Crunching began with an article by J. Ziv and A. Lempel in IEEE Trans. Information Theory, May 1977, where the method was originally described. Terry A. Welch wrote a definitive application article in IEEE Computer, June 1984 which described in detail how to apply the algorithm and some common problems encountered. Thus the name LZW compression. Crunching takes the Huffman coding method a step further as it does not include a table with the crunched file. The crunching algorithm also "learns" as it proceeds through the file. If it finds repeated strings in the file, they will be encoded into a table. This table is set up (in NARC's implementation) as a 4096 by 3 table. Each entry is formatted as <PREF>,<SUFFIX>, where PREF is a 2 byte pointer to another entry. SUFFIX is the byte for this entry. Representing the PREF's as pointers rather than values speeds up most operations in NARC. This idea came from Bob Freed and is very trick. NARC.DOC Copyright (c) 1987-90 Infinity Design Concepts, Inc. Page 25 One obvious benefit of crunched files is the fact that there is no need to include the encoding table in the compressed file as was the case with squeezing. Another great benefit is the fact that crunching is a one pass operation as opposed to the two pass situation in squeezed files. Crunching begins by creating an "atomic" table, that is a table in RAM that contains 256 entries, one for each character in the extended ASCII set. The values range sequentially from 0 to 255. The table entries are arranged as follows. Prefix Pointer (2 bytes) and Suffix byte (1 byte) In this initial table setup, the Suffix bytes are the 0 through 255 mentioned above. These are the "atomic" characters, in that they must be in the table before crunching or uncrunching can begin, since all files contain some portion of this character set. We do not know which characters will be included in any given file and which ones will be excluded,so we must include them all in our initial table. Once this table is set up, we can begin crunching. The Prefix pointer will contain a value that is a pointer to another string. When the table is initially set up, there are no other strings, so this Prefix pointer is set to a special "Null" string, that is it points nowhere. We must be careful when crunching the file, to look for this special pointer and act accordingly when we encounter it. This Prefix and Suffix business is used to "build" long strings. If we read the input file and found that the first character was the letter "I", we would look this letter up in the string table by hashing (computing an address). If we found the letter in the table (which we certainly will on the first character), then we output it's "hashed" address as a code to the output file (the crunched file). Suppose then, that the next character input from the file was the letter "D", the cruncher would then look at the I and the D together to see if they exist as a string in the table. Well of course, since this is the second character of the file, we know that it doesn't, so the cruncher forms a new entry in the string table. This entry has as its' Prefix pointer, a value that "points" to the letter "I" entry in the table, that we made a minute ago. The suffix byte in this case would be the letter "D". Now another code is output to the crunched file, representing the letter "D". Well this is great, we have now turned 2 bytes from the input file (16 bits) into 3 bytes in the output file (24 bits). You are correct, crunching begins by "not crunching" , but it is a crazy world ! The real value becomes apparent when we run into this same sequence "ID" in the input file again. Now we will find NARC.DOC Copyright (c) 1987-90 Infinity Design Concepts, Inc. Page 26 an entry for it in the string table and we can output a single 12 bit code that stands for "ID", thus saving 4 bits. The cruncher continues "learning" strings like this until the file is crunched. It should be noted that the string table is dynamically changing as the input file is processed. The early versions of crunching implemented, stopped "learning" once the string table was full. This gave a very poor compression ratio in some files. Versions 8 and 9 have an additional feature called adaptive reset, where the string table is cleared and crunching begins all over again ! This capability really helps the larger files more than smaller files. Details of Storage Versions NARC supports all of the current "skrunching" algorithms. A brief description of each follows. Version 1 - "STORED" File is simply stored (obsolete now, 25 byte header) NOTE: Files stored with this version are rare. Version 2 - "STORED" Current version of simple storage. This version was implemented with the implementation of file compression. The main difference in version 1 and 2 is the ZIP header (see header section below), version 1 has a header length 4 bytes smaller than any of the rest of the storage methods since in version 1 there was no need to store the original file length separately from the stored file length since they were the same. Version 3 - "PACKED" Repeated bytes are packed into a three byte string (see Packing above) Version 4 - "SQUEEZED" after packing. The file is first packed as described above and then squeezed Version 5 - "CRUNCHED" This is the first implementation of LZW released. Uses fixed length codes and a complex hashing function. (obsolete now) (See hashing below) NARC.DOC Copyright (c) 1987-90 Infinity Design Concepts, Inc. Page 27 NOTE: Files compressed with this version are hard to find. Version was released only one month when next version came out. Version 6 - "CRUNCHED" after packing. The file is first packed and then crunched. Uses fixed length codes and the same hashing function as version 5. Version 7 - "CRUNCHED" after packing. Same as version 6 except a faster hashing function is used. Version 8 - "CRUNCHED" after packing. Uses variable length codes in the crunched file (9 to 12 bits) and has a faster hash function (actually not hashing at all, but for the sake of consistency, we will call it that). This version also resets the string table when it becomes full which benefits the compression ratio of larger files. This resetting is commonly called an adaptive reset. Version 9 - "SQUASHED" (variation on crunching scheme). This version uses the same hashing function as version 8 but varies the crunching codes from 9 to 13 bits. There is also no packing, which affords the string table the opportunity to "learn" longer codes and thus improve the compression ratio (benefits "real large" files). NARC.DOC Copyright (c) 1987-90 Infinity Design Concepts, Inc. Page 28 ARC file header structure is same for both DOS and CP/M Byte number Value(s) Meaning ══════════════════════════════════════════════════════════════ 1 1Ah Header Flag 2 0-9 Compression Version 3-15 --- ASCIIZ compressed filename 16-19 --- Compressed file size in bytes Low Word, High Word with each word in LoHi format 20-21 bits DOS date format 15-9 Year 8-5 Month 4-0 Day (all zeroes means no date) 22-23 bits DOS time format 15-11 Hours (military) 10-5 Minutes 4-0 Seconds 24-25 --- CRC-16 in LoHi format of uncompressed file. ------- This is important. 26-29 --- Original uncompressed file size NOTE: Version 1 files are not compressed so the length can be found with bytes 16-19, also the header len for version 1 files is 25 bytes. NARC.DOC Copyright (c) 1987-90 Infinity Design Concepts, Inc. Page 29 Hashing..... Hashing is simply an arithmetic way of coming up with an address in a table. You have a set of input numbers (codes) that will map one-to-one with the output codes in an ideal situation. That is, each time you input a certain number, you can rest assured that the output will always return the same output number. This is not quite the case in the current versions of .ARC files. The reason is that the algorithm would require a MEG or so of RAM for implementation. The utilization of a smaller string table in all of the ARC programs introduces the possibility of producing the same output number for 2 or more input numbers. This is called a hash collision. This is handled separately in .ARC files with what is called a "collision table", which helps to locate the correct table entry. There are three versions of "hashing" used in ARC files. Hash1 - Key = upper 12 bits of lower 18 bits of unsigned square of (prefix code + suffix byte) OR 800h Used in stowage versions 5 and 6 Hash2 - Key = lower 12 bits of unsigned product of (prefix code + suffix byte) * 15073 Used in stowage version 7 Hash3 - Key = next available address in table. Used in stowage versions 8 and 9 NARC.DOC Copyright (c) 1987-90 Infinity Design Concepts, Inc. Page 30 CRC calculations - ARC files NARC does not use the traditional table lookup method for calculating the CRC of the file. The table approach requires the table to be in RAM and takes up more space. NARC calculates the CRC on the fly, which requires no table and saves space. The algorithm is taken from the definitive article by Aram Perez in IEEE Micro, June '83. The polynomial is X^16 + X^15 + X^2 + X^1 which is not compatible with the Xmodem CRC. - ZIP files The CRC calculated in ZIP files is a 32 bit CRC value, which offers much greater error detection capabilties. NARC builds a table and uses a table lookup method of calculating the CRC values. Note that when NARC displays the CRC-32 used in ZIP files, only the lower two bytes are displayed, but the full 32 bits are used in all crc calculations. End of NARC.DOC Copyright (c) 1987-90 Infinity Design Concepts, Inc. Page 31