Black Box 4

home *** CD-ROM | disk | FTP | other *** search

/ Black Box 4 / BlackBox.cdr / w3_prog / stream11.arj / STREAMS.DOC < prev next >

Wrap

Text File | 1992-03-31 | 20.3 KB | 590 lines

STREAMS - TP 6.0/TPW 1.0 unit to supplement TurboVision/ObjectWindows streams Copyright D.J. Murdoch (1992). DESCRIPTION "Wierd Stream Tricks" might be a good name for this unit. It contains various doodads that I've written in the past, and a few new things, all on a theme of adding functionality to the streams in Borland's Turbo Pascal libraries TurboVision and ObjectWindows. LICENSE This unit is *not* public domain code. You may use it for non-profit purposes at no charge if credit is granted to me, but written permission must be obtained from me for use of this unit in commercial products. This is the first release of the STREAMS unit. There are probably bugs; I would really appreciate reports of any, or other suggestions for improvement. Please send either one to me at one of the following addresses: dmurdoch@watstat.waterloo.edu (Internet) 71631,122 (Compuserve) DJ Murdoch at Fidonet node 1:221/177.40 D. J. Murdoch 79 John St. W. Waterloo, Ontario, Canada N2L 1B7 SUMMARY Hierarchy TStream (from Objects) TFilter Base type for filters TEncryptFilter Encrypts as it writes; decrypts as it reads TLZWFilter Compresses as it writes; expands as it reads TTextFilter Provides text file interface to stream TLogFilter Provides logging of text file activity TRAMStream Stream in memory TDOSStream (from Objects) TBufStream (from Objects) TNamedBufStream Buffered file stream that knows its name TTempBufStream Buffered file stream that erases itself when done Procedures & functions: TempStream allocates a temporary stream OvrInitStream like OvrInitEMS, but buffers overlays on a stream May be called several times to buffer different segments on different streams. OvrDetachStream detaches stream from overlay system OvrDisposeStreams detaches all streams from overlay system and disposes of them OvrSizeNeeded Calculates the size needed to load the rest of the segments to a stream OvrLoadAll immediately copies as many overlay segments to the stream as will fit CONTENTS 1. TFilter a generic object to act as a filter to a stream 2. TEncryptFilter a filter which does simple encryption/decryption 3. TLZWFilter a filter which does LZW compression/decompression 4. TTextFilter a filter to provide the Read/ReadLn/Write/WriteLn interface to a stream 5. TLogFilter a filter to allow logging of activity on text files 6. TRAMStream a stream which resides entirely on the heap 7. TNamedBufStream a buffered file stream which knows its own name 8. TTempBufStream a temporary buffered file stream, which deletes itself when done 9. TempStream a procedure to allocate a temporary stream, in RAM, EMS, or on disk, according to a specified preference 10. OvrInitStream and related procedures procedures to allow overlays to be buffered on any stream or combination of streams 11. Miscellaneous constants and types DEMO PROGRAMS Encrypt.pas - encryption program, using TEncryptFilter Compress.pas - file compressor, using TLZWFilter Logdemo.pas - simple demo of TLogFilter Ovrdemo.pas - simple demo of multiple overlay files ovr1.pas - one overlaid unit ovr2.pas - a second overlaid unit 1. TFilter = object(TStream) "Filters" are programs which take a file as input and produce a new file as output. TFilter is an adaptation of this idea to streams. Every TFilter has a base stream, which can be any kind of stream. Every TFilter is itself a stream, which means you can read and write to it. When you do, it just relays your request to the base stream. One use of this is in the fact that you can make descendants of TFilter which massage the data on the way to and from the Base. Just override the Read and Write (and possibly other) methods, and you can make changes to the data flowing to or from *any* kind of stream. No need for special coding for an TEMSStream, or for a TDOSStream, or whatever. Your code can act on any of them. Examples of things to do are in the Streams unit: encryption (TEncryptFilter) and LZW compression (TLZWFilter). The other main use is to add other kinds of functionality to a stream. You can't use the formatting capabilities of ReadLn and WriteLn with standard streams, but if you use a TTextFilter (see below) you can. FIELDS Base : PStream; TFilter.Base holds the pointer to the base stream. StartOfs : longint; TFilter.StartOfs holds the offset in the base stream of offset 0 in the filter. This allows multiple filters to work on the same stream; one could work at offset 0, another at offset 1000, etc. Just position the base to the desired starting offset before initializing the filter. METHODS Constructor Init(ABase: PStream); TFilter.Init sets Base to point to ABase^, and sets StartOfs to the current position of the base stream. Destructor Done; virtual; If Base is not nil, then TFilter.Done disposes of the base stream before calling TStream.Done for itself. Function CheckStatus : boolean; virtual; Returns true if status is stOK. If it is, but the base status is not stOK, then it assumes that someone has called a Reset for the filter, so it calls Reset for the base stream. Borland should have made Reset virtual, and this kludge wouldn't have been necessary! Procedure CheckBase; Checks the base stream for an error. If Base^.status is not stOK, then it calls Error with Code=stBaseError and Info=Base^.status. Function GetPos : longint; virtual; Function GetSize : longint; virtual; Procedure Read(var buf; count : word); virtual; Procedure Seek(pos: longint); virtual; Procedure Truncate; virtual; Procedure Write(var buf; count: word); virtual; These methods all call the corresponding method in the base stream. Offsets are translated using StartOfs. Before the call, CheckStatus is called to propagate any Reset to the base; after the call, CheckBase is called to propagate any errors to the filter from the base. 2. TEncryptFilter = object(TFilter) This is a filter which does simple encryption/decryption on the base stream. The encryption method is to XOR each byte with a Random(256) value; the starting RandSeed is the key for the encryption. FIELD key : longint; The key is used as a Randseed replacement value. Randseed itself is left unmodified by Read and Write. METHODS Constructor Init(Akey:longint; ABase:PStream); The value AKey is used as the starting key for byte 0 of the filter. Anything read from ABase is decrypted using this key; anything written to it is encrypted. Procedure Read(var buf; count : word); virtual; Procedure Seek(pos : longint); virtual; Procedure Write(var buf; count: word); virtual; These methods all encrypt/decrypt and update the value of Key to correspond to the new position in the stream. The encryption method uses the same algorithm for encryption as for decryption. TEncryptFilter.Seek is fairly slow, because it updates the Key once for every byte difference between the current position and the new position. Moving backwards is slower, because the update algorithm is written in Pascal, not assembler as is the forward algorithm. 3. TLZWFilter = object(TFilter) This is a filter which does LZW compression as it writes to and decompression as it reads from the base stream. The LZW code is an adaptation of Wilbert van Leijen's implementation of the 12 bit LZW algorithm. It's quite fast, though not as fast as Wilbert's version; it gets slowed down a lot by having to save its state after every Read and Write operation. You're best off writing large chunks to it to take advantage of Wilbert's excellent code. (Historical note: Trying to rewrite this was what inspired me to write FULLDB, a program which allows source level debugging of external .ASM files. -djm) Each TLZWFilter takes over 28700 bytes of memory, because it keeps extensive tables to record the current state. One limitation of the TLZWFilter is that it can only be opened in read or write mode, and Seek cannot be used in write mode. FIELDS Mode : word; One of stOpenRead or stOpenWrite; the mode under which this filter was opened. Size : longint; The size of the expanded stream. This is the size that users of the filter will see; if compression is working, it will generally be bigger than Base^.GetSize. Position : longint; This is the position in the expanded stream. Tables : PLZWTables; This is a pointer to the tables used by the compression engine. They're automatically allocated on the heap. METHODS Constructor Init(ABase:PStream; AMode:TOpenMode); Allocates Tables from the heap, and opens the compressor in a mode of either stOpenRead or stOpenWrite. If reading, a signature and record of the uncompressed filesize is read from the base to confirm that it is compressed by LZW, and to prime the Tables. If writing, the signature is written to the stream. Destructor Done; virtual; Flushes all data to the stream, and writes the uncompressed filesize to the head of it before calling TFilter.done. Procedure Flush; virtual; Function GetPos: longint; virtual; Function GetSize:longint; virtual; Procedure Read(var buf; count:word); virtual; Procedure Seek(pos:longint); virtual; Procedure Truncate; virtual; Procedure Write(var buf; count: word); virtual; These methods all override the basic filter methods and compress/decompress the data. They check whether the operation requested can be performed in the current mode, and call Error with Code=stBadMode and Info=Mode if the operation is not supported. Seek is not supported at all in Write mode. In Read mode, it is slow for seeking forwards, and very slow for seeking backwards: it rewinds the file to the start and seeks forward from there by expanding everything. Truncate is not supported in either mode, and always causes a call to Error. Flush may only be called once; it changes Mode to 0, so any further operations will fail. 4. TTextFilter = object(TFilter) This is a filter which provides a Read/ReadLn/Write/WriteLn interface to a stream through its Textfile field. Once the filter is initialized, Textfile acts exactly like a text file in the standard I/O procedures, but reads come from the stream, and writes go to it. FIELD Textfile : Text; This is the dummy text file to use in Read/ReadLn/Write/WriteLn. METHODS Constructor Init(ABase:PStream; AName:String); Initializes the filter with base ABase. The name AName is stored as the name in the Textfile variable. Destructor Done; virtual; Closes the TextFile variable to flush any remaining data to the stream. 5. TLogFilter = object(TFilter) This filter allows logging of activity on text files to a stream. Logging is done very transparently. Once the TTextFilter is initialized, you call the Log method to start logging a particular text file, and the UnLog method to stop. When logging is in effect, any data read from or written to the text file is copied to the stream. Multiple files may be logged to the same stream. For example, you can log both Input and Output, and keep a record of an interactive session on the stream. It's also possible to log the same file to multiple streams. Just create the different TLogFilter objects, and call their Log methods with the same file as an argument. If you then call Unlog, you must do it in the *reverse* order to the order you called Log, e.g. S1^.log(output); S2^.log(output); S2^.unlog(output); S1^.unlog(output); is the correct order to log and unlog. One detail of the implementation may cause some trouble. The data is logged to the stream at the time it is written to disk by the text file. Since text files are buffered, this may not be the time at which you write to the text file, and multiple text files logged to the same stream may not have all data appearing in the correct order. FIELD LogList : ^Text; This is a pointer to the first text file currently being logged. METHODS Destructor done; virtual; Stops logging all text files by calling UnLog for each, and closes and disposes of the base stream. Procedure Log(var F:text); Starts logging the text file F. Continues until the UnLog method is called or the file is closed or Assign'd. Function UnLog(var F:text):boolean; Stops logging the text file F. Returns true if successful, false if not. Will fail if an Assign has been done to F, or F has been closed, or F has already been Unlogged, or another stream has started logging F and hasn't been UnLog'd yet. 6. TRAMStream = object(TStream) A stream which resides entirely on the heap. The maximum length is 65520 bytes. FIELDS cp : word; The current pointer for the stream. size : word; The current size of the stream. alloc : word; The size of the allocated block of memory. buffer : Pbyte_array; A pointer to the block of memory holding the stream data. METHODS Constructor init(Asize : word); Attempt to initialize the stream to a block size of Asize; initial stream size and position are 0. Destructor done; virtual; Dispose of the stream. Function getpos : longint; virtual; Function getsize : longint; virtual; Procedure read(var buf; count : word); virtual; Procedure seek(pos: longint); virtual; Procedure truncate; virtual; Procedure write(var buf; count: word); virtual; Implement the basic stream functions. 7. TNamedBufStream = object(TBufStream) A simple descendant of TBufStream which knows its own name. FIELD filename : PString { PChar in TPW }; The name of the stream. METHODS Constructor Init(name:FNameStr;mode:TOpenMode;abufsize:word); Open the file with the given name, and save the name. Destructor Done; virtual; Close the file. 8. TTempBufStream = object(TNamedBufStream) A temporary buffered file stream, which deletes itself when done. METHODS Constructor init(abufsize:word); Create a temporary file with a unique name, in the directory pointed to by the environment varable TEMP or in the current directory, and open it in read/write mode. Destructor done; virtual; Close and delete the temporary file. 9. Function TempStream(InitSize,MaxSize : longint; Preference:TStreamRanking):PStream; This procedure returns a pointer to a temporary stream from a choice of 3, specified in the Preference array. The first stream type listed in the Preference array which can be successfully created with the given sizes will be returned, or Nil if none can be made. ARGUMENTS Initsize : longint; The initial size to allocate to the stream. MaxSize : longint; The maximum size to which the stream should be allowed to grow. Preference : TStreamRanking; An array of 3 entries specifying what sort of temporary stream is desired. Supplied constants include: ForSpeed = (RAMStream, EMSStream, FileStream); ForSize = (FileStream,EMSStream, RAMStream); ForSizeInMem = (EMSStream, RAMStream, NoStream); ForOverlays = (EMSStream, FileStream,NoStream); 10. Stream overlay procedures These procedures allow overlays to be buffered on any stream or combination of streams. Some overlays can be loaded into EMS, others kept on disk, and others can be put onto any other available stream. PROCEDURES/FUNCTIONS Procedure OvrInitStream(S:PStream); Copies overlay segment code to S as new segments are loaded, and does reloads from there. You may call OvrInitStream multiple times, and different segments will be buffered on different streams, depending on the order in which they are loaded by the overlay loader. On the first call, an exit handler is installed which will call OvrDisposeStreams upon program termination. Procedure OvrDetachStream(BadS:PStream); Makes sure that the overlay system makes no references to BadS. Call this before disposing of a stream which has been passed to OvrInitStream, or you're very likely to crash. Procedure OvrDisposeStreams; Detaches and disposes of all streams being used by the overlay system. Function OvrSizeNeeded:longint; Returns the additional size required to load any segments which still haven't been loaded to a stream. Function OvrLoadAll:boolean; Forces all overlay segments to be copied into the stream; if successful (true) then no more references to the overlay file will be made. Warning: This function calls OvrClearBuf, so that any overlay files which are already in the regular overlay buffer will need to be reloaded. 11. Miscellaneous constants and types CONSTANTS stBadMode = 1; Error signalled when an operation is not permitted in the current mode. stStreamFail = 2; Error signalled when a stream Init failed. stBaseError = 3; Error signalled by a TFilter when the base stream has an error; the base stream's error number is put in the Info field. stMemError = 4; Not enough memory for operation. stSigError = 5; Problem with LZW file signature. BufSize : word = 2048; Buffer size to use when creating a buffered file stream in TempStream. TYPES TOpenMode = $3C00..$3DFF; This is the widest possible range of open modes for a TDOSStream descendant. Values outside this range can cause very serious bugs in programs, since the high byte is used as the DOS service number when the file is opened. PLZWTables = ^TLZWTables; TLZWTables = record ... These tables are used internally to maintain the state of a TLZWFilter. PByte_Array = ^TByte_Array; TByte_Array = array[0..65520] of byte; An array type used as a buffer in several places. TStreamType = (NoStream, RAMStream, EMSStream, FileStream); The types of streams that TempStream can create. TStreamRanking = array[1..NumTypes] of TStreamType; An array giving the order from most preferred to least preferred for a temporary stream.