The Datafile PD-CD 4

home *** CD-ROM | disk | FTP | other *** search

/ The Datafile PD-CD 4 / DATAFILE_PDCD4.iso / utilities / utilsc / computils / !CompUtils / Resources / Expand / Docs next >

Wrap

Text File | 1996-03-30 | 28.6 KB | 624 lines

############################################################################ # # # Expand utility # # # #==========================================================================# # # # by David Radford # # # #==========================================================================# # # # These files are a part of !CompUtils and may not be distributed # # separately other than as laid down in the !ReadMe file. # # # ############################################################################ Overview ======== Expand is a general purpose sample decompressor. There is one piece of code for each group of sample types supported, plus a 'universal' version that incorporates all of these. Each version has an identical user interface, meaning that any program you write can quickly take advantage of new compression formats provided in future releases of !CompUtils with little or no changes needed to your code. Input/Output ============ The Expand utility passes data to and from it using a pair of buffers. These must be provided for it by your external program, from hereon referred to as the master. Data transfers are always controlled by Expand, not the master program; Expand *requests* new blocks of compressed data from the master, rather than having them forced upon it. Similarly, Expand *requests* the master to dispose of data in the output (destination) buffer when it becomes full. Expand simply treats the input and output buffers as blocks of data. Whether or not you reuse these areas is up to you (thus making them buffers). If you prefer, you could load the source sample into one or more blocks of memory then pass these one at a time to Expand. Obviously these blocks could no longer be described as buffers. For consistency the term 'buffer' will be used throughout this document since most uses of Expand will require one or the other of these blocks to be a buffer. The Expand header ================= The code has a standard header through which all parameters are passed: +0 branch instruction to machine code routine +4 filetype for compressed samples +8 filetype for sequence files +12 pointer to the source buffer (user provided) +16 length of source buffer (user provided) +20 pointer to the destination buffer (user provided) +24 length of destination buffer (user provided) +28 number of bytes written into destination buffer +32 returned reason code +36 output flags (user provided) +40 length of final sample +44 number of remaining bytes to be written to output +48 sample period for output +56 file format of source data (eg. 6 for Type 6 samples) Only those entries listed as 'user provided' may be modified; all others are read-only and provided for information. The source file is decompressed by setting the value at +32 to zero, then repeatedly calling the location +0 and examining the reason code returned at +32 to see what action needs to be taken. Typical actions are refilling the source buffer or outputting the contents of the destination buffer. The master program (ie. the one calling the decompression routine) must respond to the reason code returned in the way outlined below. There is no need to call the machine code again immediately, and it may be used quite happily by repeated calls from the polling loop of a multitasking application. This is how !Compress works. If you want to do this, then make sure the buffers are fairly small ie. 16K or so to avoid soaking up too much processing time. If your program decides to abort the operation for some reason, you MUST ensure that the reason code is at some stage reset to zero before calling the machine code routine again (otherwise it will attempt to carry on where it left off, with disastrous consequences). Note that all source buffers passed, except the last one, must be completely full. Another way of looking at this is to say that the word at +16 gives the amount of data actually stored in the source buffer, or that +12 and +16 describe a block of data to be decompressed. Destination buffers will always be completely full when returned to the master, with the exception of the very last buffer which may only be partially full (though never completely empty). When starting a new operation, there is no need to provide a source buffer until Expand actually asks for one by means of reason code 1. Equally, there is no need to provide a destination buffer until initialisation is complete (which you will be informed of via reason code 3). The code - Technical details ============================ (Basic and C programmers can ignore all this information.) On entry, R14 contains the return address and R13 must point to a full, descending stack with space for at least 32 registers. On exit, registers R0-R12 and R14 will have been corrupted, and the value at +32 will give a reason code as detailed below. The flags will be corrupted (except for mode and interrupt), but no SWIs are called so it will operate in any environment (including machines without a proper operating system). A copy of the reason code will be returned in R0 to make life easier for you. If you intend to use Expand on an Arm6-based machine (or later) other than an Acorn computer then you should ensure you are using the 26-bit programmer's model rather than the 32-bit one, since Expand assumes the flags and PC are combined into register R15. There is a potential problem when Expand is being used in IRQ mode (or FIQ mode for that matter) with interrupts enabled, because R14_irq can suddenly become corrupt. Since R14 is extensively used as a subroutine link register this can be fairly annoying (and that's putting it mildly!). The solution is to change the ARM to either to SVC or USR mode before calling Expand. Obviously this problem only applies when trying to use Expand from interrupt code, and will not affect programs written in Basic or C. Reason codes ============ Below is a description of the meanings of the various reason codes that may be returned from Expand, together with information on what action should be taken. In general, reason codes 0-3 are normal operations, 4-7 are 'soft' errors (from which a sufficiently capable master program can recover without having to abort the operation), and 8 upwards are 'hard' errors (which must always cause the operation to abort). 0 The operation has now finished. There is *no* data in the destination buffer, even though the value at +28 (the amount of data in the buffer) may be non-zero. In fact, the value at +28 is a copy of the value last returned with reason code 2 *unless* that buffer was completely full in which case +28 is zero. There is a very good reason for this, though it may not be clear. If your program is designed so that it only saves the destination buffer on reason code 2 *if* the buffer is completely full, then, on reason code 0, the amount of data still to be written is given in +28, and the data in the buffer is undamaged. A use for this may seem a little obscure, until you consider that the size of the sound system's DMA buffers has to be fixed in advance, and decompressing several samples in sequence would pose a problem. After this reason code no more calls to Expand need to be made. Since +32 is now set to zero, any future calls would start a *new* operation. 1 Source buffer request. Expand returns this reason code when it wants the master to provide it with a new block of source data. You should transfer some data from the source file to a convenient block of memory, set +12 to point to it and +16 to give the length of it, then call Expand again. Expand treats +12 and +16 as read-only fields from its point of view, so you don't have to keep resetting these if you are reusing the same block of memory. Note that the last buffer does not have to be completely full - Expand knows automatically when the end has been reached and ignores any further source data. No copy is made of the data, but copies *are* taken of +12 and +16, which makes it impossible to move the source buffer while it is in use. Reason code 1 is the only time during an operation where the position and size of this buffer can be changed, since it is never in use at this point. No source buffer needs to be provided to Expand until the first time reason code 1 is returned. (Before this, +12 and +16 are considered to be undefined and can be changed as you see fit.) You can expect this event sometime between starting the operation and receiving reason code 3. Indeed, it's even possible that several buffers would be required during the initialisation stage, so no assumptions should be made on this count. By the way, this reason code is *only* for providing a new source buffer. You should not attempt to access the destination buffer, and the value at +28 will be meaningless. Reason code 2 is used for dealing with the destination buffer. If you find it useful, +44 is preserved from the last time reason codes 2 or 3 where returned. However, this value is undefined before reason code 3, and since reason code 1 must be returned at least once before then this might be a little unreliable. 2 The destination buffer has become full. Expand returns this reason code when it has a block of data for you to output. This data is held in the destination buffer, which (with the exception of the last block returned) will always be completely full. The last buffer returned may be completely full or partially full but never completely empty. Your program should dispose of the data in whatever manner seems fit (eg. copy it to disc) then call the expand routine again. If you wish, before invoking the routine again your program may also change the buffer pointer and length fields at offsets +20 and +24 respectively. This is one of the two occasions when you may safely do this; the other is on reason code 3 (which is always returned before any attempt is made by Expand to access the destination buffer). The value at +44 (the number of bytes left to be output) will have been updated, so you can use this to calculate the percentage done so far (eg. for the hourglass). Obviously, the last block to be written will be returned with +44 set to zero, a fact you might find useful if you are trying to do anything unusual with Expand. However, you should only use reason code 0 to terminate the operation. If you want to calculate the percentage of decompression done, use the formula: I - C Percentage = 100 * ------- I where I is the initial value of the counter (ie. the contents of +40) and C is the current value (ie. the contents of +44). Note that the number of bytes in the destination buffer is given by +28. This will usually (but not always) be equal to the value at +24 (the buffer size). Always use +28 instead. 3 Initialisation is complete. The initial data has been read from the source buffer and various variables have been set up. At this point, reason code 1 will has been returned at least once, and no attempt has been made to write to the destination buffer. Reason code 3 is also provided as a chance for you to provide some initialisation of your own. For example, a destination buffer does not need to be set up until this reason code has been returned. And the sample period at +48, the output length at +40, and the byte count at +44 are only valid from this reason code onwards. Expand also returns some useful information about the file being processed in the output flags word at +36. Once you've started an operation, the entire word should be treated as read-only otherwise Expand might get confused. After receiving this reason code (which you will always get, barring errors) you may alter the destination buffer pointer and length at offsets +20 and +24. The only other times you may alter these values are after the utility has finished filling the destination buffer (ie. on reason code 2). Note that copies are taken of +20 and +24 so it is not possible to move or resize the destination buffer except on reason code 2. The values at offsets +40 and +44 may be useful in determining an appropriate size for the destination buffer, or calculating the percentage of work done after each call (eg. for the hourglass). The length at offset +40 is the full length of the sample (in bytes), and not the number of actual samples. The distinction is only important when outputting 16-bit samples; for 8-bit samples they are the same. The value at +44 is used as a counter during the operation. After this reason code, +44 will contain its initial value, which is identical to +40. The value at +48 is a single byte given the sample period of the sample in microseconds. This information is stored in the file at the time of compression, and is otherwise ignored by Expand and Compress, so *could* be used to store other information if absolutely necessary. (With type 0 samples it must be non-zero.) 4 Soft error - undefined This error is undefined at present. Since it is undefined, it should be treated as a hard error for now ie. the operation should be aborted. This is only common sense since your program cannot possibly know how to correctly recover from this error. It should never occur in code from this particular release of !CompUtils. Note that Compress *does* have a use for this reason code, and it doesn't consider it to be an error either. 5 Soft error - source buffer was invalid This reason code is only returned if there is a bug in your program, and should not occur under normal circumstances. It occurs if an invalid source buffer was passed after a request for a new source buffer (reason code 1). The only check performed on the buffer is that the length is not negative and is non-zero. You should either correct the problem and repeat the call, or abort the operation. 6 Soft error - destination buffer was invalid This reason code is only returned if there is a bug in your program, and should not occur under normal circumstances. It occurs if an invalid destination buffer was passed after a request for a new destination buffer (reason code 2). The only check performed on the buffer is that the length is not negative and is non-zero. You should either correct the problem and repeat the call, or abort the operation. 7 Soft error - undefined This error is undefined at present. Since it is undefined, it should be treated as a hard error for now ie. the operation should be aborted. This is only common sense since your program cannot possibly know how to correctly recover from this error. It should never occur in code from this particular release of !CompUtils. 8 Hard error - unknown file format This reason code is returned if you attempt to decompress a sample which uses an algorithm not supported by this particular Expand module. If the 'universal' version can't decompress it then it must be because you have an out-of-date version of !CompUtils. This error will most often occur just after the first source buffer has been passed to Expand, since that is when the file format is checked. It can be returned at other times though, if Expand suddenly finds itself unable to cope with something in the compressed data. Another common cause of this error is if the source data has been corrupted in some way. There's nothing you can do about this error really (except try to get hold of a more up-to-date version of Expand), so you should just abort the operation and perhaps display an error message. Output flags ============ These control the format of the data written to the output buffer and also control various options relating to the use of Expand. They are also used by Expand to pass back certain information you may find useful. The bottom few bits are provided by use and must be set up before you start calling Expand. The top few bits are provided by Expand on reason code 3. Once an operation has been started *none* of the flags should be modified since Expand may become confused. bit 0 - If set, this forces the output to be in linear signed format (eg. Armadeus format), otherwise it is in logarithmic (VIDC) format ready for output to the sound hardware. bit 1 - If set, 2 times oversampling is applied to the output, so it is twice the normal length with the extra bytes being formed by interpolation. This is useful if the sample was recorded at a very low rate, in which case the sampling waveform (which is superimposed on the data by the playback hardware) is of a low enough frequency to be audible, resulting in a high pitched whining sound. Oversampling output goes some way to curing this problem by letting you double the sampling frequency. Note that the sample period returned at +48 in Expand's header is not affected by the setting of this bit, and you will have to adjust it yourself (by dividing by two). bit 2 - If set, the output is to be stored as 16-bit data, otherwise it will be stored as 8-bit. (16-bit data is always stored as little-endian ie. lowest byte first.) This bit is ignored if bit 0 is clear; 16-bit data must always be linear signed since VIDC data can only be 8-bit. Note that not every compression format allows 16-bit data to be stored in it. Similarly, not every format allows 8-bit data. Expand is able to generate 8-bit data from 16-bit data where necessary (with a drop in quality). It can also generate 16-bit data from 8-bit data, though this is mainly for compatibility since there is no advantage in it and it doubles the size of the output. bit 28 - (Provided by Expand) If set, Expand intends to store the sample as 16-bit data, otherwise it will store it as 8-bit data. In certain circumstances Expand may be unable to produce the output you desire (eg. if you set bit 2 but leave bit 0 clear) so you should use this bit to check the output is what you are expecting. bit 29 - (Provided by Expand) If set, Expand intends to store the output using oversampling. In certain circumstances Expand may be unable to produce the output you want so you should use this bit to check the output is what you are expecting. bit 30 - (Provided by Expand) If set, the data stored in the file is in linear signed format, else it is VIDC logarithmic. This is only for information since Expand can generate your choice of output. bit 31 - (Provided by Expand) If set, the data is stored in the file as 16-bit samples, otherwise they are 8-bit. There is no point having an output resolution greater than that stored in the file, but Expand will attempt to generate such output if instructed. This bit is provided for information only. Filetypes ========= The filetypes I use for compressed samples and sequence files are: &350 - Compressed sample &351 - Sequence file These filetypes are not Acorn allocated and may change in the future. To facilitate a smooth transition your program should read the filetypes from the header on startup. This means that if future versions of Expand use different filetypes it is a simple matter of replacing the old code with the new code and everything will still work perfectly (barring renaming the icons in the !Sprites files). File formats ============ You should need to know as little as possible about the contents of a compressed sample for future compatibility. Most information can be read from a file by calling Expand until reason code 3 then reading the data from the header. For more information on the various formats see the !Types file in the main directory. Suggested uses ============== One possible use of this utility (or one similar to it) is in a game. If the game consists of a series of levels, there may well be sounds used on some levels and not on others. On machines with small memories, it makes sense to store these in a compressed form on disc, and decompress them directly into memory at the start of any levels that need them. If you do this, you should set up a small source buffer (eg. 1024 bytes) then call the routine. When reason code 1 is returned you read 1024 bytes from the file to the source buffer and call Expand again. WHen reason code 3 is returned, the value at offset +40 will be the actual length of the sample. You can then create a destination buffer of this length, and repeatedly call the routine again until told to stop, topping up the source buffer when asked to do so. An example of this is included as a Basic program called "Example". You will need to alter the variables in$, out$ and dir$ at the start of the program to get it to run. A second utility called ExpCode is included in !CompUtils in a separate directory, along with some documentation, and does essentially the same thing but from machine code. If even ExpCode is too much effort, there's a module to to it all for you using simple *commands! (See the LoadSample directory.) If you are writing a game, then why not use the shadow screen memory as the source buffer, as animation is unlikely to be taking place during decompression. This saves you having to claim any extra memory for the source buffer and also gives you a pretty hefty buffer to work with, which should increase the speed slightly, particularly with floppies. Most of the time Expand converts to the correct output format on the fly, so the output selected should have little effect on the decompression time. The exception to this is when both VIDC output and oversampling are enabled, where the sample is decompressed as oversampled linear then converted to VIDC when the output buffer is passed back to the user. This is completely invisible from the outside, but decompression times are slightly higher. Not all sample Types use this mechanism however - some versions of Expand can perform *all* the conversions on the fly. Also, the utility may be slowed down considerably by the screen mode selected unless you are using a Risc PC with VRAM fitted. Using Expand with your own programs =================================== Basic ----- Choose an appropriate Expand file and copy it into your application's directory. You can load it using something like this: SYS "OS_File",5,"<App$Dir>.Expand" TO a%,,,,l% IF a%<>1 THEN l%=16 DIM expand% l% SYS "OS_File",255,"<App$Dir>.Expand",expand%,0 and the code can be called with: CALL expand%+0 Although Expand returns the reason code in R0, I would not advise using USR instead of CALL. It's better to use CALL and then read the reason code from the header. It's easier for debugging for a start. Assembler --------- When including Expand in your own programs there is no reason why it has to be kept as a separate file in your application's directory. It is actually designed to be embedded somewhere in the middle of your own code. Refer to the Technical Details section for more information on calling Expand from machine code programs. ObjAsm ------ Users of Acorn's Desktop Assembler will find an AOF version of Expand in the 'asm' sub-directory. A header file is provided to IMPORT all the symbols you'll need. All you have to do is include the line: GET CompUtils:Expand.asm.h.Universal at the start of your source code, then add the appropriate object file to the list of objects and libraries to be linked. There is one object file for each group of sample types supported (eg. Type0-2, Type6, etc) plus a 'Universal' version incorporating all of these. Only one header file is provided (Universal) because the header would be the same for each object file anyway. By the way, only one Expand object can be linked to your code, unlike Compress. If you need two or more then use the Universal version. The header file imports symbols for the start of the Expand header (Expand) and the entry point (Expand_Code). It also defines symbols for offsets from the start of the header to various entries in the header (eg. Expand_SrcBufferPtr, Expand_SamplePeriod, etc). So, to read the sample period you would use something like this: LDR R0,=Expand ; finds the base address LDR R1,[R0,#Expand_SamplePeriod] ; reads the word The first instruction transfers the address of Expand's header into R0, then the second instruction transfers the contents of the sample period entry into R1. To call Expand use: BL Expand_Code which returns a copy of the reason code in R0 to make life easier for you. Apart from these symbols, the header file also defines a series of symbols for various bits in the output flags eg. OUTPUT_LINEAR, FILE_IS_16BIT, etc. These should be used in preference to actual values (eg. 1<<2 or 4+2+1) where possible to allow these to be changed in the future, should the need arrise. It can also be used to highlight trouble spots, if a bit's meaning changes. There are also some symbols giving textual equivalents for reason codes eg. SRC_EMPTY, DEST_FULL, etc which may make your source code easy to read (unless you're using jump tables of course). Have a look at the header file for more details. C/APCS ------ For C users, a header file and a series of object files can be been found in the 'cc' sub-directory. Only one header file is provided (Universal) because all the decompressors have an identical user interface, so the Universal version can be used for any of them. An object file is provided for each group of sample types (eg. Type0-2, Type6, etc) plus a Universal version combining all of these. Only one of these object files can be linked with your code; if you want more than one then use the Universal version instead. The object code is APCS-compliant (obviously) so can be used from any APCS language, such as Pascal, C++, etc. You would have to write your own header file though - the one provided is for C, and should be included in your source with: #include "CompUtils:Expand.cc.h.Universal" It defines the entries in Expand's header as global variables that can be accessed just like normal variables eg. int filetype; int count; char *buffer; char bytes[16]; filetype = Expand_SampleType; buffer = Expand_DestBufferPtr; for (count = 0; count < 16; count++) bytes[count] = buffer[count]; Apart from these variables, the header file also defines a series of symbols for various bits in the output flags eg. OUTPUT_LINEAR, FILE_IS_16BIT, etc. These should be used in preference to actual values (eg. 1<<2 or 4+2+1) where possible to allow these to be changed in the future, should the need arrise. It can also be used to highlight trouble spots: if a bit's meaning changes then so would the name attached to it, and the compiler would spot the fact that the symbol no longer exists. There are also some symbols giving textual equivalents for reason codes eg. SRC_EMPTY, DEST_FULL, etc which may make your source code easy to read, especially when using 'switch'. Have a look at the header file for more details. To call Expand just make a call to function Expand(). This takes no arguments and returns the reason code (which is also held in global variable Expand_ReasonCode). Please look at the example file provided for further details. Bugs ==== There may well be some bugs lurking around, particularly in the ObjAsm and C versions of Expand (I never use these myself). They have been tested with the example programs so they should work, but you never know. Please report any bugs, omissions or suggestions for improvement to one of the addresses in the main !ReadMe file. (c) David Radford