“Typeset” using Impression v2.14 and Publisher v4.01
Menus grabbed by GrabMenu
Musical entertainment provided by Frank Zappa
1 Overview 1
1.1 Disclaimer 1
1.2 Synopsis 1
1.3 Features 1
1.4 Documentation 2
1.5 Compressing your first file 3
1.6 Example speed 3
1.7 Example compression 4
2 BasCompress 5
2.1 Overview 5
2.2 Jargon 5
2.3 Basic 5
2.4 Numbers 6
2.5 SWI's 6
2.6 Star commands 7
2.7 Assembler 7
2.8 Routines 7
2.8.1 Main program 8
2.8.2 Start and end 8
2.8.3 Conditional routine end 8
2.8.4 END and ERROR 9
2.8.5 LOCAL ERROR 9
2.9 Multi-line structures 10
2.10 Libraries 11
2.10.1 Multiply-defined routines in libraries 11
2.11 Label reduction 12
2.12 Line numbers 13
2.12.1 Output line numbers 14
2.13 Multi-line output 14
2.14 DATA 14
2.15 Re-compressing 15
3 Cross-referencing 17
3.1 Overview 17
3.1.1 What's cross-referenced 17
3.1.2 What's output 17
3.1.3 Messages 17
3.2 Level of detail 18
3.2.1 Routine definition 18
3.2.2 Routine calls 19
3.2.3 Routine called by 19
3.2.4 Variable declaration 19
3.2.5 Variable assignment 19
3.2.6 Variable reference 19
3.3 Order 20
4 The front end 21
4.1 Starting 21
4.2 Overview 21
4.2.1 Icon bar icon 21
4.3 Control window 22
4.4 Main menu 23
4.5 Input 24
4.6 Output 26
4.7 Log 29
4.8 Special files 31
4.9 Cross-reference 32
4.10 Choices Dialogue Box 36
5 The back end 39
5.1 Invoking 39
5.2 Installing 39
5.3 Environment variables 40
5.4 The CLI parameters 41
5.4.1 Input 41
5.4.2 Output 42
5.4.2.1 Output listing 42
5.4.3 Log 43
5.4.4 Special 43
5.4.5 Cross-reference 43
5.4.5.1 What 44
5.4.5.2 How much 44
5.4.5.3 Order 45
5.5 Error handling 46
5.6 Escape handling 46
5.7 Hourglass 46
6 Special files 47
6.1 Why 47
6.2 Files 47
6.3 Format 48
6.3.1 Routines 48
6.3.2 Globals 48
6.3.3 Labels 49
6.3.3.1 Verbatim 50
6.3.3.2 Comma separated 50
6.3.3.3 Full pathname 51
6.3.3.4 Wimp menu 51
6.3.4 Variables as regular expressions 51
6.3.4.1 Example patterns 52
6.3.4.2 Limitations 53
6.3.5 Libraries 53
6.3.6 Include files 54
6.4 Limitations 54
7 Errors 57
7.1 Overview 57
7.2 Warnings 57
7.3 Errors 63
7.4 Run-time errors 69
7.4.1 Unknown or missing variable 69
7.4.2 No such function/procedure 69
7.4.3 Missing ENDIF 69
7.4.4 Logical errors 70
7.5 Internal errors 70
8 Loose ends 73
8.1 Memory usage 73
8.2 Missing THEN 73
8.3 Cross-reference 74
8.4 Statistics 75
8.5 Uncompressed output 75
8.6 Label reduction 76
8.7 Executable 76
Appendix A: Messages 77
A.1 Internationalism united 77
A.2 Format 77
Appendix B: Regular expressions 79
B.1 Definition 79
1 Overview
1.1 Disclaimer
This program is supplied “as is”. No warranty, express or implied, of the merchantability of this program or its fitness for any particular purpose is given. In no circumstances shall the author, or any provider or distributor of this program, be liable for any damage, loss of profits, or any indirect or consequential loss arising out of the use of this program.
1.2 Synopsis
BasCompress takes as input a tokenised basic file, analyses it on a routine-by-routine basis, and outputs a cross-reference and a compressed tokenised basic file.
(It was written because none of the currently available Basic squashers handled the side effects of EVAL, removed unused routines, or discarded the junk inbetween routines, and they were all far too slow).
It consists of two programs, a Wimp-based front end, and a CLI-based back end. The former is ideal for occasional use, while the later is better for scripts, make files, etc..
1.3 Features
The main features of BasCompress can be summarised as follows:
• Checks all multi-line IFs have a matching ENDIF
• Checks all CASE's have a matching ENDCASE
• Checks all WHILE's have a matching ENDWHILE
• Checks the ellipsis/quotes/brackets on each statement
• It doesn't balk at the use of line numbers (too much)
• Loads in explicit LIBRARY files
• Checks every routine exits cleanly
• Checks for multiply-defined routines
• Produces a full cross-reference on all variables and routines, with four levels of detail, and user-definable ordering
• Variable/routine name reduction, targetting the most used to be the shortest
• Remove all redundant spaces and REMarks
• Remove all between-routine junk
• Concatenation of output lines
• Remove unused code
• Reduces numbers to their shortest form
• Converts SYS and SWI strings to numbers
• Optional “special” file to handle EVALuated variables/functions, and implicitly loaded library files
• "special" file allows variables to be defined as regular expressions
• It is fast
In other words it does all you would expect, a bit more, and all at a very respectable speed.
All those syntax checks may seem superfluous until you realise that most error handling code isn't always as fully debugged as it should be.
1.4 Documentation
This document is split into several sections. First a full description of exactly what this program does to, and expects of, the input. This is followed by §3, a chapter on the powerful cross-referencing available. Not until §4 is the program itself described, the Wimp front end application. The back end application, accessible from the CLI is §5. Special files, their format and use are detailed in §6. §7 is what this author wishes every program documentation had, a complete list of errors and reasons why they occurred, and more importantly - some hints on how to get rid of them. Finally, §8 contains miscellanea, the junk that doesn't categorise too easily.
1.5 Compressing your first file
Run the application. This will install the Wimp front end onto the icon bar icon and open the control window to the center of the screen.
First of all drag the Log file icon to a Filer window (not another application, sorry). This file will contain information about the actions of the compression process. Now drag a Basic file onto the control window, to compress it.
With the default options this Basic file will be analysed, the log file created and then automatically loaded into your resident text editor. If this is Edit, then don't close the Log window for now.
Even if the Basic file was analysed without error, no output file was produced because none was defined. Just drag the output file icon from the control window onto a Filer window and start again.
Now, because Edit still had a view of the Log file, this was automatically updated, using the current window size. This gives an extremely usable environment :-)
Voila, a compressed basic file has been produced.
If the log file reports any EVAL or DATA statements were found, it is possible that the program may not run. See the chapter on Special files for a way to handle this. (The quick way is to disable all variable and routine label reduction).
Now read §2.
1.6 Example speed
It takes BasCompress less than 20s to compress itself. This needs 4 special files, contains 40-odd source files (including libraries) totalling just over 450K, and compresses it down to 100K. And all this on an Arm2 off of the standard 440 hard-disc, with all the I/O overhead that that involves.
1.7 Example compression
Here's how well BasCompress handles compressing an early version of the Wimp front end application:
192742 Total input size
29659 Maximum compression
29706 No decimal number analysis
31129 No SWI name analysis
35103 No concatenation of output lines
47421 No removal of unused code
51950 No reduction of labels
The later ones are not accumulative, the non-reduction of labels (only) really does add on 57% to the output program size!
2 BasCompress
2.1 Overview
This chapter concerns itself with Basic, and what BasCompress expects (and does) to it.
2.2 Jargon
The following terms will be used quite frequently, so I'll explain what is meant by them:
routine a procedure or function
variable an (integer | real | string)[array]
label the name of a routine or variable
name a label with its (pre | post)fix
E.g. the following table might help
name label
PROC_Zappa _Zappa
Frank% Frank
Cy$() Cy
2.3 Basic
BasCompress will only except fully tokenised basic files, it does not accept text files. This program only knows about the tokens present in Basic V, v1.04. The behaviour of this program on Basic files on a version greater than this is undefined.
Only one Basic file is parsed (but see §2.10).
2.4 Numbers
BasCompress will interpret all numbers and try to output them in as compact a form as possible. This is most effective on long binary numbers, but can shorten many large decimal values as well.
The analysis and output of (decimal) numbers requires the floating point emulator. For this reason this can be turned off, just in case you are very short on memory.
Analysis of hexadecimal and binary numbers is always done, as this does not require any floating point math.
2.5 SWI's
Acorn's interface to the operating system, the SWI is an elegant self-documenting system. However, for interpreted languages like Basic, the translation of a SWI name to a SWI number is relatively time-consuming.
BasCompress will try to do this translation, leaving just a number. This provides both large space and large execution savings, particularly for Wimp programs that use SWI's during screen redraw (that's all of them!)
In order for BasCompress to translate the SWI name, it must be a simple constant string expression. If it isn't, or it can't translate it, then it is left as it is - thus providing an (almost) fool-proof conversion. (A possible case of a complex string constant would be SYS "X"+ "OS_ReadC").
Both SYS calls (in Basic) and SWI calls (in assembler) are converted.
Note also that the any modules used by the program should be resident at the time of compression, otherwise the SWI's will be unknown and thus BasCompress will leave them as strings. This is an easy mistake to make, and results in slower and longer compacted files being produced. This cases ud detected, and generates a suitable warning.
2.6 Star commands
A backward compatibility feature of Basic V is it's allowance of *Commands anywhere in Basic. Modern programs should really put this in an OSCLI("..."), or better yet a SYS "OS_CLI", "...".
However, since this is allowed, some programmers use it to introduce comments in programs by using the *| comment construct. BasCompress will remove these lines completely. It will also remove all unnecessary *'s and spaces.
If this results in a complete removal of the statement, BasCompress may produce incorrect code if (and only if) this *command was just after an (explicit) THEN. Since this is extremely bad programming practised, this author felt the benefits gained far outweigh the possible side-effects.
2.7 Assembler
BasCompress fully understands and compacts assembler statements. As part of this process, BasCompress also checks that the square bracket “[]” count on each statement is valid - which may be useful for detecting bugs in conditionally assembled code.
All R0..R15 registers get translated to 0..15, this saves a lot of space. Also the integer EQU's can be renamed to DC's.
Note that the single space left after some mnemonics is necessary, without it the Basic assembler will refuse to work.
2.8 Routines
It is important to understand that BasCompress treats a program as a group of routines. Because of this it can remove any junk inbetween routines, i.e. comments not explicitly starting with a REM, that other squashers leave behind (and even try to analyse - producing many, many errors).
Because a record is kept of what happens inside a routine, it is possible to conclude that a routine isn't actually needed in the output file - and so none of the routines that it calls are needed that bit less as well. Thus BasCompress can remove ALL unused routines from the output file. This is a very powerful feature.
2.8.1 Main program
BasicV does not provide a clearly defined method for distinguishing the end of the main program, and the start of the subroutines. BasCompress treats everything from the start of the first file to the first routine definition (DEF PROC or DEF FN) to be the main program. This isn't ideal, as many main programs consists of an infinite loop followed by junk - however there is no easy way of recognising the true end of the main program, so you have to live with it as it is.
If you wish to compress library files separately, i.e. files that do not contain a main routine at the start of a file, then this is easily accommodated.
2.8.2 Start and end
The start of a Basic routine is very easily detected, it is a line that starts with DEF FN or DEF PROC.
However, the end of a routine isn't so easy to detect. In the ideal world, every routine will have one, and only one exit that is a simple ENDPROC, or = <expr> on a line of its' own. However, we all live in the real world, and things are more complicated than that.
2.8.3 Conditional routine end
For example, consider the following function:
DEF FNmin(a, b)
IF a>b THEN =b ELSE =a
Now, the only way for BasCompress to recognise this construct is if it kept track of all branches of a conditional. This would involve a major upgrade, and would make the program just a code-generation pass away from a true compiler.
So what BasCompress does is to ignore exits from a routine that occur on the same line as an IF.
But, in the above example, the next line will be thought to be inside the function min. This usually results in a cascade of warning/error messages being generated. The only cure for it is to amend the offending function. The simplest way is to just put a dummy terminator in, e.g.
DEF FNmin(a, b)
IF a>b THEN =b ELSE =a
= 0
Better yet, would be to code it “properly”:
DEF FNmin(a, b)
IF a>b THEN a=b
= a
There is also the problem of programs that prematurely terminate routines inside, say, a case structure. This happens quite often, because it makes for much shorter code, and so BasCompress recognises this, see §2.9.
2.8.4 END and ERROR
Currently BasCompress does not recognise the fact that a routine may end with the END command, nor an unconditional ERROR. Since these are examples of bad-programming anyway, these are easily worked around by just adding a redundant ENDPROC or =0 afterwards.
2.8.5 LOCAL ERROR
BasCompress fully recognises local error handlers, and will force the next input line to start a new output line accordingly.
2.9 Multi-line structures
BasCompress keeps track of the current nesting level of all multi-line structures:
• CASE ... ENDCASE
• IF THEN ... ENDIF
• REPEAT ... UNTIL
• WHILE ... ENDWHILE
It does this primarily to detect conditional exiting from a routine. For example, if an ENDPROC or = <expr> is detected inside such a structure, then BasCompress knows it isn't the true end of the routine, and so will ignore it. A very useful side-effect of this is that BasCompress detects programming errors in the use of these constructs, errors that the run time Basic interpreter allows through. These very often are genuine program mistakes, usually inside error-handling code that is not fully debugged.
However, there are two caveats to this. Firstly, in order for BasCompress to do this it must assume that the start of any multi-line structure is not inside a one-line IF construct. Unfortunately, some programmers put a quick WHILE ... ENDWHILE loop in a one-line IF. BasCompress does not handle this correctly and this line needs to be split into a true multi-line IF.
Secondly, and just as less seriously, BasicV is rather lax in it's attitude towards multi-line structures. For example:
REPEAT
i += 1
CASE x(i) OF
WHEN 1:
IF do_it THEN
UNTIL FALSE:ENDPROC
ENDIF
...
...
ENDCASE
UNTIL i>max_i
Here, BasicV will rather carelessly execute the UNTIL, even though it isn't at the same nesting level as the matching REPEAT. BasCompress is not so lenient and it will refuse point blank to handle such code. It is another example of bad programming being used, and should be re-coded in another way.
2.10 Libraries
The use of the LIBRARY call is recognised. It causes the appropriate file to be appended to the queue of files to be parsed, and the entire LIBRARY statement to be removed from the output.
For this to work, BasCompress assumes a simple string constant follows the LIBRARY. If this is not the case and it uses a variable/routine parameter then you will need to set up a Special file. This will tell BasCompress what to expect the variable to be so it can load the correct file. Please refer to the §6.
All LIBRARY files are only ever scanned once, even if it is included many times.
2.10.1 Multiply-defined routines in libraries
Note that the loading of libraries, as performed by BasCompress is not exactly the same as the order that BasicV would have loaded them in. This could only cause a problem if multiply-defined routines exist, and further more these multiple definitions are themselves in library files, not the main file. If this situation does arise, then the kludge is: include the library containing the preferred definition twice, once before and once after the LIBRARY containing the unwanted definition. This will work because BasCompress will use the first loaded, and the uncompressed program would use the second!
Of course, the loading of libraries will mean that the current directory and/or some system variables (e.g. App$Dir) may need to be set up - otherwise BasCompress will not be able to locate the library file.
2.11 Label reduction
A lot of the time that BasicV takes to interpret a program is spent looking up variable names. BasCompress will attempt to reduce the long labels down to size, often producing dramatic space and speed improvements. The algorithm used ensures that the most used variables are chosen for the shortest variable names, and the names themselves are chosen so as to spread evenly across the name-space.
Did you note the word attempt in the above paragraph. This is because basic is an interpreted language and provides the powerful commands EVAL and DATA, allowing expressions to be evaluated in the current context. For instance EVAL("zappa%") would yield the value of the variable zappa%. But, if BasCompress had reduced this variable down to, say, A% in the rest of the program (because it analysed the program in a different context) then what happens at run-time is you get a variable not found error.
There are two solutions to this very common problem. The first is to disable label reduction on all variables of the type that are used in EVAL or DATA statements. This is extremely wasteful, but the only option available in other squashers. With BasCompress there is a much more elegant solution - you can specify all the labels that must not be reduced. Further more, these variables can be implied from the parameters passed to a routine!
For example, many Wimp-based programs will have a menu-construction suite of routines. These will be passed a string that describes the menu. Inside this string will be references to variables that at run-time will point to more information (sub-menus or windows usually). With BasCompress, you can get it to analyse all these strings, extract the variables, and then reduce all other variables apart from those. This is a very powerful feature. See the §6 for more info.
Note that BasCompress goes to the trouble of making sure that it never produces one of the “built-in” register labels used by the assembler (R0-R15, pc). On other basic squashers this can lead to VERY obscure bugs.
Also worth mentioning is that any labels accessed inside deleted routines are automatically removed from the list of labels to reduce. This produces better results than just reducing all labels found.
BasCompress can handle any number of labels, well as many as could fit into 16Mb of RAM :-)
2.12 Line numbers
Normally line numbers are an anathema but there does exist a valid reason for using them in BasicV, and so line number handling has been included in BasCompress. The reason why line numbers may be needed is if a program claims more memory though the use of the END=<expr> construct. This has the unfortunate side-effect of removing all routine level info, so you have to GOTO the main program loop).
Obviously, line numbers found in any library files are faulted as there is no valid reason for them being there.
Note that in short programs it is just possible that there will be a GOTO to a destination outside both the main program and all other routines. Currently BasCompress does not handle this (rather rare) case. As a kludge, surround the offending code in a dummy routine.
2.12.1 Output line numbers
If no line numbers were found then line numbering is easy. For single-file programs the line numbers keep to their original values, even if multi-line compaction is enabled. For multi-file programs the output file starts from 1 and increases in steps of 1.
If line numbers were found then the line numbers in the original program are used for the original program part, and thereafter line numbers increment in steps of one for any further libraries.
Unfortunately BasicV only allows line numbers up to around 65000 odd. This could possibly be a problem if the original program has line numbers up to, say, 64000, and includes quite a few library files. This is one of the few possible errors that are not checked for, as the possibility of it occurring are just so remote.
2.13 Multi-line output
Normally as many statements are compressed onto one output line as will fit. This produces the smallest files as the overhead that Basic imposes on each line is reduced by quite a bit.
However, it can produce code that runs slower. This is because it appears BasicV only notes the statement number of an implicit jump (e.g. after a FOR, WHILE, or REPEAT). And so if this is on statement 56 of a line, say, then it has to scan all along the line to find where to continue execution. This situation may be recognised in a future upgrade by forcing the statement after one of these cases to start on a new line.
2.14 DATA
BasCompress recognises the possibility that DATA may not reside inside a routine. All “unlinked” DATA statements will still be included in the output file, but only if there is some code left that will READ it.
Please note that variables used as DATA will require the use of Special files.
2.15 Re-compressing
Although at first sight the notion of compressing an already compressed file may seem a waste of time, in actual fact it is not.
This is because BasCompress compresses a whole line at a time, and then merges together two or more lines. This isn't done quite as optimally as possible, and sometimes extra colons are inserted.
If the output is fed back into BasCompress these extra colons will be removed, as it will be obvious that they are truly redundant.
This is best achieved by first compressing with reduction of function and procedure names only (thus ensuring ‘special’ routines only parsed once), and then a second time with full reduction of variable names.
3 Cross-referencing
3.1 Overview
The cross-referencing of a large program can provide many useful insights, providing you can organise the output so as not to swamp you with “useless” information. To this end you can control what gets include, the level of detail, and the ordering (with many types of ordering available).
Note that the cross-referencing of variables and routines is completely independent.
3.1.1 What's cross-referenced
The cross-reference contains only the routines and variables that will be included in the output file. Since dead code is usually removed, you have to tell BasCompress to keep in all would-be deleted stuff if you need a complete cross-reference.
You can also control exactly what types of labels are included. Usually you'd keep the default (everything), but sometimes you don't care about all the real variables, say, and this is easily catered for.
3.1.2 What's output
The result of the cross-referencing is sent to the cross-reference file, or the screen if none is specified. Since this is a lot of data, you will almost certainly want to use a file. The front end application has the ability to automatically load this into the resident text editor.
3.1.3 Messages
The formatting of the cross-reference is defined using the external Messages file. By altering the following messages you can tailor the output of the program dramatically (see appendix A for description of format of messages):-
Name Show label and its' qualifying string (“%s%s”)
Comma Separates distinct references (“, ”)
SemiColon Separates similar references (“; ”)
They are currently set up to produce “one-line-per entry”. However, it is possible to change these three so that every distinct reference appears on each separate line, vis-a-vis:
Name: \n\t%s%s
Comma: \n\t
SemiColon: ,%
But note, you will probably need to alter all the titles used so that they start rather than end with a newline. (The \t expands to a tab character, this is usually better than many spaces, since cross-reference files are large enough as it is)
3.2 Level of detail
There are four levels of detail supported by BasCompress:
None suppresses output, obvious really
Existence useful to just list the name of all the labels used
Global gives the additional information of a count of the label usage
Routine reports only each separate routine where a reference was made, this is probably the most useful option
Line details the exact statement for every reference
For the last two, each label has separate lists. For routines there is: defined, calls, called by; and for variables there is: declared, assigned, and referenced.
Note that the main program itself is treated internally as a procedure, and so appears in the routine cross-reference.
3.2.1 Routine definition
This gives the file and line numbers of the start and end of the routine.
3.2.2 Routine calls
Lists the names of the routines called by this routine, and the line number of the call.
3.2.3 Routine called by
Lists the names of the routines that calls this routine, and the line number of the call.
If the list is empty then BasCompress knows that this routine is not needed in the output program, and so will not include it.
3.2.4 Variable declaration
Where a label was “declared”. By this BasCompress means it is a formal parameter of a routine, or explicitly declared as LOCAL.
3.2.5 Variable assignment
When ever a variable appears at the start of a statement. This includes assembler statements.
Note that BasCompress sometimes thinks variable assignments are references. This happens in the following cases:
• variable is passed to a routine with RETURN parameters
• assignment after a one-line IF that is without a THEN
Here's an example of the later:
IF x<y x=y
Because Bascompress doesn't try to understand the conditional, it doesn't know that a new statement has started and so can't categorise the second reference to x as an assignment. See §8.2.
3.2.6 Variable reference
Any other instance of a variable other than the above two is taken to be a reference.
BasCompress will not recognise the fact that x += 1 is actually both an assignment and a reference.
BasCompress will also fail to recognise that var!0 = 1 only references var and does not assign to it.
3.3 Order
There are many uses you can put to a list of labels, provided you can order them in the way you need. BasCompress allows you to specify as many levels of sorting as you would need, with all the types of ordering that are relevant to the labels!
You can sort in either direction - top to bottom, or the more usual bottom to top.
These are the following sort types currently supported:
Name in Ascii order
Type for routines: function, procedure
for variables: integer, real, string, integer array, real array, string array
Dictionary by name, but as it would appear in a dictionary
Location the location of the reference
Usage for routines: number of times it is called
for variables: sum of the assignments and the references
Note the needed for multi-level sorting. You would normally sort labels by name, and type; and references by name, type, and location.
Trying to sort labels by location has no meaning, and will result in a seemingly random order. So it is not possible to list routines in the order they were defined in.
4 The front end
This describes the Wimp front end application. This allows the user to set up the parameters for the back end program in a friendly way.
4.1 Starting
A standard Archimedes application, just double-click the Filer icon to install the application onto the icon bar.
For foreign users, see the appendix A describing the Message file .
4.2 Overview
Because the back end works on, and produces, a number of whole files the format of this application is slightly unorthodox.
Basically you use the standard Risc-Os method of dragging file icons to Filer windows (to define the output files), and then drag a basic file onto the application to invoke the back end application to compress it. This generates the new log, cross-reference, and output files.
Not all of the output files need to be defined. Usually there is no need for a cross-reference. But, you will almost certainly want the log file defined, as otherwise you would not know what errors occurred, or anything else.
It is almost a pre-requisite to have Edit running alongside this front end in order for you to view the log that the back end application produces. The loading of this log will normally be automatic.
4.2.1 Icon bar icon
The icon bar icon shows some messages while the back end is active. This gives a visual reference to what is going on.
Clicking SELECT on the icon bar icon brings the control window to the front of the window stack.
Clicking ADJUST in the icon bar icon re-loads the last input file, using the new options. This is extremely handy.
4.3 Control window
This is automatically opened to the center of the screen when the application starts. It allows you to quickly set up all the files to be used, and as a side effect it gives you the chance to open a menu somewhere other than in the bottom right of the screen :-). The left-most three icon groups act just like the save as dialogue boxes to be found on the menu.
The special file is defined by dragging a Text file onto this window. More special files can be defined by editing the text field, appending a comma separated list of file names.
4.4 Main menu
There is only one menu.
Because the sub-menus are rather on the large side, it is recommended that you bring the menu up over the control window. This was the main reason for having a control window, as a convenient anchor for the menu.
4.5 Input
This sub-menu defines various parameters affecting BasCompress's analysis of the input file.
Allow multiply-defined routines
It is better to leave this option un-ticked, so that if a routine is defined more than once an error is generated. This is the default.
Ticking this option allows a routine to be defined more than once, with only warnings being given. See §2.10.1.
Report multiple exits from a routine
BasCompress needs to know when each and every routine ends. If it finds more than one exit it will report it. For large programs this can produce many warnings, and so these warnings can be disabled. See §2.8.3.
Parse numbers
This toggles whether BasCompress will try to reduce a string of decimal digits. Hexadecimal and binary numbers are always compressed. See §2.4.
Convert SWIs to numbers
There doesn't appear to be any reason to disable this. See §2.5.
Force malformed SWI's to generate error
Malformed SWI's are those that aren't simple strings, e.g. "X"+ "Wimp_Poll". With this enabled these generate an error, otherwise just a warning. See §2.5.
Report unknown SWI's
In order for BasCompress to convert a SWI string to a number the Module must be resident at the time of compression. If the module is not resident, then this warning will be given. This toggles the appearance of such warnings, and is usually left enabled. See §2.5.
Process as a library file
This option allows the input file to be treated as a library file. This means that no main program is expected, and any undefined variables and routines do not generate an error. Of course, using this option is usually pointless without also disabling all label reductions.
4.6 Output
This sub-menu allows you to tailor how much compression is applied to the output programs. Redundant spaces and comments are always removed, since there seems little point in using baScompress without doing this.
By default all compression is on.
Save as
This leads to a standard dialogue box, used to define the basic file that will be produced if the input is analysed without error.
Concatenate lines
Forces as many statements as possible onto each output line. This is usually very desirable as it produces quite substantially shorter code, but at the possible loss of a bit of execution speed. See §2.13.
Remove unused routines
Because BasCompress can work out exactly which routines are, and are not needed in the final program, then it can remove unused routines. This is the default, and there is little point in disabling it, other than creating a full cross-reference.
List
This leads to a simple sub-menu where you can specify the screen mode to use. When enabled, as BasCompress produces the output file it will switch to that screen mode and scroll the source and output in two separate columns. Although "pretty", it is also pretty useless, as this takes at least an order of magnitude longer to do, what with all that screen scrolling and printing.
Reduce variable names
By default, all variable types are reduced. The only conceivable use for disabling the reduction of these would be to circumvent the use of EVAL or DATA variables, although BasCompress provides a much better method, via the use of Special files. See §2.11.
The final option, ‘E’ suppression is used to stop BasCompress shortening any variables to a name beginning with an ‘E’. For the reason why this might be desirable, see §8.12.
Reduce routine names
By default, all procedure and function names are reduced. The only conceivable use for disabling the reduction of these would be to circumvent the use of function names used inside an EVAL construct, although BasCompress provides a much better method to handle this, via the use of Special files.
4.7 Log
The log sub-menu controls what additional information may be inserted into the log file, along with the name of all files scanned and any warnings and errors.
Save as
This leads to a standard save as dialogue box, used to define the file to which the log will be dumped. This will be a standard Text file.
Statistics
This shows how many of each type of variable and routine there is altogether in the program. This includes all deleted labels.
EVAL keyword
DATA keyword
READ keyword
These will list out every line that contains such a keyword, indicating where BasCompress may introduce errors because of its' reduction of labels. Note that only the use of these keywords outside special routines is reported, as it is assumed the use of the keyword was fully handled by the Special file. Further note that DATA found outside any routine is just reported as unlinked.
List input
With this option ticked, the input basic is also listed inside the log file. This produces humongous files, as this is plain text Basic not tokenised Basic.
4.8 Special files
Special files are used to help BasCompress to handle the EVALuation of variables. A special file consisting of a list of routines that are expected to contain this keyword, and/or a secondary list of any particular labels that must not be reduced.
The writable menu item allows you to type in a full pathname. It is easier just to drag the file onto the control window, though.
Warn undefined
Normally it wouldn't matter that you have told BasCompress about special routines that aren't used in the input file. However there is still this option that will report superfluous definitions found in Special files.
Show expansion
With this item ticked, every special label constructed will generate a warning message into the log file. This can be useful for showing what you think should appear, and what BasCompress thinks should appear.
4.9 Cross-reference
From here you define all aspects of the (rather large) cross-referencing sub-system.
Save as
This leads to a standard save as dialogue box, used to define the file to which the cross-reference will be dumped. This will be a standard Text file.
Include deleted
When ticked, this will force all the superfluous routines and variables to be included in the cross-reference. This is usually not such a good idea, as the process of deletion removes all the reference information. For a full cross-reference, also disable the removal of unused routines (see Output), which leaves this information in tact.
Reference order
This leads to an order dialogue box (see below) used to set the sorting criteria for the references. This is probably best left as the default: Name, Type, Location.
Variables
Routines
These two items lead into identical sub-menus. They allow the cross-referencing to be defined for both independently.
Verbosity
This leads to a sub-menu where you can define the level of detail of the cross-referencing information. See §3.2.
Types
Used to define what types of variables or routines are included in the cross-reference. Usually this would be all types, but sometimes you might only want to know about the integers, say. Just de-tick all the others.
Order
This dialogue box is quite complicated. Basically, you are trying to define several layers of ordering. First of all everything is sorted according to the left hand column. Now BasCompress goes through these sorted items looking for a sequence of items that are the same, according to this ordering. Now, for each sequence found it will sort them again using whatever you have defined in column two. This recurses on each smaller and smaller sub-sequence to the columns on the right.
Obviously, it isn't much use having two columns sorting by the same criteria, so BasCompress will not allow you to select it.
You can delete a column by ADJUST clicking on the ordering already used. This will cause all columns to the right to shuffle one column to the left (this is quite a pleasing graphical effect :-)
It isn't possible to have a “blank” column. If you attempt to create a blank column then your new column will be shifted to the left (yet another pleasing graphical effect :-)
It is possible to delete all columns. This will cause the output to appear in a seemingly random order. Not much use, apart from seeing how good the hashing function is!
Ordering by location is only meaningful for references. Using this option for variables or routines will result in a (seemingly) random order.
The Dict. standards for dictionary, and is similar to Name, except instead of ordering by ASCII, order as it would appear in a... (guess).
4.10 Choices Dialogue Box
This dialogue bog controls aspects of the front end application itself, not the back end. It grabs the input focus so that some keyboard short cuts can be used, must notably RETURN and ESCAPE. Also the bottom row of buttons can be pressed using F2, F3, etc..
Set
This accepts the above values for the current session of BasCompress. For the effects to be permanent use either Save or Save full. Also see the note below about using outline fonts in dialogue boxes.
Cancel
This disregards any edits you have made to the above options.
Save full
Along with all the expected switches and sort ordering, also saved with the choices are the default names of the log, cross-reference and output files. This option saves the full pathname of each of these files, and would be used while developing a project, to save dragging the log and output files each session.
Save
As Save full but only the leaf names of the log, cross-reference and output files are saved. This is handy for more generic environments. For instance, if you always call your Wimp programs Wimp, then to make a standard application from this, the output file would be !RunImage, and this would be a better default to have than Output.
Default
This sets up everything to a default state, for when you've made a complete mess of the options. This default state can also be achieved by deleting the Choices file inside !BC.!fe, and restarting the application.
Run BasCompressed file
With this option ticked whenever an output file is specified it is immediately executed after being generated. This is useful while constructing a Special file, so the program can proceed and report the missing variables found :-)
Load log file
If a log file was specified then this is “opened” immediately after it is generated. On a properly set up system this would cause Edit to be loaded up automatically if necessary, and replace any version currently held if Edit is already resident. Multi-tasking at its best. Note that if you edit the “old” log file, then this smoothness is ruined, because Edit will open up a new window containing the new version. C'est la vie.
Load cross-reference file
Same as the above option, but loads the cross-reference file generated, if any.
Name output as <Main>
This is a kludge to reduce the size of both cross-reference and log files. As any Wimp application must use the full pathname of every file, this is what is passed to the back end application. However this causes the full pathname to be printed in every error message and reference. As a stop gap, this front end can be told to set the environment variable <Main> to the full pathname, and pass this to the back end application. It does mean you don't see the actual name of the file being processed, but that usually isn't a problem as you know that anyway.
Output file leaf name same as input
This option is useful when compressing lots of files from one directory to another, for example having compressed versions of all your library files. Having defined the output directory, the full pathname is constructed from this directory and the leaf name of the input file, saving quite a bit of typing.
Use outline fonts in dialogue boxes
BasCompress comes with two versions of all its’ windows − with and without outline fonts. Users of low-resolution monitors may prefer to use the system font versions. Also, RiscPC users may have to use the “System font” windows for two reasons: first because the fontified windows use fonts at a different size (slightly larger for low resolution users) which looks ugly but more importantly because of a bug in the Window Manager that causes clicks on (genuine) font icons to cause the Wimp to “forget” the text font :(
Note that for historical reasons changes to this option are not updated immediately but only take effect the next time BasCompress is run.
5 The back end
This chapter describes the underlying CLI program, and how to use it effectively.
By default all text output is to the screen. This can be redirected to separate log and cross-reference files, as required.
5.1 Invoking
Double-clicking on the application directory from a Filer window will automatically start up the Wimp front end application. This is because no parameters were passed to the !Run file.
If any parameters are passed to the application, i.e. you activated it from a command shell then the back end application is called direct.
This application requires the floating point emulator module to be resident. This is automatically loaded from the current system resources. (Floating point arithmetic is only used to analyse and output decimal numbers, this can be disabled and so there will be no need to load the floating point emulator - this is not a recommended procedure, though).
Also required to be resident is the authors' own library module: CAssembler.
These modules are automatically loaded if not resident.
5.2 Installing
There are two methods of installing BasCompress for easy use from a shell. One is to just install the application onto your Run$Path, and use it by *!BC file.
This has the disadvantage in that you must remember the !. The much better alternative is to use the Alias$BasCompress that is set up for you in the !Boot file. Thus it is only required to run this !Boot file in your boot-up sequence, and then call BasCompress by *BasCompress file.
In both the above it is essential that the !Run file gets executed, as this ensures auxiliary modules are resident and checks that there will be enough memory to start the application.
5.3 Environment variables
Because the CLI limits command lines to (a paltry) 256 bytes, several tricks were used in order to make the front end application work effectively with the back end program (with full pathnames). These can be put to great advantage for CLI users. In all cases an unset, or empty variable is ignored.
BasCompress$Options
This should be a string of the same format as a normal parameter set, except missing the name of the input file (of course). With this you can override any of the built-in default values with your own preferred set.
BasCompress$Output
The name of the tokenised basic file produced.
BasCompress$Log
The name of the log file produced.
BasCompress$XRef
The name of the cross-reference file produced.
BasCompress$Special
The name of the special file(s) input (a comma separated list).
BasCompress$Path
A comma separated list of directories used to find Special files
It should be noted that the front end application resets all of these variables (except the Path), so keep this in mind.
5.4 The CLI parameters
The ordering of the parameters has been carefully set up so there is usually no need to name any parameters.
BasCompress [-In] <file>
[[-Out <file>]
[[-Special] <files>]
[[-Log] <file>]
[[-XRefFile] <file>]
[[-XRef] <n>]
...
So for normal work you would have set up BasCompress$Options and you'd just do BasCompress @.MyProg @.Result @.Special @.Log "" 0.
There is a -Help option to display a brief description of the available options.
As with all all native Archimedes applications, switches toggle the previous state (OS_ReadArgs does not allow -f and +f).
5.4.1 Input
-In <file> this must always be given
-Library ignore undefined routines & main program (OFF) §2.8.1
-MultiDEF allow multiple definitions of the same routine (OFF) §2.10.1
-Reduce <type> reduce these types of labels (irsIRSpf) §2.11
-Single no concatenation of output lines (OFF) §2.13
-Unused delete unused routines and variables (ON) §2.8
<type> is a string of these letters:
p procedure
f function
r real variable
i integer variable
s string variable
R real array variable
I integer array variable
S string array variable
5.4.2.1 Output listing
First the mode is selected (-1 would select the current mode), and an appropriate warning or error generated if that mode is not available. Next a blue-on-white colour scheme is selected. Then the display is split into two columns, with the left column about 61% of the screen wide (80 columns in mode 16). The source listing is scrolled here. On the right column the output is scrolled.
The source listing does not include any deleted routines, but does include all inter-routine junk. The output listing is as per the output file, in its' entirety. The right column is only updated each time a whole line has been amassed, which could take a while if maximum compression is selected.
There isn't really much use for this option, as the time taken to scroll the screen takes far too long.
5.4.3 Log
-DATA log lines containing DATA (ON)
-EVAL log lines containing EVAL (ON)
-Goto log lines containing line numbers (ON)
-List echo input to log (OFF)
-Log <file> output file (see BasCompress$Log)
-READ log lines containing READ (ON)
-Stats log program statistics (ON)
5.4.4 Special
-Special <files> use comma list of <file> to resolve implicit usages (see BasCompress$Special and BasCompress$Path)
-UnusedS report undefined special routines (OFF)
-WSpLabel report label expansion results (OFF)
5.4.5 Cross-reference
-XRefFile <file> output file (see BasCompress$XRef)
5.4.5.1 What
-Deleted xref includes deleted variables and routines (OFF)
-XIncVar <type>
-XIncRtn <type>
-XInc <type> xref these types of labels (irsIRSpf) §3.1.1
<type> is a string of these letters:
p procedure
f function
r real variable
i integer variable
s string variable
R real array variable
I integer array variable
S string array variable
5.4.5.2 How much
-XRef <n> level of xref detail for both routines and variables §3.2
-XRtn <n> level of xref detail for routines (0)
-XVar <n> level of xref detail for variables (0)
<n> evaluates to a number from 0 to 4:
0 None
1 Existence - name
2 Global - name / count
3 Routine - reference to routine level
4 Line - reference to line level
5.4.5.3 Order
-Sort <sort> order of all variable, routine, and reference sorting §3.3
-SRef <sort> order of cross–references (NTP)
-SRtn <sort> order of routines (TN)
-SVar <sort> order of variables (TN)
<sort> is a string of these letters:
N sort by name
T sort by type
D sort by name (in dictionary sense)
P sort by position in source
U sort by usage
e.g. to sort by usage, name, then type = “UNT”.
Lower case letters order top to bottom.
5.5 Error handling
By errors, it is meant system errors (i.e. those generated by calls to the operating system).
Admittedly, error handling is quite primitive in BasCompress. All calls to the operating system are tested for error condition, and if set this is immediately propagated all the way back to the user, with an appropriate message dumped in the log file.
Errors occurring before the log file is opened are reported to the screen.
In any case, an error safely closes all files opened by this application.
5.6 Escape handling
As with error handling, this is fairly primitive.
BasCompress can be aborted at any time, terminating immediately after closing all open files.
Note that during the middle of program output the Basic file will be Type'd to Data, as a safeguard against loading it into BasicV. (This does not handle zero-length files very well - it hangs the machine).
5.7 Hourglass
The hourglass is used to indicate BasCompress's progress.
During parsing, the percentage shows how much has already been analysed. Note that because BasCompress can not know the total length of input beforehand, the percentage may actually go backwards as LIBRARY statements are parsed.
During cross referencing, there is no percentage, but the bottom L.E.D. is on.
Finally, during program output an accurate percentage is displayed, with the top L.E.D. on.
6 Special files
This chapter explains the format of the Special files. These are auxiliary Text files used to give BasCompress some more information to help it compress better. See also the examples supplied on disc.
6.1 Why
Special files are used to tell BasCompress how to cope with the following types of situation:
• EVAL("Variable")
• DATA Variable
• LIBRARY "%."+ RoutineParameter$
• EVAL("FN_"+ RoutineParameter$)
Because BasCompress reduces labels (see §2.11) using a global context, at run-time when BasicV tries to find these labels in the current context, it fails. Special files tell BasCompress which labels not to compress, either explicitly as in the first two cases or above, or implicitly (from routine parameters) as in the later two.
6.2 Files
Special files are just plain Text files, in the format specified below. BasCompress allows more than one Special file to be defined (using a comma separated list), but it is far easier to use the #include directive inside a Special file, as shown below.
Special files are found using the environment variable BasCompress$Path. By default this is not set up, but you may like to create a sub-directory BAscompress in your library and set this variable to ",%.BAscompress." in your boot-up sequence (note the trailing dot, as with all path variables). Here you would keep the one or two special files that handle your particular set of library files, and BasCompress will find them for you without you having to type in a full pathname every time.
6.3 Format
The format of a Special file is fairly simple.
Basically, (for variables) all you're trying to tell BasCompress is:
• routine X has a string in parameter Y
• inside routine X there is an EVAL
• this constructs a new label Z from Y
• don't reduce this label Z
So all you do is give the name and expected parameters of a routine and tell BasCompress how to construct label Z.
6.3.1 Routines
You give the name of a routine by:
procedure foo(1)
or,
function bar(,1)
The brackets are necessary if a routine doesn't have any parameters.
The procedure definition above tells BasCompress to expect a (simple) string expression to be passed to the procedure foo at every call, as the sole parameter. The function definition says the first parameter is not important, but the second parameter will be a simple string.
These prototypes are checked against what is actually found in the program proper, and an appropriate error generated if they don't match.
There can be up to ten distinct string parameters [0-9], but you'll probably only ever need just one.
6.3.2 Globals
Sometimes the EVAL isn't in any particular routine (or is used inside a nested routine structure not amenable to the above simplification), so you'd like to give the name of the labels not to reduce just once. This is easy because at the start of each Special file, BasCompress puts an implicit declaration of the main program as a routine, with no parameters. So you'd just start defining the global labels as normal routine labels.
6.3.3 Labels
First of all an example:
PROC_List(a, "Zappa", "Good")
...
:
DEF PROC_List(RETURN x, y$, z$)
IF EVAL("Frank_"+ y$)<>0 THEN
x += EVAL("FN_"+ z$+ "_Name(Cy%)")
ENDIF
ENDPROC
this would be implemented as:
procedure _List(, 1, 2)
real Frank_\1
function _\2_Name
integer Cy%
Note the spaces to the left of the implicit declarations, and the case of the keywords as both of these are important. Also note the need to declare Cy% as another variable that must not be shortened.
There is no attempt to verify that the <n> used was declared in the parameter list, but all undeclared <n> are reset to the empty string. Also there is only rudimentary checking done on the expansion template, so be sensible.
The following table shows the exact words to use - note they must be lower case:
real a real variable e.g. pi_by_2
integer an integer variable, e.g. count%
string a string variable, e.g name$
real_array a real array variable, e.g. times()
integer_array an integer variable, e.g. books%()
string_array a string variable, e.g. titles$()
procedure a procedure, e.g. PROC_total
function a function, e.g. FNmin
library load library file, e.g. %.Wimp
BasCompress can currently interpret the simple string in a number of ways. These template expansion types are:
6.3.3.1 Verbatim
\<n> causes the formal parameter <n> to be substituted verbatim.
6.3.3.2 Comma separated
,<n> treats parameter <n> as a comma separated list of strings to substitute, with optional spaces after each comma.
Each value is extracted from the string in turn. Note that the string is expected to contain real variables only, i.e. no %, $, or (.
For example:
PROC_Music("Zappa,Varese,Stravinsky,Belew)
...
:
DEF PROC_Music(p$)
LOCAL i, dummy
i = INSTR(p$, ",")
WHILE i>0
dummy = EVAL("FN_"+ LEFT$(p$, i-1))
p$ = MID$(p$, i+1)
i = INSTR(p$, ",")
ENDWHILE
ENDPROC
could get coded as:
procedure _Music(1)
function _,1
6.3.3.3 Full pathname
@<n> parameter <n> is considered as a full pathname, and just the leaf name is substituted.
For an example see §6.3.4 below.
6.3.3.4 Wimp menu
»<n> <n> is taken to be a Wimp menu string. This means that after ">" and ":" a variable name is expected, terminated by a comma or the EOS.
It is assumed that the implicit variables are all of the same type. If not just define several rules, one for each type.
For example
PROC_menu("Prog,Info>Info%,dSave as>SaveAs,Quit")
may get coded as follows:
procedure _menu(1)
integer »1
real »1
6.3.4 Variables as regular expressions
As well as the above special meta-characters BasCompress will now allow you to specify special variables (not routines or libraries) as a regular expression (a fancy wildcard expression). This means that instead of specifying one variable, you specify any variables that match a pattern.
Special variables so specified can not contain any parameter substitution commands. They also behave in a slightly different manner.
To process special variables defined as regular expressions BasCompress must use a separate pass, between parsing and output. This scans every variable (of the particular type) to see if it matches the specified pattern. If so, then the variable is flagged as being un-renamable − as expected.
So what is a regular expression, and why would you want to use it? For a full description of regular expressions please see Appendix B, but all the useful features (two of them!) will be shown in the examples below.
You would want to use regular expressions to specify a range of variables with the same sort of name − instead of specifying every single one individually. Eg say you have defined many constants such as 'Wimp_Initialise', 'Wimp_CloseDown', 'Wimp_GetRectangle', etc. and then used these constants in some DATA statements. Instead of having a (long) list of these variable names in the special file you would recognise that all of the variables begin with the pattern 'Wimp_' and tell BasCompress to mark all such variables as unrenamable.
6.3.4.1 Example patterns
Here's how you would write the above example in a special file:
procedure dummy()
real Wimp_.*
the important bit being the ".*". This should be read as "any character", "repeated 0 or more times". Ie as long as a variable starts with the five characters "Wimp_" the pattern is matched.
Another frequent case is when variables have the same postfix - eg the pattern:
integer .*CMOS\>
would match all integers that end ("\>" means "match end of variable") in the letters CMOS, ie MosROMFrugalCMOS%, FontMaxCMOS%, etc... The specification that CMOS be at the end stops, say, MyCMOSvar% being matched by the pattern, as it otherwise would if this were not included.
6.3.4.2 Limitations
Although a regular expression may be defined dependent on a special routine being called, this fact is currently ignored − all special variables specified using a regular expression are used during the pass. For this reason it is probably worthwhile to place all wild-card specifications as global − ie in the main program section of the special file. This limitation will not last long.
Currently a variable that is recognised as matching a pattern is flagged as being referenced from the main program (in the cross reference) and not inside the special routine that specified the wildcard − but this will change in a later version.
6.3.5 Libraries
As well as labels, libraries may be implicit. For example, here's a simple library handler:
PROC_Load_Library("<Basic$Lib>.Wimp", 2)
...
:
DEF PROC_Load_Library(f$, v)
LOCAL x
LIBRARY file$
x = EVAL("FN_Init_"+ FNleaf(f$))
IF EVAL("Version_"+ FNleaf(f$))<v THEN
ERROR 1, "Library too old"
ENDIF
ENDPROC
Well, in real life you'd set up error handlers and note which libraries have already been loaded, but you get the idea. This could be coded as follows:
procedure _Load_Library(1,)
library \1
function _Init_@1
real Version_@1
Note the use of @<n> to extract the leaf name.
In the output program the entire statement containing the LIBRARY is removed.
Note that because BasCompress analyses the program in a linear fashion, the order of library routines may well be different from that used in the uncompressed program, (see §2.10).
6.3.6 Include files
It is possible for special files to include other special files. Thus you will probably set up BasCompress$Path and a special file to handle your own set of standard libraries, and then...
#include mylibs
somewhere in the special file for each particular program. (Note that there are no spaces to the left of the #include, and don't put any quotes on the name either).
Nesting level is not limited, and you can recurse. This will result in an infinite loop, with the only resort being to press escape to quit, and then examine the log file.
6.4 Limitations
There can only be 250 odd total Special routines defined. If any one actually reaches this limit, then contact the author, it's easy to alter - at the expense of a slight increase in memory usage.
There is of course one major limitation in that you have to specify the name of all the special files before processing the program. It would be nice to set up a system whereby if the program (maybe implicitly) loads in library X, then load up special file Y. However this would require re-finding all calls to the (new) Special routines - something that seems quite hard to do :-(
Finally, you'd like to be able to take into account the scope of the variables so only the use of variables in scope are not reduced. But since BasicV only has one scope, this is not possible to implement safely, without some explicit help from the programmer by marking the scope of variables in some way.
7 Errors
7.1 Overview
Errors and warnings can be generated at various stages of the game. They all look the same and are all sent to the current log (file / screen). The format is:
file 1234 warning 3: message
If there is no filename, then it indicates the error is global in scope, for example the “undefined routine” error generated when all files have been parsed.
Warnings report non-fatal deficiencies. Errors report a major problem, and will stop any output file being generated.
As with all text used by BasCompress, these error texts are held in the Messages file.
It is worth pointing out that many warnings and errors may be generated after the seemingly innocent warnings 9 and 10. This means that BasCompress may have lost touch with what routine is being defined, and starts interpreting the junk inbetween routines as genuine code. So do check the log file from the start.
7.2 Warnings
Most of the warnings are self explanatory.
1 Expecting PROC or FN after DEF
In standard Basic, these are the only valid things after a DEF. This warning has never been encountered in debugged source!
2 DEF inside a routine
This is generated because an earlier routine did not exit cleanly, and the program believes you are still in this routine. You need to fix the errant routine by terminating it properly, or simply appending a dummy routine end (ENDPROC or = <expr>).
3 Unexpected period ‘.’
A period on it's own is treated as the start of a number (e.g. “.7”). This warning is generated if a statement starts with a period and it is not in the assembler.
4 Mismatched quotes ‘"’
Every line should have matching quotes.
5 Malformed hexadecimal number
It is expected that a digit or a letter [0-9a-fA-F] comes after an &.
6 Malformed binary number
It is expected that a 0 or 1 comes after a %.
7 Line number reference found
This is given if line numbers are found in any libraries, since there is no valid reason for them being there.
8 Expecting routine name after PROC or FN
A non-label character was found after one of these tokens. This very unlikely.
9 ENDPROC not at end of procedure
This is generated if ENDPROC is found on a line containing an IF, i.e. it is ambiguous as to whether this is the genuine end of the procedure. Most of the time this causes no problems, but for short routines where this is the only way it exits then BasCompress will get confused as to the exact end of the routine. In this case you must either re-code the routine (recommended) or insert a dummy routine terminator (ENDPROC or = <expr>).
10 = not at end of function
This is generated if an “= <expr>” is found on a line containing an IF, i.e. it is ambiguous as to whether this is the genuine end of the function. Most of the time this causes no problems, but for short routines where this is the only way it exits then BasCompress will get confused as to the exact end of the routine. In this case you must either re-code the routine (recommended) or insert a dummy routine terminator (ENDPROC or = <expr>).
11 Code found on last line of function
There were more statements on the same line as the terminating “= <expr>” statement. Since these will never be executed by Basic then they should be removed. This warning could well turn up quite often if BasCompress is fed the output of other squashers.
12 Expecting a string after LIBRARY/INSTALL
In order to be able to load in libraries BasCompress expects a simple constant string expression to follow these keywords. If this is not the case then you will need to set up a Special file to handle this situation. This warning is suppressed if inside a Special routine.
13 OVERLAY file(s) not scanned
The OVERLAY keyword is not supported by BasCompress.
14 Unexpected ‘[’
A running total of square brackets is kept. This indicates that something like “LDR R0, [[R1...]” was found.
15 Unexpected ‘]’
A running total of square brackets is kept. This indicates that something like “LDR R0, [R1...]]” was found.
16 ENDPROC inside a %s structure
Found a conditional exit from a procedure - this indicates a bad programming style.
17 = inside a %s structure
Found a conditional exit from a function - this indicates a bad programming style.
18 Conditional %%s
Found a conditional termination of a multi-line structure - this indicates a bad programming style.
This may occur on lines like the following:
IF x>0 THEN WHILE x>0:y+=FNy(x):x-=1:ENDWHILE
Unfortunately BasCompress does not like this because it expects all multi-line constructs to start unconditionally. To cure this split the one-line IF into a multi-line IF.
19 Too many ‘(’ found
A running total of ellipsis is kept. This indicates something like “(1+2)(” was found.
20 Unexpected ‘)’ found
A running total of ellipsis is kept. This indicates something like “(1+2))” was found.
21 Routine %s already defined at %s
This warns about multiply-defined routines. Since there are sometimes very good reasons for doing this this warning can be suppressed.
22 Malformed decimal number
A bad decimal number was found. A decimal number is something beginning with a digit or a period and continuing as expected. The form of the number is always checked, but the compression may be turned off.
23 The (global) special routine %s is not defined anywhere
This speaks for itself. It is only generated if the user has asked for it, as it is not usually relevant.
24 SWI name missing, or not simple
It is normal to just have a string, variable, or number immediately after a SYS (SWI). This indicates that found something else - which may indicate a syntax error.
25 SWI ‘%s’ is unknown
The operating system has no record of what this SWI means. This indicates that either the module is not loaded at the time of compression, or it is misspelt. BasCompress will leave it as it is, producing longer and slower code.
26 Can't change to screen mode %d
If the listing Mode could not be found, but a suitable alternative was possible.
27 Special variable %s has been used
28 Special routine %s has been used
29 Special library %s has been used
These warnings are generated if the user has turned them on. They show the results of label expansion done at the instance of a call to a Special routine.
7.3 Errors
Any errors encountered will stop the final output phase, but the program will soldier on parsing, so you can rectify all the errors in one hit.
2 ENDPROC found in a function
This invariably follows warning 2, about DEF found inside a routine, which itself follows a warning 10, because the previous function did not terminate cleanly.
3 = found in a procedure
This invariably follows warning 2, about DEF found inside a routine, which itself follows a warning 9, because the previous procedure did not terminate cleanly.
4 Routine %s already defined at %s
It's either an error or a warning (21), and this is the error.
5 File error: %s
Since there is minimal file accessing done (files are loaded in one hit), this usually just reports not being able to find a particular file/invalid file name - it is dependent on the underlying filing system.
6 End of file in middle of routine
It is invalid for a routine to span across a file boundary.
7 Too many files (>254)
I can't foresee anyone getting this error message! It is unlikely that this limitation would be lifted, either.
8 Undefined routine: %s (used %s)
Generated after all files have been parsed, it warns of routines used but never defined. These are usually obscure routines only called in error handlers. If you know for sure that this routine will never be called, just put an empty version in your main file to suppress this error.
9 Undefined variable: %s (used %s)
Not generated.
10 FOR not at start of statement
This may be of some use - not much, but what the hey? (Maybe be on a one-line IF)?
11 Escape pressed, parsing aborted
Not generated. Escapes just terminate program immediately in the current version.
12 File %s does not have a Basic file type
This program only accepts tokenised Basic files - maybe the file is just not Type'd correctly. (Could be that it was saved under the CFS or some such, and you only gave the SCSIFS?)
13 File %s is too small to be a Basic type
A rudimentary check on each Basic file is done - this is generated if it fails.
14 File is corrupted/not basic
Each line of tokenised Basic has a header. As the file is parsed each header is checked and if invalid this error is generated and parsing of the file is aborted.
15 Escape pressed, xref aborted
Not generated. Escapes just terminate program immediately in the current version.
16 Escape pressed, compression aborted
Not generated. Escapes just terminate program immediately in the current version.
17 Unexpected %s in a %s structure
This detects any foul-ups in multi-lined structures. The Basic interpreter allows some of these through without reporting an error. These errors allow you to fix the (very probably) mistakes. See warning 18.
18 Unexpected %s
As error 17 above, but obviously outside any surrounding structure.
19 End of file in middle of assembler
A special case of error 6, i.e. a routine did not end before the end of a file was found.
20 OTHERWISE not at start of line
A simple error for a simple mistake.
21 WHEN not at start of line
A simple error for a simple mistake.
22 Computed GOTO/GOSUB found
Now, I ask you, is this how to program? BasCompress will refuse to work with such things, and who can blame it?
23 INSTALL/LIBRARY not at start of line
This indicates a syntax error, probably. Or maybe an IF without a THEN, who knows?
24 INSTALL/LIBRARY can not be on a conditional line
If you must, then split into a multi-line IF and try again.
25 The special routine ‘%s’ was declared as having different parameters from those found here
The number of parameters do not match - probably a wrong number of commas in the Special file.
26 The special routine ‘%s’ wants a string constant for one of its' parameters
This program only handles very simple constant string expressions, and this error indicates that the expression was not simple enough.
27 Line too long (max 256)
This seems a reasonable upper limit on the line length of a Special file.
28 Line ‘%s’ is not of a recognised form
Syntax error for the Special file line. Note that if you specified a Text file full of control codes and such, then you may only get to read the last few words of this message.
29 Label ‘%s’ is invalid
In Special files, the label probably starts with a digit, or some such.
30 Parameter list ‘%s’ is invalid
In Special files: what more needs to be said?
31 Parameter list ‘%s’ contains a duplicate
In Special files: it is not exactly meaningful to have two or more parameters assigned to the same expansion number is it?
32 Implicit label substitution overflowed
When a call to a Special routine is found, the label templates get expanded using the actual parameters passed. This error shows BasCompress checks for overflow in its' internal buffers!
33 Too many special routines (max 255)
A perfectly reasonable limit it seems. This could easily be extended, though.
34 Unable to load special file (not Text?)
Or maybe it wasn't found. Only Text files are accepted.
35 SWI name missing, or not simple
It is normal to just have a string, variable, or number immediately after a SYS (SWI). This indicates that found something else - which may indicate an error.
36 Illegal screen mode
If the listing mode could not be found, and no suitable alternative was possible, then this error is generated.
37 Bad main program, unterminated %s structure
This is a special case, if the first routine found (DEF PROC, or DEF FN) was inside a multi-line structure. This indicates an error in the main program.
38 Can't find closing ellipsis in special routine parameter
During label expansion, BasCompress found a label ending in (, which it took to mean the start of an array index. No closing ) was found before the end of the string.
39 Bad char `%c' following > or : in special routine parameter
During label expansion, BasCompress expects the label to be terminated by a comma or the EOS. This indicates that this condition was not satisfied.
7.4 Run-time errors
Here is some help with possible errors that BasCompress may introduce to the compressed Basic output.
7.4.1 Unknown or missing variable
The quick and easy method to solve this problem is to turn of all label reduction. This is not recommend, as it makes the use of BasCompress almost worthless.
This error has been generated because you have not told BasCompress about the side effects of an EVAL or DATA statements. See §2.11 for why this produces the error, and §6 for how to cure it.
In essence you have to go through the program, locate the EVAL's and possibly the DATA statements as well, (look in the log file for exactly where to find these), and then build a Special file to tell BasCompress how to handle these.
Usually this entails defining some “global” labels that will never be reduced, although sometimes it is possible to build a more sophisticated parameter-based rule.
It may just be possible that this error is generated inside assembler. If so, then this indicates an error on BasCompress' part. Examples of such should be sent to the author for inclusion in upgrades. This possibility is extremely remote as BasCompress has been tested with a wide range of sources, but most cases where it has failed in the past has been in this area.
7.4.2 No such function/procedure
This occurs for exactly the same reasons as detailed in §7.4.1 above.
7.4.3 Missing ENDIF
These errors are generated very rarely, but they are just possible. Consider the following code:
IF FN_SaveFile THEN :::REM comment
What BasCompress could produce, although it does tries not to, is:
IFFNxTHEN
Obviously this is wrong, wrong, wrong. From a simple one-line conditional (that does nothing), BasCompress has produced a multi-line conditional construct where there shouldn't be any, and without the ENDIF.
Since this situation is fairly rare, and BasCompress does handle it most of the time, it is not likely to be remedied - unless enough people request it.
The work around is to not to ignore the return value from the function, which is probably an error status (indicating a very lazy programmer!), but to assign it to a variable, and then act on it accordingly.
7.4.4 Logical errors
By this it is meant that a program doesn't generate any overt errors, it's just that it doesn't do the same as it did before compression.
BasCompress has been tested quite thoroughly with a wide source of Basic programs, and it is hoped that all of these exceptional cases have been noted and duly handled. However if you do get this type of error, it is still probably because of EVAL and DATA being used with undeclared (and probably short, e.g. A%, x, etc.) variables. These variables may have been “renamed” by BasCompress, or even re-mapped to different variables, and hence contain an invalid value.
If it is not because of this, then the author would appreciate it if an example of the incorrectly compressed source was sent in, so BasCompress could be updated to handle that case.
7.5 Internal errors
These are “system” errors generated by BasCompress itself. These messages are hard-coded into the executable, and use the error number chunk &DCBC0 to &DCBFF:
Error(s) during parsing
This should never appear, as it is an internal error used to indicate an erroneous parsing stage.
Invalid sort character '%c'
Sort string too long
Invalid type character `%c'
These indicate rudimentary errors on the parameters passed on the CLI.
Invalid option in system variable BasCompress$Options
Indicates an invalid system variable. Unfortunately it is not possible to show exactly what was wrong with it, just that some part of it was wrong.
1.1
...
10.1
These errors should never be generated. They indicate where BasCompress has found some inconsistencies in its' own internal data structures. Any occurrences of such should be sent, along with the source that produced them, to the author for correcting in future upgrades.
8 Loose ends
The fact that this chapter exists shows that I'm no good at documentation :-) It mainly describes features that may (or may not) get implemented in future upgrades.
8.1 Memory usage
Let's face it boys and girls, this program eats RAM like nobody's business. It's really just a fact of life for 32-bit/RISC machines.
This is compounded by the need for some oomph (technical term) in the program execution stakes. To this end most things are kept on two separate lists - e.g. if a routine X calls Y then this is recorded both in X's calls list, and Y's called by list. This bit of redundancy probably doubles the speed of cross referencing, so it was felt worth it.
Also all the source is kept in RAM, and all the output.
There is some consideration to low-memory users in that the hash table sizes grow on request, but then again they start pretty big anyway :-)
If enough people request it then BasCompress may be upgraded to include the option of smaller initial hash-tables and demand-loading the source - a simple hack & slap.
8.2 Missing THEN
Basic allows the programmer to omit a THEN statement when used on a single-line IF compound. However, this can cause problems. Because BasCompress removes all spaces, if the condition ended with a digit, and the statement after the (missing) THEN starts with an ‘E’ then Basic gets confused – thinking a real number (in exponent format) has been given. Eg the BasCompress produced line:
IF x<>0E%=1
is wrong. There should be a space after the 0. This is very rare. A temporary cure is to stop BasCompress shortening any variables to names starting with an ‘E’. Although this will solve most problems, there is still the unresolved problem of the un-shortened variables. The only cure is to make sure the THEN is always used.
A similar problem is the following BasCompress produced line:
WHILE x<>0E%=2
The cure for this is to always separate the expression from the first statement of a WHILE compound by a colon.
While testing this program, it became evident that the missing THEN was part of a general problem - skipping a Basic expression. This is harder than at first seems, because there is no point at where you can say “yes, here is the end”, as you can in a procedural call, say (e.g. you've found a comma, close ellipse and the bracket and quote count is zero). E.g. how would you handle this:
IF a=b=c THEN P."hello"
Intuition get's it wrong. Basic treats this as:
IF (a=b) THEN =c ...
Which means you have to build a tree/stack. But, once you can do this then this program could become quite interesting. It could evaluate constant sub-expressions. This could produce some very compact stuff, as you'd also be able to substitute constant variables with their values, and even eliminate dead code. I.e. debugging stuff that would never be executed because it is known that, say, Debug is always FALSE.
But, tie that in with a bit more knowledge of the structure of the Basic, and it is almost easy to generate not tokenised Basic, but code. Well, I'm sure if enough people register, it will give this author the incentive to develop BasCompress into the fastest (and cheapest) Basic compiler.
8.3 Cross-reference
It would be nice to have a toggle between multi-line and single-line (the current) output. This is fairly easy to do, but it'll require some experimentation :-)
Also handy could be some ordering based on label length, or even the (approximate) size of a routine.
Some rationalisation of the messages used is in order, too. Currently the program outputs some line feeds of its' own accord, which is decidedly against the grain.
How about not listing the file that a routine is in? Or just the leaf of the filename. This would reduce the size of the output by quite a lot. The former would require the user to cross-reference (the cross-reference!) to find the file of the reference, though.
8.4 Statistics
There are loads more statistics that could be done;
• libraries that aren't needed
• variables only ever declared (e.g. LOCALised but never used)
• variables only ever assigned to once (constants)
• lines where the memory indirection operators ?, | and ! are used
• don't log all the deleted routines and variables, just count them
• give a percentage of the code that is unused
• percentage compression achieved on the used code
• lines containing any user-defined basic token (e.g. PRINT)
• time taken to load, parse, cross-reference, compress, write
8.5 Uncompressed output
It would have been nice if there was an option to just remove the unused routines and merge all the libraries into one big file. This would give a nice listable program containing only the routines used. However this is currently unpossible, as BasCompress was never intended to produce anything but compressed code. (There are many places where “skip until hit non-space” is hard-coded, and it would take a rather long time to rationalise these so you could toggle it on/off.) Oh well.
8.6 Label reduction
The reduction of labels isn't always as good as it could be. Currently BasCompress only judges the fitness of a label to be reduced on its' global usage. A better algorithm would take into consideration the nesting level of each usage, and use this to bias the reduction of labels towards those used inside loops.
8.7 Executable
The current situation, with two separate applications is far from ideal. A better situation would be to have the back end program inside a module. This would have the added bonus of not needing to load in the Messages file every time it is run.
Better yet would be to have the whole thing as one big Wimp application. Here many extras are possible because there would be direct access to the cross-reference database. For example it could create graphs of variable name distribution, provide the cross-reference in it's own window, where you could filter it given user criteria (such as name matching a regexp AND usage>3). But I don't think so, do you?
Appendix A: Messages
A.1 Internationalism united
For users of a foreign extractment, the Messages directory contains the country-specific text files. Copy Default, to (say) Iceland and edit the messages, and if that is the configured country of the machine, then those messages will be loaded. Note it is sensible to keep a Default file as this will be loaded if no file of the configured country could be found.
If you do make a translated messages file, the author would appreciate it greatly if you could send it in so every one can benefit.
A.2 Format
The message format is fairly obvious - just look at them to see what can and can't be done, but here's a quick summary.
• Comments start with a #
• Messages names start on a new line, and end in a colon
• Messages start at the first non-space after the colon, and finish at the end of the line
• Spaces can be done as “% ”
• Percents can be included using “%%”
• The “parameter”s %s and %d are supplied by BasCompress, these are standard C printf() strings
• The standard C escape sequences \b, \t, \n, \r, \xXX, \000, etc. can be used to insert those characters inside the messages
• A message may be a selection: starts with a “%?” and consists of a comma separated list of alternatives
Appendix B: Regular expressions
B.1 Definition
A regular expression (RE) is a fancy wild-carding system – i.e. you don’t need exact matches. A regular expression is really a mini–programming language with the following “commands”:
. any character
^ start of line
$ end of line
[...] set of characters*
Any other character matches itself
There are also modifiers:
| or, “re1|re2” match either RE1 or RE2
~ not, “~c” matches anything but the next character
+ many, “re+” matches RE repeated one or more times
* repeat, “re+” matches RE repeated zero or more times
? optional, “re?” matches RE repeated zero or one times
Parenthesis () can be used to group REs
Finally, there are memories:
\{ start recording
\} end recording, and place in next memory (starts with 1)
\<n> \1, ... \9 will then match the corresponding memory
* character sets:
Character sets are used to match one of a group of characters. All characters are taken literally except “–”, “]”, and “\”. The “–” indicates a range, unless it is the first or last character, or the first character after a previous range. The “\” can be used to include a “]” or a “–” in the set.
