Black Box 4/BlackBox.cdr/database/datamage.arj/DOCS.ZIP/BASICS.DOC

Text File | 1991-08-25 | 18.0 KB | 338 lines

COMPUTERIZED DATA STORAGE AND RETRIEVAL:

Before learning the DATAMAGE data management system you need to know some basic
information,  hence  the  title  of this section  of  the  documentation.   The
facility  offered the DATAMAGE system is the storage and retrieval of data,  be
it  your  customers and their balances, your payables and their  amounts,  etc.
DATAMAGE can be used to store any FORMATTED data.

DATABASES:

There  are  basically  two  types of  databases:   Those  programs  that  store
unformatted  text  and those that handle formatted data.  DATAMAGE  is  of  the
latter type, which are commonly called DATA MANAGEMENT SYSTEMS.

The former type of program might be used to record articles from newspapers  or
periodicals and find the text pertinent to specific needs of the user.
You might be a rocket scientist, and have a need for information about the  Van
Allen  radiation  belts that surround this planet.  With a  TRUE  DATABASE  you
could  scan all information in your database, that might well be about the  sex
habits  of  the Tasmanian devil, and have it spit out all the  articles  making
mention of the radiation belts.

DATA MANAGEMENT SYSTEMS:

A  data management system lacks the ability to store  unformatted  information,
but  offers a host of other capabilities pertinent to the use and  manipulation
of formatted data.  This kind of program MANAGES data.

You  might  be  that same rocket scientist, having shot  one  of  your  rockets
complete  with  instruments of measurement, through the radiation  belts.   The
data  returned  by  the rocket during it's flight might  concern  it's  current
altitude, the intensity and type of radiation encountered there, etc. Your need
would then be to record FORMATTED (that is, predictable) data about the  flight
of  the  rocket, to be able to quickly find the data generated at  three  miles
into the belts, and to compute totals and averages concerning certain  segments
of the experiment.  These are the facilities offered by DATAMAGE.

PAPER FILING SYSTEMS VERSUS DATA MANAGEMENT SYSTEMS:

An old-fashioned paper filing system consists of ink stains dried on sheets  of
paper,  grouped into files in the drawers of your filing  cabinets.   Computers
store  files  on disk, as 1s and 0s.  These 1s and 0s are grouped  into  BYTES,
which  correspond to an individual ink stain on a piece of paper.  Though  it's
just  a blob of dried ink, you know it's an A.  And, though it's just eight  1s
and 0s, the computer also knows it's an A.

Paper files are REALLY inconvenient when they become large.  It takes HOURS  to
search  through  four thousand paper files to select the group with  which  you
need to work.  But to search through four thousand records on a disk drive will
require less than a minute.  The 1s and 0s are a lot smaller, you see.

There are differences in paper files versus disk files.  Foremost among them is
uniformity.   All records in a disk file have the same format though  the  data
they  contain will differ.  In a paper file you can keep shoving in  sheets  of
paper until you run out of drawer space to hold it.

RECORDS:

Here's a graphic representation of a disk record:

NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNPPPPPPPPPPPPDDDDDDDDDD

The first thirty-five spaces in the record are for a name, the next twelve  for
a phone number, and the last ten for a date.  If you can imagine four  thousand
of  these  strung  together  with no spaces  in  between,  you  can  accurately
conceptualize what you're working with, though we'll use more fields.

This  compact  data storage format would hardly be acceptable for  people.   We
prefer  white  space between the data we see, and headings to  identify  it  as
well.   For the computer this is a perfect storage format; the essence  of  the
computer is speed and it needs no white space.

You  will  never  see  the data in the compact  storage  format.   When  it  is
displayed  or  printed for your consumption white space and  headings  will  be
added.  There's no reason to record the white space on the disk drive, it would
only waste the space available and slow the computer down by forcing it to deal
with larger records.

FIELDS:

Disk  files have FIELDS that store data.  Each record in a given file  has  the
same  fields.   These fields have three general purposes:   1>    identify  the
record in which they appear  2>  store data  3>  store transient information.

A  field that identifies a record in a disk file stores the name, part  number,
description,  etc.;  the data by which we identify the record.   This  kind  of
field is called a KEY field.

A  field  that stores data contains information pertinent to the  record.   The
data can be numeric, dollar, string or date formatted.

A field that stores transient data is a field that contains a message; a  state
of  the record.  An example of this could be a field called "STATUS: O  or  P."
If there is an O in this field you know that the record, be it a check, invoice
or whatever, is outstanding.  If it contains a P then you know it's paid.

Indeed,  paper files are more flexible than records on disk; they  may  contain
any  piece  of paper you shove in them, including one that belongs  in  another
folder.   But  when  processing many paper files  that  flexibility  slows  the
process  tremendously due to the necessity of looking through different  pieces
of paper that may or may not contain the data you need to see.

This  not  to  say that records on disk can not contain  comment  fields  whose
purpose  will differ between records; merely that if such fields  are  included
then  each  record  will have the same number of comment fields,  of  the  same
length.  All records within a given disk file will be identical.

CLASSES OF DATA STORAGE - FIELD TYPES:

The  primary  goal  of the DATAMAGE system is to  provide  computer-based  data
storage,  retrieval and manipulation to your average clerical employee who  has
not  taken  a  computer  science degree, nor suffered a  night  course,  as  is
required by most of the competing software.  This means keeping things  simple.
In the world of computers, SIMPLICITY IS POWER, and LESS IS MORE!

The  fields in the records of DATAMAGE files are different, one  from  another,
depending on the class of data they are designed to hold. Four types of  fields
are  proffered by the system:  STRING (alphanumeric), NUMERIC, NUMERIC  DOLLAR-
FORMATTED, and DATE.

Alpha and date fields hold exactly what you type in, in ascii format.  DATAMAGE
supports  numeric  storage of sixteen digits of numeric  precision  with  user-
defined  decimal precision or full floating point.

Numeric  and  dollar fields are recorded in the MICROSOFT  binary  format,  and
occupy  eight  bytes regardless of their content.  If you use  a  disk  utility
program to look at a YOURDATA.RAD. file (this is the filename of all  datafiles
produced by DATAMAGE) you will see what looks like disk garbage in the  numeric
fields.

This  format is a MEMORY DUMP; the content of the computer's memory is  written
directly  to  disk.   It was chosen due to it's compatibility  with  the  BASIC
interpreter and it's usability by C and BASIC compilers.  Due to the fact  that
no  conversion from the ascii format to the computer's internal storage  format
is necessary the programs can manipulate this data very quickly.

FORMATTING YOUR DATA:

If  you  are  starting  fresh, having no previous paper  files  to  convert  to
computer  files,  your task will be simple.  If you  have  current  paper-based
filing systems and wish to convert them to computerized files you will need  to
bridge  the gap between the formats and you will need to convert  your  current
data to fit the new requirement of uniformity imposed by the computer.

CONVERTING YOUR CURRENT PAPER-BOUND FILES:

In many paper files some folders contain more sheets than others.  If a data is
present in only one folder and is foreseen to be available for or pertinent  to
few others, it should not be included in the datafile.

To know what information to include in your datafile, you need to know how your
computer  works.  In general, computers are neither intelligent  nor  creative.
But they can perform a great number of the same operation, using different data
for each, at frightening speed.  And, they can store huge amounts of data in  a
very small space.

Let's  say  that  you  need to computerize your  employee  file.   The  folders
contain, in addition to the obligatory information on name, scale, tax  status,
credit   union  and  insurance   deductions,  etc,   performance   evaluations,
commendations   and  write-ups   (disciplinary)  for  some  employees.    These
"comment" sections of  the employee records can consist of several pages.

In  this  case,  you  would not store any of the  comment  information  in  the
datafile.  You could place fields in the datafile to indicate the presence  and
type of comments for that employee  outside the computer system.  If  you  need
the portability of the comments as well, have a steno  to  record them  all  on
disk with your word processor, which is designed for this job.

You  could  place  the filenames of the comment files,  produced  by  the  word
processor,  in  the datafile.  Then, the system could produce a  list  of  all
comment  files on an employee, or a group of employees, made  by  your  sorting
parameters.

The  date of the comment may also be considered important enough to be  in  the
datafile.  You could produce lists of all employees, having a comment in  their
files  dated  after a certain date.  The filenames of these comments  could  be
included in the datafile, and the disks containing the comment files could then
be loaded by the word processor and reviewed.

INSTALLING YOUR OWN DATAFILE:

You  will need to do some planning before you set up your own datafile.   These
instructions  can only offer you some helpful hints because we have no idea  of
the type of data you want to store.

In  general, the more time taken to design a datafile, the more useful it  will
be.  You will have a lot of work to do,  entering the records.  You should take
time, now, to maximize the return you will get from your investment.

When you are ready, start the DBSEMAKR program.  It is #5 on the menu of the GO
program.   The documentation for this program is included as DBSEMAKR.DOC.   If
you  have  a  floppy-based machine have the target disk ready  to  receive  the
datafile  definition files.  The  datafile  definition  will be written to  the
chosen directory when you finish.   Now, you enter the records by starting  the
BASE program, and loading the new datafile.

PLANNING YOUR DISK SPACE:

The  raw  materials  available to the DATAMAGE system  vary  from  computer  to
computer.   With a floppy based computer, having 40 track drive(s),  the  gross
storage   available  is around 300 KB.  Contrary to popular belief,  there  are
1,024  bytes  in  a  K-BYTE.  The file index can occupy  up  to  40K,  and  the
definition files take some room, etc.

You should know the approximate number of records your datafile will be  called
upon  to  hold.  Divide 307,200 by this number, and you will  get  the  maximum
allowable record size, in bytes.

For  example,  suppose you need to place up to  2000 records on a  forty  track
floppy.  307,200 divided by 2000 is 154.  And, at 154 bytes, the fields must be
small if they are to be many.  Using four string segments length 35 bytes,  and
a  single  numeric  segment (8 bytes) covers 148 bytes  of  disk  space.   Most
datafiles have between twenty and fifty fields.

A  DATAMAGE datafile can occupy 320 million bytes on disk.  (32,000  records  X
10,000 bytes per record), and this does not count the file index or the control
file.  The control file for the monster datafile would occupy 64,000 bytes  and
the file index would occupy 320,000 bytes.

A  single datafile can fill ANY hard disk!  No matter what kind of  disk  drive
you are using you should consider the storage it offers as a VALUABLE RESOURCE,
and avoid wasting it.

NUMERIC STORAGE CONSIDERATIONS:

All  numeric  data stored by the program is double precision, having  16  digit
capacity,  and  full  floating point.  To store this type  of  number  on  disk
requires  8  bytes.  All numeric fields,  whether their class is  numeric,  or
numeric dollar format, reside in eight of their own bytes on disk.

Calculations  done  by the program have 16 digits to work with,  and  are  more
precise  than the same calculations done with calculators.  The result  of  all
calculations  done will be rounded to the decimal precision specified  for  the
field in which they are recorded.

STRING STORAGE CONSIDERATIONS:

STRING  fields  can  store any type of data, including numbers.   You  are  now
reading  STRING  data.   The  name STRING was  originally  coined  by  computer
programmers  as  a  STRING  of  untyped  characters.   It  is  also  known   as
ALPHANUMERIC  data.  If you store numeric data in STRING fields that data  will
not be eligible as the input to or output of calculations.

The   maximum  length of a string field is 250 characters, and  each  character
requires  one  byte of disk space.  Using  too many possible  characters  in  a
string  field  causes  wasted disk space, and too few causes  the  data  to  be
difficult to read, due to over-abbreviation, etc.

SWITCHES:

String fields offer more than the ability to store a name, address or the like.
A  one byte string segment can be declared as a "switch"  This could  be  used,
for  example,  in a sales history datafile, to indicate whether or not  a  sale
was   pre-paid.   The name of the data segment might be:  PREPD?  (Y   or   N).
When entering records, you would enter a "Y" if the sale was prepaid, or a  "N"
if it was not.

CODES:

An expansion of the switch structure is the code.  A code occupying a two  byte
string on disk could have 416 possible meanings, working with the 26 lower case
letters,  only.  Rising to 2,704 if case is incorporated.  Both the switch  and
the code are ways to save disk space.  A paper record of the possible codes for
each  field should be prepared, and used to interpret the codes stored  in  the
records.

An  good  example of the proper use of a code is to identify the  source  of  a
part, as one of 127 possible vendors.  Space on disk is saved, since a  lengthy
field holding a company name is replaced by a one byte segment holding a  code,
but the datafile needs the code sheet to be read.  Of  course,  you could  also
record the record number detailing that vendor in another datafile.

GENERAL-PURPOSE FIELDS:

While  using the BASE program you may find it  convenient to write the  results
of some computation to disk, for  temporary storage.  You will need unassigned,
general-purpose numeric fields to hold the transient data.

COMMENTS:

There will probably be many times when you find yourself on the phone,  calling
people  in your datafile to some purpose or another.  You may be attempting  to
sell them something, you may be attempting to collect moneys owed you.

This situation, as well as many others, may produce data that won't really  fit
into  any  of  your pre-defined categories, such as:  This guy  is  simply  not
interested, I'm going to have to sue him to collect.

This  is the purpose of COMMENT FIELDS.  In the planning of your datafiles  you
should be able to anticipate the need of these fields.  Try not to use too many
as  this will cause undue waste of your available disk space.  But  use  enough
comment space to fill your need.

One  way to deal with really unpredictable situations of this nature is to  use
the APPLICATIONS MAKER program to produce a sheet of paper with the  customer's
name and other pertinent information on it.  Then jot down your notes.

FLAGS:

Another  use  of unassigned fields is for flags.  A flag is a message  that  is
left in the record, to indicate whatever you like.  A valid example of flag use
is  to  identify  newly entered records, so that they can be  sorted  into  the
survivor group after entry, to be processed in some manner.  Provided that  the
numeric format of the entry date of the record to the datafile is not  included
in  the record, (it seldom has any relevancy) there is no way for the  operator
to identify the newly entered records from records entered ten years ago.

An   unused field is selected as the flag and, when records are  entered,   the
value   held  in that field is set to a pre-determined value.  After  the   new
records are entered a sort is done to detect the presence of the flag value  in
the  flag field.  Only the flagged records are included in the survivor  group,
for exclusive processing of any type you wish.

You  must, of course, reset the flags to the "off" state when you are  finished
processing,  in order to prevent the newly entered records you  have  completed
being  included in the survivor group, the next time the sort for active  flags
is done.

If you combine the formats of flag and code, you can leave a very great  number
of   messages,  meaning anything you wish, in a two byte string  field.    But,
with  the  almost infinite flexibility of a program that  allows  user-designed
flagging processes,  you also inherit the responsibility of keeping track of it
all.  We recommend you start simple.

WRAP-UP:

The DATAMAGE data management system is a very loose, obliging and  co-operative
piece of software.  It is designed to store many types of FORMATTED data.  You,
the user, may specify the format of your datafiles, and manipulate then in  any
manner pertinent to your own purposes.

You  may  initialize  and fill with data as many datafiles  as  you  have  disk
storage to accommodate.  Each of these datafiles can exceed the capacity of any
disk  drive that can be connected to a PC computer.  The system WILL NOT  allow
the recording of individual datafiles on multiple disk drives, but will  access
datafiles recorded on any disk drive connected to your machine.