PC World 2000 February

home *** CD-ROM | disk | FTP | other *** search

/ PC World 2000 February / PCWorld_2000-02_cd.bin / Software / Servis / FFE / DATABASE.SWG / 0005_XBASE.pas < prev

Wrap

Pascal/Delphi Source File | 1997-05-11 | 58KB | 1,387 lines

#### #### ###### ## ## ## ## ## ## ## ## ###### ###### ##### ### ###### ## ## ## ## ## ## ## ## ####### ######## ####### ## ## ## ## ## ## ## ## #### #### ###### ######## ####### ##### Xbase file format description by Erik Bachmann Clickety Click Software 1997-03-10 WHAT IS XBASE "Generic term for the dBASE family of database languages. Coined in response to threatened litigation over use of the copyrighted trademark 'dBASE'." http://wombat.doc.ic.ac.uk/ Xbase is a complex of data files, indexes and eventually note files for storing large amounts of formatted data in a structured form. Basically Xbase is three different types of files: NN.dbf Data files NN.dbt Memo files (i.e. a separate file containing full text fields) *.ndx Index files pointing to a field in the *.dbf Xbase is almost compatible with dBASE and is actually a clone of dBASE. The creator of dBASE Ashton-Tate (and later Borland) has the copyright for the name dBASE, but NOT the structure. Therefore the name Xbase (It smells like - but it's not :-) has been widely used for look-alike data structures. There has been an attempt to create a standard for Xbase. I've found a lot of dead links while searching on Alta Vista's World Wide Web search engine. I'm at the moment not aware of the result of the Xbase comityÆs work. Xbase has - like most dBASE clones - it's offspring in the dBASE III+ file structure. I've tried to add all later enhancements as notes to the structure. --------------------------------------------------------------------------- Index * Top of document * WHAT IS XBASE * Index * WHY THIS DOCUMENT * WHAT SORT OF DATA CAN BE HANDLED * WHAT DO THE FILES LOOK LIKE * The structure of data files (*.dbf) * What to check when opening a .DBF File * The Structure of Index files * Search algorithm for index files * The Structure of Memo Field Files (*.dbt) * The Structure of Memmory Files (*.mem) * The Structure of Clipper Index Files (*.ntx) * The Record Lock Field on a dBASE IV Table * Read/write locks using share * dBASE II FILES * The structure of dBASE II data files (*.dbf) * The Structure of dBASE II index file * Examples * Description of test.dbf * Description of test.ndx * Description of test.dbt * Hex dump of test.dbf * Hex dump of test.ndx * Hex dump of test.dbt * dBASE Specifications * * HISTORY * REFERENCE LIST --------------------------------------------------------------------------- WHY THIS DOCUMENT I've been looking for a simple database tool for my work with C programming and bibliographical database systems. During my investigation of a lot of BBS's, FTP sites, toolboxes etc. I've discovered at lot of almost-ready-to-fly Xbase clones. This made me wonder why no one has collected a description of Xbase files AND made a simple package of C functions to handle this type of data files. THAT was my goal - and THIS is the result. --------------------------------------------------------------------------- NOTICE ! ! ! .----. ((*)(*)) DO *NOT* \ /\ / attempt to manipulate any critical database files #### *WITHOUT* making a proper and validated backup first. \__/ DO *NOT* access any database files in a multi-user environment! --------------------------------------------------------------------------- Any comments, corrections, additions etc. are welcome. You can reach me at: e_bachmann@geocities.com or by snail mail: Erik Bachmann, Grydeh°jvej 62, DK-4000 Roskilde, Denmark, Europe Third stone from the Sun - and turn left :-) --------------------------------------------------------------------------- WHAT SORT OF DATA CAN BE HANDLED Data files (*.dbf) contains a header with information on the data structure AND the record themselves. The data field is predefined as carrying data of a specific type ( Character, Numbers, Longs, Dates, Memo). Each data type has its limitations: ASCII text < 254 characters long. Character fields can be up to C Character 32 KB long (in Clipper and FoxPro) using decimal count as high byte in field length. It's possible to use up to 64KB long fields by reading length as unsigned. N Number ASCII text 18 characters long. Number fields can be up to 20 characters long in FoxPro and Clipper. Boolean/byte (8 bit) Legal values: ? Not initialised (default) Y,y Yes N,n No L Logical F,f False T,f True Logical fields are always displayed using T/F/?. Some sources claims that space (ASCII 20h) is valid for not initialised. D Date Date in format YYYYMMDD. A date like 0000-00- 00 is *NOT* valid. M Memo Pointer to ASCII text field in memo file 10 digits representing a pointer to a DBT block (default is blanks). F Floating (dBASE IV and later, FoxPro, Clipper) 20 digits point Character N/Aname 1-254 characters (64 KB in FoxBase and Clipper) variable B Binary (dBASE V) Like Memo fields, but not for text processing. G General (dBASE V: like Memo) OLE Objects in MS Windows versions P Picture (FoxPro) Like Memo fields, but not for text processing. Index files (*.ndx) are sorted references to fields in the data files. There might be more than one index for each data file. dBASE III Memo files are primitive 512 byte hunks of ASCII text data creating a way for text fields to exceed the 255 characters barrier of ordinary text fields. Other versions may store up till 64 KB of text in a fixed sized text field. Larger amounts of text (and binary data) is still stored in memo-like files. The size off the blocks can be user defined while creating the memo file (see byte 4-7 of the memo file structure). --------------------------------------------------------------------------- WHAT DO THE FILES LOOK LIKE They all look brown to me :-) All Xbase files have a header describing the structure of the data in the file. Schematically they look like this: The structure of data files (*.dbf) _______________________ _______ 0 | Version number *1| ^ |-----------------------| | 1 | Date of last update | | 2 | YYMMDD | | 3 | | | |-----------------------| | 4 | Number of records | Record 5 | in data file | header 6 | ( 32 bits ) *14 | | 7 | | | |-----------------------| | 8 | Length of header | | 9 | structure ( 16 bits ) | | |-----------------------| | 10| Length of each record | | 11| ( 16 bits ) *2| | |-----------------------| | 12| ( Reserved ) *3| | 13| | | |-----------------------| | 14| Incomplete transac.*12| | |-----------------------| | 15| Encryption flag *13| | |-----------------------| | 16| ( Reserved for | | _ |=======================| _______ | multi-user dBASE ) | | / 0| Field name in ASCII | ^ : ( dBASE III+ - ) : | / : (terminated by 00h) : | : : | | | | | 27| | | | 10| | | |-----------------------| | | |-----------------------| For 28| MDX flag (dBASE IV) *4| | | 11| Field type in ASCII | each |-----------------------| | | |-----------------------| field 29| Language driver *5| | / 12| Field data address | | |-----------------------| | / | *6| | 30| ( Reserved ) | | / | (in memory !!!) | | 31| *3| | / 15| (dBASE III+) | | |=======================|__|____/ |-----------------------| | <- 32| | | ^ 16| Field length (binary)| | | |- - - - - - - - - - - -| | | |-----------------------| | | *7 | | | | 17| Decimal count (binary)| | | |- - - - - - - - - - - -| | Field |-----------------------| | <- | | | Descriptor 18| ( Reserved for | | :. . . . . . . . . . . .: | |array 19| multi-user dBASE ) | | : : | | |-----------------------| | n | |__|__v_ 20| Work area ID | | |-----------------------| | \ |-----------------------| | n+1| Terminator (0Dh) | | \ 21| ( Reserved for | | |=======================| | \ 22| multi-user dBASE ) | | m | Database Container | | \ |-----------------------| | : *15 : | \ 23| Flag for SET FIELDS | | : : | | |-----------------------| | m+263 | | | 24| ( Reserved ) | | |=======================|__v_ ___ | : : | : : ^ | : : | : : | | 30| | | | Record structure | | | |-----------------------| | | | | \ 31| Index field flag *8| | | | | \_ |=======================| _v_____ | | Records |-----------------------| | | | | _ |=======================| _______ | | | / 0| Field deleted flag *9| ^ | | | / |-----------------------| | | | | / | Data *10| One | | | / : : record | |____|_____/ | | | : : | | | _v_____ : :____|_____ |=======================| : : | \ | Field deleted flag *9| | | | \ |-----------------------| | | | \ | | | | | \ | | | | | \_ |-----------------------| | | | |=======================| | |__End_of_File__________| ___v____ End of file ( 1Ah ) *11 *1 dBASE III+: 03h file without DBT. 83h file with DBT. i.e. bit 0-3 version, 3-5 SQL, 7 DBT flag dBASE IV: bit 0-2 Version no. 3 Presence of memo file 4-6 Presence of SQL table 7 DBT flag Examples: 03h dBASE III w/o memo file 04h dBASE IV or IV w/o memo file 05h dBASE V w/o memo file 83h dBASE III+ with memo file F5h FoxPro w. memo file 8Bh dBASE IV w. memo 8Eh dBASE IV w. SQL table 30h Visual FoxPro w. DBC *2 Sum of lengths of all fields + 1 (deletion flag) *3 (dBASE IV) Filled with 00h. *4 (dBASE IV) Production index / Multiple index file 01h MDX file present 00h no MDX file (index upon demand). (FoxBase) 01h CDX compound index file present, 00h no CDX file. (Visual FoxPro) 02h With memo 04h Database Container (DBC) 07h DBC (incl. memo & indexes) *5 (Foxpro) Code page: 01hDOS USA code page 437 02hDOS Multilingual code page 850 03hWindows ANSI code page 1251 C8hWindows EE code page 1250 *6 (FoxPro) 12-13. offset of field from beginning of record. *7 (FoxPro, Clipper) 16-17: Field length for non-numerical fields. Byte 16 is normally field length (0-255) and byte 17 represents the high byte for field length (256-65535). *8 (dBASE IV) Index field flag: 00h No key for this field 01h Key exists for this field *9 2Ah record is deleted, 20h record is NOT deleted *10 There are no field separators for record terminators. *11 dBASE II regards any End-of-File 1Ah value as the end of the file. dBASE III regard an End-of-File as an ordinary character, however it appends an extra End-of-File character at the physical end of the file. *12 (dBASE IV) Incomplete transaction 00h Transaction ended (or rolled back) 01h Transaction started *13 (dBASE IV) Encryption flag 00h Not encrypted 01h Data encrypted *14 Stored at binary i.e. value is generated as: byte#1 + (byte#2 * 256) + (byte#3 * 256²) + (byte#4 * 256³) ....... *15 (Visual FoxPro) Database Container (DBC) 263 bytes for backlist. Included in header structure. --------------------------------------------------------------------------- What to check when opening a .DBF File Records: * Length of record must be > 1 and < max length. (max length = 4000 B in dBASE III and IV, can be 32KB in other systems). * The number of records must be >= 0. Fields: * The .DBF file must have at least one field. * The number of fields must be <= the maximum allowable number of fields. File size: * File size reported by the operating system must match the logical file size. Logical file size = ( Length of header + ( Number of records * Length of each record ) ) --------------------------------------------------------------------------- The Structure of Index files The index file is a B+ tree. The essential structure is in an inverted tree, with an anchor note, multiple root nodes, and leaf nodes. The header is called the anchor node. --------------------------------------------------------------------------- NOTE! Normally the header is read once only by the application, when the file is opened. --------------------------------------------------------------------------- The size of a page is ( as far as I know ) always 512 bytes. _______________________ _______ 0 | Starting page no | ^ 1 | | | 2 | | | 3 | | | |-----------------------| | 4 | Total no of pages | File 5 | | header 6 | | | 7 | | (page 0) |-----------------------| | 8 | (Reserved) | | 9 | | | 10 | | | 11 | | | |-----------------------| | 12 | Key length | | 13 | | | |-----------------------| | 14 | No of keys per page | | 15 | | | |-----------------------| | 16 | Key type: 0 = char | | 17 | 1 = Num | | |-----------------------| | 18 | Size of key record | | 19 | | | 20 | | | 21 | | | |-----------------------| | 22 | (Reserved) | | |-----------------------| | 23 | Unique flag | | |-----------------------| | 24 | String defining the | | | key | | : : | __ |=======================| : : | / 0| Pointer to lower level| | | | / 1| (next page) | 511| | | / 2| | |=======================| _v_____ / 3| | 0| | ^ / |-----------------------| 1| | _|_____/ 4| Record number in | 2| Array of key entries | | 5| data file | 3| | Page 6| | 4| | | 7| | | | | |-----------------------| | | _|_____ 8| Key data | : : | \ : : : : | \ N| | 511| | | \_____ |=======================| |=======================| _v_____ --------------------------------------------------------------------------- Search algorithm for index files 1. The anchor node always resides in memory, while the file is open and determines which root node to access for at given key. 2. The root node is read and sequentially scanned until a key is found that is >= to the desired key. 3. A second-level node is accessed and scanned in a manner similar to the root node. 4. The process continues until the pointer to the next-lower-level node has a value of zero. If the key matches the leaf key at this level, the record number for the key is returned. That's a discipline NOT to be discussed here :-) Way out of the scope fore this document and way too complex. One of the advances of the B+ tree is that the higher levels of the tree does not have to be updated when a certain node is changed - unless the tree has become unbalanced. --------------------------------------------------------------------------- The Structure of Memo Field Files (*.dbt) _______________________ _______ 0 | Number of next | ^ ^ 1 | available block | | | 2 | for appending data | | Header 3 | (binary) | | | |-----------------------| _|__v__ 4 | ( Reserved ) | | | Size of blocks *1 | | | | | 7 | | | |-----------------------| | 8 | ( Reserved ) | | | | | | (i.e. garbage) | | : : First block : : | 511| | | |=======================| _v_____ 512| | ^ | | | | | 512 Bytes | | text blocks : : | : : | | | | |-----------------------| _|_____ | Field terminator (1Ah)| | ^ |-----------------------| | |Terminating field | Field terminator (1Ah)| | |within the block |-----------------------| _|__v__ : ( Unused ) : | 1023 : | |=======================| _v_____ | | ^ | | | | | 512 Bytes | | text blocks : : | : : | | | | | | _v_____ |=======================| *1) (FoxBase, dBASE IV ??) Size of blocks in memo file (SET BLOCKSIZE). Default is 512 bytes. A memo field can be longer than the 512 byte block. It simply continues through the next block. The field is logically terminated by two End-of-file marks in the field. The reminder of the block is unused. --------------------------------------------------------------------------- NOTE! Every time you re-write a memo field to the memo file in dBASE the field is APPENDED to the memo file (i.e. the file expands for each update). dBASE V for DOS (and perhaps others) may reuse the space from the deleted text. --------------------------------------------------------------------------- The Memo file itself tells nothing about it's use. You need the corresponding DBF file to interpret the content of the memo file. Many of the clones and dBASE 5 have binary elements stored in the memo file marked with file type B or G. --------------------------------------------------------------------------- .----. ((*)(*)) NOTICE ! ! ! \ /\ / #### DO *NOT* \__/ attempt to treat these binary data as simple text ! ! ! --------------------------------------------------------------------------- --------------------------------------------------------------------------- The Structure of Memmory Files (*.mem) _______________________ _______ 0 | | ^ ^ 1 | A null-terminated | | | : string containing : | Header 9 | the variable name | | | 10 | | | | |-----------------------| _|__v__ 11 | Variable type | | |-----------------------| | 12 | ( Reserved ) | | : : | 15 | | | |-----------------------| | <- 16 | Field length (binary)| | | |-----------------------| | | *1 17 | Decimal count (binary)| | | |-----------------------| | <- 18 | ( Reserved ) | | : : | : : | 31 | | _v_____ |=======================| *1 (FoxPro, Clipper) 16-17: Field length for non-numerical fields. Byte 16 is normally field length (0-255) and byte 17 represents the high byte for field length (256-65535). --------------------------------------------------------------------------- The Structure of Clipper Index files (*.ntx) The CA-Clipper index file is a modified B+ tree. The size of a page is 1024 bytes. The first page is the header: _______________________ _______ 0 | Signature byte | ^ 1 | 03h = Index file | | |-----------------------| | 2 | Indexing version no. | | 3 | | | |-----------------------| | 4 | Offset in file for | File 5 | first index page. | header 6 | | | 7 | | (page 0) |-----------------------| | 8 | Offset to an unused | | 9 | key page. | | 10 | | | 11 | | | |-----------------------| | 12 | Key size + 8 bytes | | *1 13 | | | |-----------------------| | 14 | Key size | | 15 | | | |-----------------------| | 16 | No. of decimals in | | 17 | key | | |-----------------------| | 18 | Key expression. | | 19 | (NULL terminated) | | : : | 277 | | | |-----------------------| | 278 | Unique index flag | | *3 |-----------------------| | 279 | (Unused) | | : : | : : | 1023| | | |=======================| _v_____ 0| | ^ : : | : : Page : : | __ |=======================| : : | / 0| No. of used entries |*4 | | | / 1| on current page | | | | / |-----------------------| | | | / 2| Array of unsigned | |=======================| ___|____/ 3| longs *5 | | | | : : : : | : : : : ___|____ : : |=======================| | \ N| | | | | \_____ |=======================| : : | |__End_of_File__________| ___v____ End of file ( 1Ah ) *1 (Clipper) Distance between key pages. *2 (Clipper) The first, or root page of an indes has a minimum of 1 entry regardless of this value. *3 (Clipper) 1: Unique 0: NOT unique *4 (Clipper) This number will bi between the minimum and maximum defined in the header unless it is the root page. *5 (Clipper) The array length is ewual to the maximum number of key entries per page +1. --------------------------------------------------------------------------- --------------------------------------------------------------------------- The Record Lock Field on a dBASE IV Table In a DOS multiuser environment, each user can place record locks on a shared table. For example, if user JSMITH is editing record number 12 of Stock, user MBROWN cannot access that record until it is unlocked. This prohibits one user from unintentionally overwriting another user's work. The dBASE table type gives you the Record Lock option to show you information about a locked record. If you check Record Lock, Database Desktop adds a hidden character field named _DBASELOCK to the table. This field shows you when a record was locked and by whom. --------------------------------------------------------------------------- NOTE: Although Database Desktop adds the Record Lock field to the table, you will not see it when you view the table. You see a record's Record Lock field only if you are locked out of that record. --------------------------------------------------------------------------- Use the Create dBASE Table dialogue box to create the Record Lock field for a dBASE table. Record Lock is not available for dBASE III+ tables. The information you see when you find a locked field depends on the Info Size you specify. The Record Lock field can be from 8 to 24 characters. The default is 16. 0-1 The first two bytes (binary integer) tell whether a user has changed the record. Every committed change is counted encreasing the count by one. 2-4 The next three characters tell the time a user placed the lock. (10h 09h 07h i.e. 16:09:07) 5-7 The next three characters tell the date a user placed the lock. ( 60h 09h 0Bh i.e. (19)96-09-11 ) 8-24 The remaining 16 characters are optional. They tell the name of the user that placed the lock. The default size of 16 displays the changed status of the record, the time and date of the lock, and the first 8 characters of the user who placed the lock. --------------------------------------------------------------------------- NOTE! You cannot see from the locking info if the user releases the record again. You have to test the record locking. On DOS systems SHARE has to be loaded. Turbo C, DJGPP and compatible C compilers have functions like: lock() _dos_lock() locking() for setting locks on a region of a file and unlock() _dos_unlock() unlocking for removing the lock again. It's important that YOU remove the locks BEFORE closing the file. --------------------------------------------------------------------------- Read/write locks using share by Phil Barnett (philb@iag.net) Share holds the locks, but it doesn't know anything about the structure or program that is requesting the lock. All share does is record the lock offset, type (record or file) and file name with path and where (station) it came from, and remove it when the correct station requests removal. Lets make a scenario. You have a 1000 byte header on your dbf file, and each record is 99 bytes + one for the delete flag. (this is to make the math simple) If you locked record one at the actual offset it would lock a range from byte 1001 to 1100. Any other station requesting READ or WRITE would be denied. Of course, we can read through locks in xBase, so how do they do it? The xBase languages add an arbitrary number to the actual offset. Originally, Clipper used 1,000,000,000 + record offset. Fox used another offset, and I am not sure but I think it was 4,000,000,000. We'll call this huge number the Lock Offset. Later, Clipper changed the Lock Offset to the same as Fox for certain RDD's. (.CDX and .IDX) For years, even though Clipper could create .NDX indexes, it was not Lock Offset compatible with dBase, and using both platforms on the same data caused incorrect locking. The Lock Offset numbers are arbitrarily large to keep actual data files from crossing the limit. (hopefully, we aren't creating any 1 trillion byte files) (or 4 trillion for newer schemes) So, in an xBase file, record locks and writes are at Lock Offset + File Offset and reads are at File Offset, so they do not run into a lock. This allows "read through" Locks between xBase systems are only able to be used concurrently when the arbitrary Lock Offset is the same in both systems. So, in the above example, to lock the first record the Lock Offset would be 4,000,001,001 to 4,000,001,100. If both languages used the same offset, then the write lock would be correctly enforced. --------------------------------------------------------------------------- These offsets are mentioned in online documentation (for CA-Clipper) (read 52eint.txt or clip53a.txt). The 1,000,000+RECNO() is used as default. If you want to use offset 4,000,000 you have to link the ntxlock2.obj file. For CDX indexes you should link cdxlock.obj to be compatible with Fox locking scheme (but only for Clipper != 5.2e) Confirmed by: r(ycho)GLAB, VIP Software (rglab@waw.pdi.net) --------------------------------------------------------------------------- For more information on record locking read : "Transactional Locking, Part 1" by Thomas Wang (wang@cup.hp.com) on http://www.concentric.net/~Ttwang/tech/locks1.htm --------------------------------------------------------------------------- dBASE II FILES The Structure of dBASE II data files (*.dbf) _______________________ _______ 0 | Version number *1| ^ |-----------------------| | 1 | Number of records | Record 2 | in data file (16 bit) | header |-----------------------| | _ |=======================| _______ 3 | Date of last update | | / 0| Field name in ASCII | ^ 4 | YYMMDD | | / : (terminated by 00h) : | 5 | | | | | | | |-----------------------| | | 10| | | 6 | Length of each record | | | |-----------------------| For 7 | ( 16 bits ) *2| | | 11| Field type in ASCII *4| each |-----------------------|--|--- | |-----------------------| field 8 | | | ^ | 12| Field length (binary)| | 9 | Field descriptor | | | ____/ |-----------------------| | : Array : | | 13| Field adress in memory| | : (Terminated by 0Dh) : | | 14| | | : : | | ______ |-----------------------| | : : | | \ 15| Field decimal count | | : : | | \_ |=======================| _v_____ | | | | 519| | | | |-----------------------| | | 520| Terminator *3|__v__v_ |=======================| | 521| Records | | : : | : : | : : | : : | n| | | |__End_of_File__________| ___v____ End of file ( 1Ah ) *11 *1 dBASE II version no: 02h *2 Sum of lengths of all fields + 1 (deletion flag) *3 0Dh if all 32 fields present, otherwise 00h. *4 Field type: C, N or L --------------------------------------------------------------------------- The Structure of dBASE II index file This was previously presented as the structure of dBASE III index files. Checking with the actual header of an index made it obvious, that it did NOT fit with dBASE III index files ( In that case several of the most essential key fields should be NULL values ). _______________________ _______ 0 | (Reserved) | ^ 1 | | | |-----------------------| | 2 | Node number of | | 3 | root node | | |-----------------------| | 4 | Node number of | Anchor 5 | next available node | node |-----------------------| description 6 | Length of key | | |-----------------------| | 7 | Size of key entry | | |-----------------------| | 8 | Maximum keys/node | | |-----------------------| | 9 | Field type | | |-----------------------| | 10| Key expression in | | | ASCII | | : : | : : | | | | 109| (00h terminated) | | |-----------------------| | 110| (Unused) | | | | | _ : : | / |=======================| : : | / 0| Pointer to lower level| | | | / 1| (next page) | 511| ( 16 bits ) | | / |-----------------------| |=======================| _v_____ / 2| Record number | 0| No.of keys in node | ^ / 3| | |-----------------------| _|_____/ |-----------------------| 1| Array of key entries | | 4| Key expression | | | Index | | | | record : : |-----------------------| _|_____ : : : : | \ N| | 511| | | \______ |=======================| |=======================| _v_____ --------------------------------------------------------------------------- Examples Description of test.dbf Name of data file : test.dbf Version no. : 5 Memo file present : 131 No. of records : 3 Length of header : 193 Length of records : 279 Date for latest change : 19960814 No. of fields : 5 Current record : 0 Crypted : no mdx/cdx file : no broken transaction : no Multi user information : 00 00 00 00 00 00 00 00 00 00 00 00 00 Field descriptions : Name Type length Decimal # ID N 5 0 # MSG C 254 0 # NOTE M 10 0 # BOOLEAN L 1 0 # DATES D 8 0 Description of test.ndx A index made on ID field in test.dbf Description of test.dbt A memo file containing the NOTE field from test.dbf --------------------------------------------------------------------------- 000000 83600811 03000000 C1001701 00000000 .`.............. 000010 00000000 00000000 00000000 00000000 ................ 000020 49440000 00000000 0000004E 0700DA3F ID.........N...? 000030 05000000 01000000 00000000 00000000 ................ 000040 4D534700 00000000 00000043 0C00DA3F MSG........C...? 000050 FE000000 01000000 00000000 00000000 ................ 000060 4E4F5445 00000000 0000004D 0A01DA3F NOTE.......M...? 000070 0A000000 01000000 00000000 00000000 ................ 000080 424F4F4C 45414E00 0000004C 1401DA3F BOOLEAN....L...? 000090 01000000 01000000 00000000 00000000 ................ 0000A0 44415445 53000000 00000044 1501DA3F DATES......D...? 0000B0 08000000 01000000 00000000 00000000 ................ 0000C0 0D202020 20203152 65636F72 64206E6F . 1Record no 0000D0 20312020 20202020 20202020 20202020 1 0000E0 20202020 20202020 20202020 20202020 0000F0 20202020 20202020 20202020 20202020 000100 20202020 20202020 20202020 20202020 000110 20202020 20202020 20202020 20202020 000120 20202020 20202020 20202020 20202020 000130 20202020 20202020 20202020 20202020 000140 20202020 20202020 20202020 20202020 000150 20202020 20202020 20202020 20202020 000160 20202020 20202020 20202020 20202020 000170 20202020 20202020 20202020 20202020 000180 20202020 20202020 20202020 20202020 000190 20202020 20202020 20202020 20202020 0001A0 20202020 20202020 20202020 20202020 0001B0 20202020 20202020 20202020 20202020 0001C0 20202020 20202020 20202020 20203120 1 0001D0 31393936 30383133 2A202020 20324E6F 19960813* 2No 0001E0 20322020 20202020 20202020 20202020 2 0001F0 20202020 20202020 20202020 20202020 000200 20202020 20202020 20202020 20202020 000210 20202020 20202020 20202020 20202020 000220 20202020 20202020 20202020 20202020 000230 20202020 20202020 20202020 20202020 000240 20202020 20202020 20202020 20202020 000250 20202020 20202020 20202020 20202020 000260 20202020 20202020 20202020 20202020 000270 20202020 20202020 20202020 20202020 000280 20202020 20202020 20202020 20202020 000290 20202020 20202020 20202020 20202020 0002A0 20202020 20202020 20202020 20202020 0002B0 20202020 20202020 20202020 20202020 0002C0 20202020 20202020 20202020 20202020 0002D0 20202020 20202020 20202020 20202020 0002E0 20202020 20325931 39393630 38313420 2Y19960814 0002F0 20202020 334D6573 73616765 206E6F20 3Message no 000300 33202020 20202020 20202020 20202020 3 000310 20202020 20202020 20202020 20202020 000320 20202020 20202020 20202020 20202020 000330 20202020 20202020 20202020 20202020 000340 20202020 20202020 20202020 20202020 000350 20202020 20202020 20202020 20202020 000360 20202020 20202020 20202020 20202020 000370 20202020 20202020 20202020 20202020 000380 20202020 20202020 20202020 20202020 000390 20202020 20202020 20202020 20202020 0003A0 20202020 20202020 20202020 20202020 0003B0 20202020 20202020 20202020 20202020 0003C0 20202020 20202020 20202020 20202020 0003D0 20202020 20202020 20202020 20202020 0003E0 20202020 20202020 20202020 20202020 0003F0 20202020 20202020 20202020 334E3139 3N19 000400 39363031 30321A 960102. --------------------------------------------------------------------------- Hex dump of test.ndx 000000 01000000 02000000 00000000 08001F00 ................ 000010 01001000 00000000 49442000 80028403 ........ID ..... 000020 31002B00 03000100 03000100 01000100 1.+............. 000030 01000100 01000100 01000100 01000100 ................ 000040 01000100 01000100 01000100 01000100 ................ 000050 01000100 01000100 01000100 01000100 ................ 000060 01000100 01000100 01000100 01000100 ................ 000070 01000100 01000100 01000100 01000100 ................ 000080 01000100 01000100 01000100 01000100 ................ 000090 01000100 01000100 01000100 01000100 ................ 0000A0 01000100 01000100 01000100 01000100 ................ 0000B0 01000100 01000100 01000100 01000100 ................ 0000C0 01000100 01000100 01000100 01000100 ................ 0000D0 01000100 01000100 01000100 01000100 ................ 0000E0 01000100 01000100 01000100 01000100 ................ 0000F0 01000100 01D3EA03 DABA5800 03D03BDA ..........X...;. 000100 770CB8FE FFA20E69 5F5E8BE5 5DCB33DB w......i_^..].3. 000110 8EC3B452 CD218CC0 0BC37410 268E47FE ...R.!....t.&.G. 000120 26803E00 004D7504 8C060069 8E060269 &.>..Mu....i...i 000130 8CC1268B 1E010026 8B160300 83EA58BE ..&....&......X. 000140 00008B3E 0669A100 69C70600 6900008E ...>.i..i...i... 000150 C03BC174 153BC774 1C263B1E 01007515 .;.t.;.t.&;...u. 000160 4626A103 00050200 03D0833E 00690075 F&.........>.i.u 000170 048C0600 6926803E 00004D75 0A8CC026 ....i&.>..Mu...& 000180 03060300 40EBC889 56FE8936 0C69F746 ....@...V..6.i.F 000190 06100075 6AF74606 0100745C 1EB067B4 ...uj.F...t\..g. 0001A0 35CD218C C88ED8BE 15A5BF0A 00B90800 5.!............. 0001B0 F3A61F75 43B440CD 670AE475 3BB446CD ...uC.@.g..u;.F. 0001C0 670AE475 33C86F7C 26D632D6 326A01A5 g..u3.o|&.2.2j.. 0001D0 3420007C 26E4326C 10881411 346431F0 4 .|&.2l....4d1. 0001E0 32643104 00FA3340 07881415 34643133 2d1...3@....4d13 0001F0 003000EB 3F494400 F7460610 00754BF7 .0..?ID..F...uK. 000200 03000000 00000000 01000000 00000000 ................ 000210 0000F03F 00000000 02000000 00000000 ...?............ 000220 00000040 00000000 03000000 00000000 ...@............ 000230 00000840 00000000 5F03C102 5F035101 ...@...._..._.Q. 000240 7B007C00 7D005101 6C036D03 57026E03 {.|.}.Q.l.m.W.n. 000250 AE025702 54025101 A7025101 FB015900 ..W.T.Q...Q...Y. 000260 31005101 0C025D03 5101C302 C402C502 1.Q...].Q....... 000270 C602C702 C802C902 CA025101 C5010802 ..........Q..... 000280 AE034201 9B000902 2F02DA00 0A02FD02 ..B...../....... 000290 06000800 09000A00 0B000C00 0D000E00 ................ 0002A0 0F000700 AB00AD00 AE00AF00 B000B100 ................ 0002B0 B200B300 B400AC00 C5011F03 51014201 ............Q.B. 0002C0 5101DD01 5101DA00 5101FD02 C5015101 Q...Q...Q.....Q. 0002D0 51014201 51015101 51015101 5101FD02 Q.B.Q.Q.Q.Q.Q... 0002E0 51015101 51015101 51015101 43035101 Q.Q.Q.Q.Q.Q.C.Q. 0002F0 5101FD02 D2025F03 DC025F03 E0025F03 Q....._..._..._. 000300 DB025F03 DF025F03 DD025F03 DE025F03 .._..._..._..._. 000310 5702E102 57025402 51015602 E202D302 W...W.T.Q.V..... 000320 5201D402 AE02AF02 B002B102 B202B302 R............... 000330 52015402 51015E00 5F006000 61005101 R.T.Q.^._.`.a.Q. 000340 5602D502 D3025201 D802D902 5201DA02 V.....R.....R... 000350 AF02B002 52015702 54025101 CB02CC02 ....R.W.T.Q..... 000360 CD02CE02 CF02D002 51015602 AD02AE02 ........Q.V..... 000370 AF02B002 B102B202 54025702 51015602 ........T.W.Q.V. 000380 D602D302 5201AE02 D7025402 57025101 ....R.....T.W.Q. 000390 80023100 51018002 84033100 51018002 ..1.Q.....1.Q... 0003A0 31005101 11015F03 51014800 49005101 1.Q..._.Q.H.I.Q. 0003B0 A600C102 51014C03 31005101 4C038602 ....Q.L.1.Q.L... 0003C0 31005101 56021B00 54021D00 56025201 1.Q.V...T...V.R. 0003D0 5F031C00 51025602 AE02AF02 5802B002 _...Q.V.....X... 0003E0 B1025002 52015702 54025101 50025402 ..P.R.W.T.Q.P.T. 0003F0 1E001F00 51022000 5602B102 B302B502 ....Q. .V....... --------------------------------------------------------------------------- Hex dump of test.dbt *1 *2 vvvvvvvv vvvvvvvv 000000 04000000 8701021D 03000002 5E126431 ............^.d1 000010 03000000 00000300 00008701 021D0300 ................ \ 000020 00025E12 64310300 00000000 02000000 ..^.d1.......... | 000030 8701021D 03000002 5E126431 03000000 ........^.d1.... | 000040 00000100 00006572 69632020 44617465 ......eric Date | 000050 20202020 204C6F67 6963616C 20204D65 Logical Me | 000060 6D6F2020 20202000 FF7604E8 26CF83C4 mo ..v..&... | 000070 0C5E5FC9 C3908B76 048B4606 50568976 .^_....v..F.PV.v | 000080 EA8946EC 900EE855 9B83C404 C45EEA2A ..F....U.....^.* | 000090 E4268A47 24C45E0A 268907C4 5EEA268A .&.G$.^.&...^.&. | Header 0000A0 47048E46 10268905 C45EEA26 F6471610 G..F.&...^.&.G.. | (mostly 0000B0 7503E9A3 00C45E0A 26832F02 5E5FC9C3 u.....^.&./.^_.. | garbage) 0000C0 6A01EB9A 16261F11 1F110300 C1009219 j....&.......... | 0000D0 00B8D80A BE197C26 16262411 24110300 ......|&.&$.$... | 0000E0 D500A619 00B8E20A D2197C26 BF186DD0 ..........|&..m. | 0000F0 0670A107 A1080672 CC026431 7C26BA19 .p.....r..d1|&.. | 000100 DC01CC19 D2196431 2F001100 21002F00 ......d1/...!./. | 000110 6D00D21A 0318CC19 21000300 D2196431 m.......!.....d1 | 000120 11002100 1E1B6431 03002020 20006163 ..!...d1.. .ac | 000130 74657200 5E5FC9C3 8B760023 2300268A ter.^_...v.##.&. | 000140 44032AE4 8CC1C45E 0A268907 8EC1268A D.*....^.&....&. | 000150 44028E46 10268905 5E5FC9C3 C8060000 D..F.&..^_...... | 000160 57568B4E 108B7E14 32C0F406 F10A1200 WV.N..~.2....... | 000170 F10A4400 8E1A3C1A F6006C22 08000F00 ..D...<...l".... | 000180 07000002 0A00246F 0A02DE13 D0367C26 ......$o.....6|& | 000190 2E018221 06727C26 0814820A F10A0700 ...!.r|&........ | 0001A0 0A00D036 00020A00 D0360000 00020700 ...6.....6...... | 0001B0 00060000 0000D136 2C14E707 F10A0F00 .......6,....... | 0001C0 10001900 D036726D 7C260000 07000000 .....6rm|&...... | 0001D0 00008C6D 08002A14 00007C26 9A018221 ...m..*...|&...! | 0001E0 46727C26 4614CB00 F10A0700 00000000 Fr|&F........... | 0001F0 00000700 0F060000 00008C6D 7C264E14 ...........m|&N. / 000200 54686973 20697320 61206D65 6D6F2066 This is a memo f \ <- 000210 6F726520 7265636F 7264206E 6F206F6E ore record no on | | *3 000220 651A1A11 C4D92E20 4C656176 65206D65 e...... Leave me |<- 000230 6E75202D 201B1A2E 20426C61 636B626F nu - ... Blackbo | 000240 61726420 2D204631 302E0052 616E6765 ard - F10..Range | 000250 2076616C 7565206D 75737420 6265206E value must be n | 000260 756D6572 69632E20 20507265 73732061 umeric. Press a | 000270 6E79206B 65792074 6F20636F 6E74696E ny key to contin | 000280 75652E00 2500EA08 00AA209A D0027F1D ue..%..... ..... | 000290 2400EA00 00911F9A D0027F1D 1100EA00 $............... | 0002A0 00451E9A D0027F1D 0800EA14 08D21D9A .E.............. | 0002B0 D0027F1D 1000EA00 00451E9A D0027F1D .........E...... | 0002C0 1900EA00 00D21D9A D0027F1D 0C00EA00 ................ | 0002D0 00D21D9A D0027F1D 1800EA00 00D21D9A ................ | 0002E0 D0027F1D 0A00EA00 00D21D9A D0027F1D ................ | 0002F0 1300EA00 0068219A D0027F1D 1200EA00 .....h!......... | 000300 0068219A D0027F1D 1400EA00 0068219A .h!..........h!. | 000310 D0027F1D 1600EA00 0068219A D0027F1D .........h!..... | 000320 1600EA05 0368219A D0027F1D 1500EA00 .....h!......... | 000330 0068219A D0027F1D 1200EA24 0968219A .h!........$.h!. | 000340 D0027F1D 1200EAC8 0E68219A D0027F1D .........h!..... | 000350 1200EA82 1268219A D0027F1D 1200EABD .....h!......... | 000360 0168219A D0027F1D 1200EAA8 1468219A .h!..........h!. | 000370 D0027F1D 1200EA7D 0068219A D0027F1D .......}.h!..... | 000380 1200EA3D 0068219A D0027F1D 1200EAB2 ...=.h!......... | 000390 0068219A D0027F1D 1200EA1A 0D68219A .h!..........h!. | 0003A0 D0027F1D 1200EAE3 1268219A D0027F1D .........h!..... | 0003B0 1200EA44 1368219A D0027F1D 2400EA35 ...D.h!.....$..5 | 0003C0 09911F9A D0027F1D 2400EA4F 0B911F9A ........$..O.... | 0003D0 D0027F1D 2400EA56 1E911F9A D0027F1D ....$..V........ | 0003E0 2400EAC7 1E911F00 00000000 00000000 $............... | 0003F0 BE000026 8B8C0400 41742E26 8B940800 ...&....At.&.... _| 000400 54686973 20697320 6D656D6F 20666F72 This is memo for \ <- Second 000410 20726563 6F726420 321A1A1C 00040000 record 2....... |<- entry 000420 E54F5445 30332020 4E445800 00000000 .OTE03 NDX..... | 000430 00000000 00002099 8C1F8945 00040000 ...... ....E.... | 000440 E54F5445 30202020 44425400 00000000 .OTE0 DBT..... | 000450 00000000 00009C94 821F9345 00040000 ...........E.... | 000460 E54F5445 30202020 44424600 00000000 .OTE0 DBF..... | Second 000470 00000000 00009C94 821F9445 28020000 ...........E(... | block 000480 E54F5445 30342020 4E445800 00000000 .OTE04 NDX..... | 000490 00000000 00002232 8D1F6E1C 00040000 ......"2..n..... | 0004A0 E5455820 20202020 20202000 00000000 .EX ..... | 0004B0 00000000 00004C32 8D1F8A45 FD2F0000 ......L2...E./.. | 0004C0 E545447B 45442020 45444320 00000000 .ED{ED EDC .... | 0004D0 00000000 00008532 8D1F0000 00000000 .......2........ | 0004E0 E545447B 45443120 45445420 00000000 .ED{ED1 EDT .... | 0004F0 00000000 00008532 8D1F0000 00000000 .......2........ | 000500 E545447B 45443120 45444220 00000000 .ED{ED1 EDB .... | 000510 00000000 00008532 8D1F0000 00000000 .......2........ | 000520 E545447B 45442020 45445320 00000000 .ED{ED EDS .... | 000530 00000000 00008532 8D1F8D45 00100000 .......2...E.... | 000540 00000000 00000000 00000000 00000000 ................ | 000550 00000000 00000000 00000000 00000000 ................ | 000560 00000000 00000000 00000000 00000000 ................ | 000570 00000000 00000000 00000000 00000000 ................ | 000580 00000000 00000000 00000000 00000000 ................ | 000590 00000000 00000000 00000000 00000000 ................ | 0005A0 00000000 00000000 00000000 00000000 ................ | 0005B0 00000000 00000000 00000000 00000000 ................ | 0005C0 00000000 00000000 00000000 00000000 ................ | 0005D0 00000000 00000000 00000000 00000000 ................ | 0005E0 00000000 00000000 00000000 00000000 ................ | 0005F0 00000000 00000000 00000000 00000000 ................ _| 000600 54686973 20697320 6D656D6F 20331A1A This is memo 3.. <- 3rd entry *4) *1) First 4 bytes indicates no. of next free block *2) (FoxBase) Size of blocks. This is a dBASE III+ file and byte 4-511 is just garbage. *3) First entry (terminated by 1Ah 1Ah) *4) NOTE! Last entry is NOT a complete block. dBASE does not write the full block to the memo file. The block contains the memo + terminator (1Ah 1Ah) --------------------------------------------------------------------------- dBASE Specifications dBASE II dBASE III dBASE IV Max. no. of records 1,000,000,000 1,000,000,000 Max. fields per record 128 255 Max. data files open 10 10 Max. index files per open database 7 47 Max. record size 4,000 bytes 4000 bytes Max. records per file 65,535 Max. bytes per file 8,000,000 Max. memory variables 256 Max. memory for variables 6,000 bytes *3 No. of data types 5 6 Max. size of MEMO file 5000 bytes 64000 bytes Max. character field size 254 bytes 254 bytes Max. numerical field size 19 bytes 20 bytes Size of logical field 1 byte 1 byte Size of date field 8 bytes 8 bytes Size of memo pointer in DBF 10 bytes Size of floating point - - 1-2 *3 Can be expanded to 31 KB using CONFIG, 6 KB is mentioned as default. --------------------------------------------------------------------------- HISTORY 1997-03-10 Still more references and updates of references. 1997-01-27 Description of .MEM (Memory index file) and .NTX (Clipper Index File). Description of Database Container in Visual FoxPro More references and various minor changes in text. 1997-01-09 Description of .MDX (Multiple index file) 1996-11-27 Document converted to HTML 1996-11-19 Minor fixes: Elements in Field Descriptor Array are exact 32 B long. Typing error in data type Float. "What to check on opening a .DBF file" added. 1996-10-18 Chapter on read/write locks using share by Phil Barnett (philb@iag.net) 1996-10-15 Change in description of structure of index files. Major mistake! The previous version described the structure for dBASE II index files. Only excuse: This mistake was made by several of my sources :-( 1996-10-03 First public version of this document (as far as I remember :-) --------------------------------------------------------------------------- REFERENCE LIST 1. Bachmann, Erik: Xbase : File format description / Erik Bachmann. - Roskilde, Denmark : Clickety Click Software, 1996-1997. - aprox. 17 pg. ; ASCII text file URL: http://www.geocities.com/SiliconValley/Pines/2563/Xbase.html or URL: http://www.wotsit.demon.co.uk/formats/dbf/xbasefmt.txt (ASCII Text) 2. CA-Clipper File Structures : Detailed Structure Information. . - ( Xbase Techniques. ; Computer Associates International, Inc. ). - 1993. - Pg. 7-10 Reference from: Ben Engbers 3. Castro, L.: Advanced programmer's guide featuring dBASE III and dBASE II / L. Castro. - Culver City, California, USA : Ashton-Tate, 1985. - 10+664 pg. - ISBN 0-912677-05-8 Note: Literature list and Terminology list 4. dBASE .DBF File Structure. - (Borland Technical Documents ; TI12821) URL: http://loki.borland.com/winbin/bds.exe?getdoc+2821+Delphi 5. Eacker, Brad: DBF : A set of tools and library routines to manipulate xbase files / Brad Eacker : Silicon Graphics, 1994. - Computer files ; C source ; ASCII text Note: UUNET vol. 43 1994 Abstract: UUnet : volume43 (Period: 08 Jun 94 - 24 Jul 94), dbf/part0[1-2] : dbf - xbase manipulation package URL: ftp://ftp.uu.net/usenet/comp.sources.misc/volumen43/dbf/part01 6. Err : "Not a Database File". - (Microsoft Knowledge Base. ; Article Q98743) URL: http://www.microsoft.com/kb/developr/fox/q98742.htm 7. Gmelch, Rasca: libdbf : simple routines for accessing xbase data files / Rasca Gmelch. - Berlin, 1996. - Computer files ; C source ; ASCII text URL: http://home.pages.de/~rasca/dbflib/ or URL: http://www.komm.hdk-berlin.de/home/rasca/dbflib-0.6/index.html 8. Hogan, T.: The Programmers's PC sourcebook : Reference tables for IBM PCs and compatibles, PS/2 systems, EISA-based systems, MS-DOS operating system through version 5, Microsoft Windows through version 3 : Hundred of new charts and tables / T. Hogan. - Redmond, Washington, USA : Microsoft Press, 1991. - Multiple pagination. - ISBN 1-55615-321-X 9. How to Modify the Table Header from FoxPro. - (Microsoft Knowledge Base. ; Article Q139758 ; ) URL: http://www.microsoft.com/kb/developr/fox/q139758.htm 10. Pereira, Pratap Xbase.pm v. 1.07 : a module for perl 5 Author Source: http://eewww.eng.ohio-state.edu/~pereira/software/xbase/ Author Address: pereira@ee.eng.ohio-state.edu 11. Simpson, Alan: dBASE III PLUS : Programmer's Reference Guide / Alan Simpson. - Almeda : Sybex Inc., 1987. - 1029 pg. - ISBN 0-89588-301-5 Abstract: Data file structure and memo file structure for xBase 12. Spence, Rick: Clipper Programming Guide / Rick Spence. - 2.nd edition. - Microtrend Books, Slawson Communications, Inc. - ISBN 0-915391-41-4 13. Stevens, A.: Turbo C : Memory-resistent utilities, screen I/O and programming techniques / A. Stevens. - Portland, Origon, USA : MIS Press, 1987. - 10+315 pg. - ISBN 0-943518-35-0 Note: Video Windows and TSR utility programs 14. Townsend, Carl: Mastering dBASE III : A Structured Approach / Carl Townsend. - Berkeley : Sybex Inc., 1985. - ISBN 0-89588-301-5 Abstract: Data file structure and index file structure for xBase 15. Walden, J.: File formats for popular PC software : A programmer's reference / J. Walden. - New York, New York, USA : Wiley, 1986. - 16+287 pg. - ISBN 0-471-83671-0 16. WANG, Thomas: Transactional Locking, Part 1 / Thomas Wang (wang@cup.hp.com) URL: http://www.concentric.net/~Ttwang/tech/locks1.htm 17. What FoxPro Checks When Opening a .DBF File. - (Microsoft Knowledge Base. ; Article Q119763) URL: http://www.microsoft.com/kb/developr/fox/q119763.htm --------------------------------------------------------------------------- END OF FILE ---------------------------------------------------------------------------