Batch Indexing and Searching

dtSearch batch scripts allow you to automate indexing and searching tasks. A script is a text file that lists a series of indexing commands that dtSearch will carry out.

Batch scripts were used in dtSearch 5 to automate index updates and other tasks. In dtSearch 6, integration with the Windows Task Scheduler provides an easier way to automate index updates, and the dtSearch developer API provides a better way for Visual Basic, C++, Delphi, and other developers to work with the dtSearch Engine. However, some dtSearch 5 users have developed libraries of scripts that they use to maintain and work with their indexes. To provide a way for dtSearch 5 users to continue to use their batch scripts, dtSearch 6 comes with the scriptrun utility.

Running a Batch Script

The scriptrun utility can be run from the command prompt from the dtSearch Engine BIN folder. To use scriptrun, just type "scriptrun", a space, and the name of the batch script. If the batch script name contains a space, it must be quoted, like this:

scriptrun "A script with a space.job"

Any errors encountered during the job will be recorded in a file with the same name as the script but with the extension LOG.


Batch Scripts

A script consists of a series of SET statements that specify the documents to be indexed and the index to use, and commands such as ADD and COMPRESS. The first line of a script must consist of the words DTSEARCH SCRIPT. dtSearch ignores any line beginning with a *, so you can include comments in a script. Here is an example:

DTSEARCH SCRIPT

* Create the index

SET INDEXPATH = "c:\MyIndex"

CREATE

* Add documents from C:\DOCS

SET DOCPATH = C:\DOCS

SET FILTER = *.DOC *.TXT *.WPF

ADD

* Compress the index

COMPRESS

This script would: (1) create an index called "MyIndex," (2) add to the index documents in the directory C:\DOCS, and (3) compress the index. (You can combine several different indexing and searching operations in a single script.) To run this script, put it in the file UPDATE.JOB in the dtSearch BIN directory and execute the command:

scriptrun update.job

The scriptrun utility will execute the script, logging any error messages to the file UPDATE.LOG.


Batch Indexing Commands

Creating an Index

To create an index, set up a script like this:

DTSEARCH SCRIPT

SET INDEXPATH = C:\DOCINDEX

CREATE

There are two options you can use with CREATE: CASE and ACCENT. Use CASE if you want the index to be case-sensitive. Use this ACCENT if you want the index to be accent-sensitive. To use one of these options, simply list them on the same line as the CREATE command.

WARNING: If you create an index that already exists, dtSearch will delete the old index and create a new, empty index in its place. In interactive mode, you will get a warning message before this happens, but in batch mode dtSearch assumes that you really want to create a new index.

Adding Documents to an Index

To add documents to an index in a batch script, you need to select an index to add to, list the directories to index, and list the file name filters you want to use. Use either the path (SET INDEXPATH) to specify the index. Use SET DOCPATH = to tell dtSearch the directory in which the documents are located. Add a <+> to the end of a directory name to index subdirectories of a directory:

SET DOCPATH = C:\DOCS<+>

Use quotation marks around directory names that include spaces. If the list takes more than one line, end each line except for the last with a backslash ("\"). To specify file name filters, use SET FILTER =. Use SET EXCLUDE = to specify files that you do not want to index. After you have provided the information dtSearch needs to index your documents, use ADD to add the documents to the index. Example:

DTSEARCH SCRIPT

SET INDEXPATH = "c:\MyIndex"

SET DOCPATH = C:\DOCS C:\DOCS2 "C:\Long Directory Name<+>"

SET FILTER = *.DOC *.TXT *.WPF

SET EXCLUDE = A*.DOC

ADD

Compressing an Index

To compress an index, use SET INDEXPATH to specify the index to compress and then use the COMPRESS command. Example:

DTSEARCH SCRIPT

SET INDEXPATH = "c:\MyIndex"

COMPRESS

Verifying an Index

To verify an index, use SET INDEXPATH to specify the index to verify, and then use the VERIFY command. The index log will record the results of the Verify operation (either "PASSED" or "FAILED").

DTSEARCH SCRIPT

SET INDEXPATH = "c:\MyIndex"

VERIFY

Merging Indexes

To merge a group of indexes, use SET INDEXES to specify the indexes to merge, and then use the MERGE command. The indexes will all be merged into the first one listed. To clear the target index before the merge (so the resulting index will contain only the second and following indexes) add CLEARTARGET after the MERGE command

DTSEARCH SCRIPT

SET INDEXES = "c:\Indexes\First" "c:\Indexes\Second" "c:\Indexes\Third"

MERGE

or

MERGE CLEARTARGET


Batch Searching Commands

Searching

The following is an example of a batch script that would perform an indexed search:

DTSEARCH SCRIPT

SET REQUEST = apple and pear and banana

SET INDEXES = "c:\Index1" "c:\Index2" "c:\Index3"

SET SORT = HITS

SEARCH

SET REQUEST tells dtSearch what you want to search for.

SET INDEXES tells dtSearch which indexes you want to search.

SET SORT specifies how the search results will be sorted. Options are:

HITS Sort by hits.

NAME Sort by name.

DATE Sort by date.

NONE Unsorted.

You can use ASCENDING or DESCENDING after the type of search to specify an ascending or descending sort. For example:

SET SORT = NAME DESCENDING

By default, name sorts are ascending and hit and date sorts are descending.

Search Results

To view your search results, create a listing file, which is simply a list of the retrieved documents in ANSI text format. Listing files are useful if you want to use another program to process the results of your search. To create a listing file, add a > and the name of the file to create after the SEARCH command, like this:

SEARCH > listing.txt

To append your search results to an existing file, use >> instead of >, like this:

SEARCH >> listing.txt

The listing file will contain a header with your search request, the date of the search, and the number of files retrieved. To suppress this header, add SET HEADER = NO to your script before the SEARCH command.

Ordinarily, a listing file will contain only the names of the retrieved files. If you want the number of hits in each file to be included in the file, add COUNT after the SEARCH command, like this:

SEARCH COUNT > listing.txt


Batch Listing Commands

The LIST command will list words in an index, files in an index, information about an index, or the text of a file. By default, the output of a LIST command will go into your LOG file. To redirect it to another file, use the same > and >> symbols used in the SEARCH command, like this:

LIST WORDS > words.lst

LIST WORDS

LIST WORDS will create a list of all of the words in the currently selected index. Example:

SET INDEXPATH = c:\MyIndex

LIST WORDS > words.lst

LIST WORDS COUNT will list, next to each word, the number of times it occurs in the index.

To selectively list words in an index, use a filter, like this:

LIST WORDS MATCH "APPL*" > words.lst

This will list all words that match appl*. The filter can contain wildcards like * and ?, the phonic search character #, the stemming character ~, and the fuzzy search character %. For example,

LIST WORDS MATCH "AP%%PLICATION"

LIST WORDS MATCH "#APPLICATION"

The first command would list all words that match a fuzzy search for application. The second command would list all words that sound like application.

LIST FILES

LIST FILES works exactly like LIST WORDS except that it lists filenames instead of words. The > and >> symbols work the same way, and you can use the same types of filters. For example,

SET INDEXPATH = c:\MyIndex

LIST FILES MATCH "SMITH*.DOC" > smith.lst

This would list all files in MyIndex whose name matches SMITH*.DOC.

LIST INFO

LIST INFO lists the information about an index that appears in the Index Information dialog box. This includes the date the index was created and last modified, the number of documents in the index, the size of the index, etc. Example:

SET INDEXPATH = c:\MyIndex

LIST INFO

LIST TEXT

LIST TEXT provides an easy way to extract the text from a file that is in a format that dtSearch recognizes. Example:

LIST TEXT smith.doc > smith.txt

If SMITH.DOC is a WordPerfect file, SMITH.TXT would contain the text of that file as it would appear in dtSearch. Another example:

SET INDEXES = c:\MyIndex

SET REQUEST = apple w/5 pear

SET RESULTS = results.dat

SEARCH

REPORT results.dat report.dat 30

LIST TEXT report.dat > report.txt

This script would search MyIndex for apple w/5 pear, write the results to RESULTS.DAT, prepare a search report based on RESULTS.DAT, and then extract the text of the search report to REPORT.TXT.


Other Batch Commands

QUIT terminates execution of a script and also causes dtSearch to exit.

The option command from dtSearch 5 is no longer supported.


Batch Language Reference

Words in capital letters are keywords. Items in braces -- { and } -- are optional. The > filename parameter means that output from the command can be redirected to a file. Use >> instead of > if you want the output to be appended to an existing file.

If a filename or directory name contains a space, use quotation marks around the name. Example:

SET DOCPATH = "C:\1994 Documents"

If a command is too long to fit on one line, use the backslash ("\") to continue the command on the next line. Example:

SET REQUEST = apple or orange or grape or \

pear or banana

If you include an environment variable in a batch script command, dtSearch will substitute it with the value of the variable in the current environment. For example, if the variable %DTSDIR% has the value C:\DTSEARCH, then a script could include the line:

SEARCH > %DTSDIR%\RESULTS.LST

dtSearch would interpret this to mean:

SEARCH > C:\DTSEARCH\RESULTS.LST

ADD

Begin adding documents to an index. Use SET INDEXPATH to specify the index, SET FILTER to specify filename filters, and SET DOCPATH to specify directories.

COMPRESS

Compress an index. Use SET INDEXPATH to specify the index to compress.

CREATE {CASE} {ACCENT}

Create a new index. Use SET INDEXPATH to specify the directory. Options:

CASE Create a case-sensitive index.

ACCENT Create an accent-sensitive index.

LIST FILES {MATCH "filter"} > filename

List files in an index. Use SET INDEXPATH to specify the index. The optional filter, if present, will select the files to be listed.

LIST INFO > filename

List information about an index (the information that is displayed in the Index Information dialog box). Use SET INDEXNAME or SET INDEXPATH to specify the index.

LIST WORDS {COUNT} {MATCH "filter"} > filename

List words in an index. Use SET INDEXPATH to specify the index. The optional filter, if present, will select the words to be listed. The filter can include wildcards (* and ?), fuzzy (%) and phonic searching (#), and stemming (~).

LIST TEXT filename1 > filename2

List the text of filename1

MERGE {CLEARTARGET}

Combine a group of indexes into a single index. Use SET INDEXES to specify the indexes to merge. The first one listed will be the target for the merge. Use CLEARTARGET to clear the target index before the merge.

OPTION option-text

Change a user option in the dtSearch OPTIONS.DAT file. The text following the word OPTION is treated as if it were a line in OPTIONS.DAT.

QUIT

Exit dtSearch.

REMOVEDELETED

Check each file in an index to see if it has been deleted and, if it has, remove it from the index. Use SET INDEXNAME or SET INDEXPATH to specify the index.

SEARCH {COUNT} {HITS} > filename

Perform a search. Use SET INDEXES to specify the indexes to search, SET REQUEST to specify the search request, SET RESULTS to specify the name of the search results file, and SET SORT to specify how the results should be sorted. SEARCH > filename will list the full name of each retrieved document. COUNT adds the number of hits in each file to the listing. HITS adds a list of the word offsets of the hits in each file to the listing.

SET DOCPATH = path1 path2 ...

Specify the directories to be indexed. If the list extends over more than one line, each line (except the last) should end in a backslash ("\") so dtSearch knows that the list continues on the next line. Add <+> to a directory name to index the directory and any subdirectories. Directory names with spaces in them must be quoted.

SET EXCLUDE = filename filters

Specify files not to be indexed. More than one filter can be included (use spaces to separate them).

SET FILTER = filename filters

Specify files to be indexed. More than one filter can be included (use spaces to separate them).

SET HEADER = YES|NO

A search listing will contain a header with your search request, the date of the search, and the number of files retrieved. To suppress this header, use SET HEADER = NO.

SET INDEXES = list of indexes

Select indexes to be searched. List the names (in quotation marks) of each index that you want to search.

SET INDEXNAME = name

Specify the name of an index to update or create. This was supported in dtSearch 5 and is now obsolete. Only SET INDEXPATH should be used to specify an index.

SET INDEXPATH = path

Specify the directory of an index to update or create.

SET REQUEST = search request

Enter a search request for a SEARCH command.

SET SORT = HITS|DATE|NAME|NONE ASCENDING|DESCENDING

Specify how you want the results of a search to be sorted.

VERIFY

Verify that an index is not corrupt.

WINDOW MAX|MIN|NORMAL

Change the appearance of the dtSearch main window.