Copyright: FormIDEA for Microsoft Excel 97 Help. Copyright 1998. TechVision, Inc.
Citation: FormIDEA Help. Copyright 1998, TechVision, Inc. All right reserved.


  What Is FormIDEA for Excel?

TechVision FormIDEA for Microsoft Excel is a Form Image Document Enabled Add-in for Excel, which enables users to convert paper-based form documents into electronic formbooks and manage them with the combined power of FormIDEA and Excel.

FormIDEA is also an intelligent Form Image Data Entry Assistant for users who want to capture data on paper forms and store or transfer them in electronic formats using Excel.


What Intelligent Features does FormIDEA Add to Excel?

FormIDEA provides the following advanced document imaging features to enable the incorporation of Excel functionality with intelligent forms processing:

•	Form identification

•	Form registration (auto deskew, rotate, scale and align)

•	Data field image auto-tracking

•	Check box recognition (optical mark recognition or OMR)

•	Data entry verification

•	Form image document retrieval and review

These special features make it possible for Microsoft Office users to build electronic form image database (formbooks) that can be manipulated directly within Excel.


How does FormIDEA Work with Excel?

FormIDEA for Microsoft Excel consists of two basic modules: FormIDEA add-in and FormDefine. The FormDefine module is used to train your computer on the form type you want to process, define data items you want to read, set validation rules on data items, and map the data items to a formbook you want to create in Excel. The FormIDEA add-in, on the other hand, is built as an ActiveX automation server (Form Document Server) controlled by Excel, which adds intelligent forms processing services, such as input and verify data, index and review form documents, into Excel. It enables you to process form documents directly in Excel.

To get started, you can follow the steps described in “Previewing FormIDEA with Sample Forms” in this help file. It will help you appreciate how the FormIDEA is integrated seamlessly in Excel and how it behaves.


What is the Form Document Server?

The FormIDEA Form Document Server is an ActiveX automation server application that enables Excel to "automate" some specific functions provided by the FormIDEA add-in through an internal ActiveX programming interface. In this way, intelligent form document imaging services, such as form identification, registration, data image auto-tracking and data transfer from server to Excel spreadsheet (form sheet), including data attributes, validation rules, recognition results, can be made available to Excel users, as if they were originally provided by Excel.


 What Is Formbook?

FormIDEA’s formbooks are Excel workbooks created by FormIDEA, which inherit all workbook properties and features provided by Excel. The only difference between the workbook and formbook is that formbook can also communicate with the FormIDEA’s Form Document Server.

A new formbook is created when you start the Form Document Server to load a reference form for a form type you would like to process. The formbook becomes a workbook when you finish working with FormIDEA and shut down the Form Document Server.


Loading a Reference Form and Launching the Form Document Server

For a form type you want to process, normally you need to define and create a reference form using the FormDefine module. However, you can begin to learn how FormIDEA works without having a reference form created by yourself. There are three sample form types; each has a pre-defined reference form and a number of sample document forms that are located in your FormIDEA installation directory. You can follow the steps listed below to load a reference form and start processing document forms of the same type:

•	Select the Image Data Entry Assistant... menu item in the Excel Tools menu. This will activate the FormIDEA add-in toolbar.

•	Select New... in the Start menu on the FormIDEA add-in toolbar. In the opened file dialog box, locate one of the pre-defined sample reference forms by selecting its path and name, e.g.:

<FormIDEA Installaton Dir>\Reference\INFO.ref

then click Open. The Form Document Server will be launched, which shows the sample reference form you selected, together with a new formbook opened in Excel. The formbook contains a form sheet with automatically loaded column names and data validation rules provided by the Form Document Server.


Starting Processing Document Forms

Once the reference form has been loaded, it can be used to process document forms of the same type. To start processing, you should perform the following steps:

•	Click the Process menu button in the FormIDEA add-in toolbar. In the opened file dialog box, select a document form of the same type as the reference by specifying its path and name, e.g.:

<FormIDEA Installaton Dir>\Forms\INFOForm\INFO201.tif

then click Open. The form you selected will be loaded in the Form Document Server for processing. You will notice that all check boxes are automatically recognized and interpreted on the current working row of the form sheet.

•	Position the cursor to entries you’d like to key in data. You will see that the data images are automatically tracked and highlighted in the Form Document Server and controlled by the cursor positions in Excel.  The data image auto-tracking feature will help you to fast key in what you see on the screen. It will also help you to verify check box values that are interpreted automatically by the build-in OMR (optical mark recognition) engine.

•	Move the cursor to the next row on the form sheet then click the Process button (or press the Alt+p key on your keyboard) again. Since the default processing mode is set to Contiguous, it will load the next form (e.g. INFO202,tif) in the same directory for processing.

•	Repeat the last step above (i.e. move the cursor to the next row and click the Process button) to process the next form in the same directory.


Verifying Data or Reviewing Form Image Documents

You can verify data entry results or review form image documents for any recorded rows on the form sheet by performing the following steps:

•	Select the Review mode in the FormIDEA add-in toolbar.

•	Move the cursor to a row where you would like to verify the data or retrieve the form image document for review, then click the Process button. The form image document associated with that row will be retrieved.

•	Move cursor from entry to entry on that row to verify or review the data with displayed data images.


Understanding the Report Sheet

To help you track information for each form processed, the FormIDEA add-in records the processing status and form registration information automatically into a report sheet of the formbook.

The Form Registration Info tells you how FormIDEA matches each input document form with the reference to capture data images on the input form. The parameters used for form matching (or registration) include the form Skew, Offset, Scale, Orientation and Similarity to the reference. If the similarity value is above a specified threshold, the registration info is used to capture data images, otherwise, the input form will be marked as Rejected to call for your attention.

The underlying technology is the TechVision intelligent AccuForm engine inside the Form Document Server. You can customize optional settings of the server to fit special requirements for your data entry and processing tasks (see help topic on About the Form Document Server).


Shutting Down the Form Document Server

After finish processing your form documents, you can close the Form Document Server by clicking the Shut Down menu item in the FormIDEA add-in toolbar Start menu. The formbook created by FormIDEA becomes a standard workbook in Excel so you can continue working on it with any Excel provided functions. You can save your formbook and re-open it with FormIDEA Open... menu to continue adding more form entries of the same form type, retrieving form image documents or verifying data in Excel.


Loading / Unloading the FormIDEA Add-in

The FormIDEA add-in for Excel should be automatically loaded after you run the setup to install the FormIDEA add-in module. You can verify this in the Excel Tools menu where a new menu item Image Data Entry Assistant... should be added, which indicates that the FormIDEA add-in was already loaded.

To unload (or reload) the FormIDEA add-in module, click the Add-Ins... menu item in the Excel Tools menu, then uncheck (or check) the "formidea" box and click OK. Unloading the FormIDEA add-in will remove the Image Data Entry Assistant... menu item from the Excel Tools menu. Reloading the add-in will insert the Image Data Entry Assistant... item in the Excel Tools menu.


About the Form Document Server

The Form Document Server is an ActiveX automation server application that enables Excel to "automate" some specific functions provided by the FormIDEA add-in through an internal ActiveX programming interface. In this way, intelligent form document imaging services, such as form identification, registration, data image auto-tracking and data transfer from server to Excel spreadsheet (form sheet), including data attributes, validation rules, recognition results, can be made available to Excel users, as if they were originally provided by Excel.

To activate the Form Document Server, you can either create a new FormIDEA formbook in Excel by selecting the New... menu item in the FormIDEA add-in toolbar Start menu, or open an existing FormIDEA formbook by selecting the Open... menu item in the FormIDEA Start menu. In both cases, a pre-defined reference form for the working form type is required (see help topic on Creating Reference Form for FormIDEA in the FormDefine module).

Once the Form Document Server is launched, you can set processing options directly from the Form Document Server menu bar, which allows you to control the server behavior, such as form image viewing and form registration parameter setting.

When you finish your work with FormIDEA, you can close (shut down) the Form Document Server in two ways:

•	Save your formbook then close it. This will shut down the automation server automatically.

•	Click the Shut Down in the Start menu on the FormIDEA add-in toolbar. This will shut down the automation server but leave the formbook open as a standard Excel workbook so you can continue working on it with Excel functions.


Zooming Form Image

The zoom functions in the View menu allow you to view the form document image at different resolutions. When a form is first loaded, it is displayed at its full resolution. You can select the Zoom Out or Zoom In menu options to get lower and higher resolution views, respectively.


Adjusting the View Window

The default screen layout is horizontal, i.e. the form view window of the Form Document Server is displayed on the top half of the screen, and the Excel main window is displayed on the bottom. You can adjust the form view and Excel display window layout manually to maximize the form image viewing area.

The Toolbar and Status Bar menu items in the View menu toggle the display of the toolbar and status bar of the server’s view window. If your screen size is small or display resolution is low, you can increase the form viewing area by hiding the toolbar and status bar in the view window.

To show or hide the toolbar, status bar: On the View menu, click Toolbar, Status Bar, the screen will change to reflect your selection.


Preparing Document Forms

To prepare document forms for processing, you have to deal with different aspects of document form images. Below are some factors that you may need to consider:

•	Input Device

Form document images can come from different input devices, either scanners or fax machines. If you use FormIDEA in a stand along desktop application, you can use an inexpensive desktop scanner that usually has an attached document feeder, so you can scan about a dozen forms at a time. If you process large volume of forms in a client-server application, you may install a central scanner capable of scanning several pages in a second, and use FormIDEA in each client station to access form documents stored in the central server.

•	Image Orientation

Form documents can have two different page layout styles, i.e. portrait or landscape. When form documents are transferred into images, each page layout style can have different orientations. FormIDEA can detect and correct form image orientation only if forms are transferred in the same layout style. In other words, only page upside down can be corrected automatically. When prepare form documents for scanning or faxing, you should keep the form documents in their original page layout style, i.e., you can not scan or fax portrait form documents in landscape orientation or vice versa.

•	Image Format

Form images can be generated in any FormIDEA supported file formats (TIFF, PCX or BMP). To reduce the storage space, however, we strongly recommend you to use the TIFF format with CCTIT Group 3 or Group 4 compression.

•	Image Resolution

The scan resolution you choose only need to be good enough for document viewing and data verification purpose, which normally can be set in 100 dpi to 300 dpi range, depending on the form document size.

Ideally, all form images have the same resolution so that data images can be captured by the Form Document Server without performing scale match. In practice, however, form images may come from different input devices, it is difficult to ensure that the resolution of all document form images would be the same as the resolution that was set for scanning the reference form. To deal with the inconsistency in image resolution, FormIDEA has build-in intelligence that is capable of matching the difference in resolutions automatically.

•	Image Quality

Form images coming from scanners or fax machines are often less than perfect. Some distortions, such as image skew, misalignment, or noise may be added to images due to either imperfect scan mechanics or fax transmission.  To handle these problems which are common in processing existing form documents, FormIDEA provides intelligent forms processing features that can correct most form image distortions and maximize accuracy in data extraction.

•	Image Location

Once form images are generated from input devices, they are usually stored on the system disk. To take the advantage of the Contiguous or Batch mode processing provided by FormIDEA, we recommend you to get all your input forms well organized before you start processing. For example, you can add a subdirectory for each new form type under the Forms directory that was created by the FormIDEA installation program, and store all forms of the same type in that subdirectory.


Understanding Forms Processing Concepts

When a document form is loaded for processing, there is a sequence of operations performed in the Form Document Server:

•	Form Identification

In FormIDEA, form identification is based on advanced neural network and fuzzy logic technologies that do not require user input or any pre-knowledge about forms. Form documents are identified via global fuzzy analysis that compares similarities of the FGLS (Form Graphics & Line Structure) between the document forms and a target reference form. You can set a similarity threshold (warning level) in the Form Document Server to determine if an incoming form is of the same type as the reference, i.e., if it should be accepted or rejected.

•	Form Registration

Once the form has been identified, a complete registration process must be carried out to find the positional difference between the reference form and the input form. In FormIDEA, the following registration parameters are detected automatically for each incoming form, which requires no pre-printed anchor marks:

•	Skew angle, i.e. the degree of rotation difference between the reference form and the incoming form.

•	Orientation, forms may be scanned or faxed in different orientation. The Form Document Server can detect and correct form orientation, e.g., whether it's in normal position or upside down.

•	Scale factor, forms may be scanned in different resolution (dpi). The Form Document Server can detect the size difference between the reference and the incoming form so that the possible variations in scan resolution can be adjusted automatically.

•	Offset, i.e. the translation that is required to align the incoming form with the reference in both horizontal (x) and vertical (y) directions.

•	Data Image Extraction

After form registration parameters are detected for each input form, data images can be extracted by applying the geometric transformations on each data region defined on the reference form. Consequently, data images can be automatically tracked and sent to different recognition components for further processing.


Skew Angle Settings

Skew Angle specifies the maximum allowance of possible skew in your document form images. The Form Document Server can rotate the data regions automatically within the specified maximum angle to correct the document image skew. However, this does not imply that you should always select the largest value for form images that actually have small skew. Try to set the value just large enough to cover the maximum skew in your form images because the smaller the skew allowance you set, the faster the form matching process you can archive.

The default Skew Angle setting is +/- 3 degree. To change the skew angle settings, go to the Option… menu; on the Skew Angle page, set the maximum skew allowance, then click OK.


Scale Match Settings

Form images that are of the same type may be different in resolutions (dpi). The Form Document Server can detect the size difference between the reference and the incoming form so that the possible variations in scan resolution can be adjusted automatically.

Select the Scale Match option if you are not sure whether the form document images are all in the same resolution. If they are, however, you may uncheck (disable) the Scale Match to speed up the registration process.

To change the scale match settings, go to the Option… menu; select the Scale Match page, then check or uncheck the Auto Match check box, and click OK.


Orientation Detection Settings

Select this option if automatic upside down detection and correction are desired.

The default Correct Upside Down setting is ON. However, if all your document forms were presorted and scanned in normal position, you may uncheck this option to speed up the form registration (matching) process.

To change the orientation settings, go to the Option… menu; on the Orientation page, check or uncheck the Correct Upside Down box, then click OK.


Warning Level Settings

The Warning Level specifies the confidence level (similarity threshold) in form identification.

When a document form is loaded for processing, the Form Document Server will first identify the form type by performing global fuzzy analysis on the similarity of the FGLS (Form Graphics & Line Structure) between the document form and the target reference. Form documents may be rejected if their similarity value is below the specified similarity threshold (warning level).

There are three warning levels you can choose from the Warning Level page in the Option… menu:

•	Level 1: Similarity < 50%

•	Level 2: Similarity < 65%

•	Level 3: Similarity < 80%

In the Single or Contiguous processing mode, a warning will be displayed in dialog box if the similarity value is below the setting. You can determine if the form should be accepted, rejected, or you want to retry with auto-match. If you are working with the Batch processing mode, warnings or errors will not be displayed but will be logged into the report sheet in your formbook.


Show / Hide the FormIDEA Add-in Toolbar

The FormIDEA add-in toolbar allows you to use the functions that FormIDEA adds in Microsoft Excel. When the FormIDEA add-in is loaded, you can pop up the FormIDEA add-in toolbar by selecting the menu item Image Data Entry Assistant… in Excel Tools menu.

You can hide the FormIDEA add-in toolbar by simply closing it. To show the toolbar again, you can select the Customize...  in the Excel Tools menu, then check the FormIDEA box.


Creating a New Formbook

For each new form type you want to start working with, you can create a new FormIDEA formbook in Excel by loading a pre-defined reference form of that type.

To create a new formbook, select New... in the Start menu on the FormIDEA add-in toolbar. In the opened file dialog box, locate a reference form by selecting its path and name, then click Open. This will create a formbook that has a form sheet loaded with column names and data validation rules. It will also launch the Form Document Server, load the reference form and link the reference with the formbook.


Opening an Existing Formbook

If you have a FormIDEA formbook saved, and you want to continue your data entry work, verify data entry results or retrieve form image documents from the formbook, you can re-open it by selecting the Open... in the FormIDEA add-in toolbar Start menu.

Opening an existing FormIDEA formbook will activate the Form Document Server and load the reference form associated with the formbook.


Single Mode

The Single processing mode allows you to specify the form location in a file dialog box for each form entry.

To work with the Single processing mode, select Single in the combo box displayed on the FormIDEA add-in toolbar.  Then, every time when you press the Process button, a file dialog box will appear, which allows you to select a single form image for processing.


Contiguous Mode

The Contiguous mode is the default processing mode that allows you to load the next form image stored in the same directory automatically.

In the Contiguous mode, the first time you press the Process button, a file dialog box will be displayed, which allows you to select the first form image in a specified directory for processing. When data entry is completed on the current form, the next form image in the same directory will be loaded automatically every time when you press the Process button again. You can repeat this process until all forms in the same directory are completed.


Batch Mode

The Batch processing mode allows you to process all form documents stored in the same directory automatically. The Form Document Server will continuously look at the same directory for form image documents. If form image files present, they will be loaded one by one for processing.

In the Batch mode, form identification and registration are performed. The resultant information will be logged into the report sheet in your formbook. Check boxes, if any, will be automatically recognized, but all text fields on the form sheet will be left as blanks. This means that you may not benefit from batch mode processing unless your form documents have some check boxes that you want to read them first. In this case, you can fill in blanks, if any, with the Review mode after batch processing; i.e. you can retrieve them back at any time to key in data with the image auto-tracking assistance.

To work with the Batch processing mode, select Batch in the combo box displayed on the FormIDEA add-in toolbar.  Then press the Process button, a file dialog box will appear, which allows you to select a directory and the first form document in batch for processing.


Review Mode

The Review mode allows you to retrieve form document images which processing results were already recorded in your form sheet. You can use the Review mode to verify the correctness of data entries or review form documents directly from selected data entries in the form sheet.

To work with the Review mode, select Review in the combo box displayed on the FormIDEA add-in toolbar, point a row where you would like to verify the data entry results or review form image document, then click the Process button. The form image associated with that row will be retrieved and data images will be auto-tracked when you position cursor in an entry, so you can review, verify or modify data easily.


Keying Data with Image Auto-Tracking

Image auto-tracking is activate in every processing mode except the Batch. This means that you can perform fast data entry with the image auto-tracking assistance in Single, Contiguous, or Review mode.

After a form image is loaded, the corresponding row in the form sheet will be highlighted, which indicates that the current row is active. If you move the cursor from entry to entry on that row, you will see data images requested for those entries will be automatically tracked, highlighted by the Form Document Server, so you can simply key what you see on the screen. You can adjust image zooming to get the best fit between form images and screen display resolutions.


Checking OMR Results

When a form image is loaded for processing, check boxes on the form will be recognized automatically by the OMR (optical mark recognition) module that is built in the Form Document Server. The check values are automatically interpreted and entered in your form sheet. You can check the recognition results against the displayed check box images by position cursor to the entries that are associated with the check boxes.

Each check box entry in the form sheet is actually defined as a drop down combo box with a drop down handle on the right side, which contains all possible choices for that check box entry. If a recognition error occurs in a check box entry, you only need to select a right choice from the drop down menu.


Retrieving Form Documents and Verifying Data

When you select the Review mode in the FormIDEA add-in toolbar and press the Process button, the Form Document Server will retrieve the form document image for the selected row in your form sheet.

To verify data entries, move the cursor from entry to entry and check the results against the auto-tracked and highlighted data images on the screen.


Scrolling

The image scrolling allows you to control the image display position in the Form Document Server directly from Excel, so you do not need to switch to the Form Document Server every time when you want to scroll the image.

To scroll image directly in Excel, select the Image menu on the FormIDEA add-in toolbar, then click the Up/Down or Left/Right menu term. Usually, this can be done more conveniently by using short cut keys while you are keying data on the keyboard.


Zooming

The image zooming allows you to control the image display resolution in the Form Document Server directly from Excel, so you don not need to switch to the Form Document Server every time when you want to change the image display.

To zoom image directly in Excel, select the Image menu on the FormIDEA add-in toolbar, then click the Zoom In or Zoom Out menu term. Usually, this can be done more conveniently by using short cut keys while you are keying data on the keyboard.


From Sheet
Form Sheet

Like workbooks, a formbook can consist of any number of separate sheets, such as worksheets, chart sheets or macro sheets, but it also has its own unique form sheet and report sheet that interact with the Form Document Server.  The form sheet is a special worksheet in the formbook, which is associated with a particular form type you want to process.

The form sheet structure is determined by the reference form that you defined for a form type. That is, its column names, data types and data validation rules are loaded automatically when you create or open a formbook with the FormIDEA add-in.

There are additional columns in the form sheet (colored in gray) that are appended to the form definition columns, including document and reference form locations, form sheet ID, etc. The contents of these columns are filled automatically by the FormIDEA add-in and used internally. Unless you have to move your source document or reference location, you may not change the contents recorded in these columns. Otherwise, it may cause error or incorrect behavior when you re-open the formbook with the FormIDEA add-in.

You can work on the form sheet with any Excel provided functions for data calculation, analysis, graphics charting, etc., but you may not change the form sheet structure by inserting or deleting any form definition columns without changing the form definition in its reference file accordingly.


Report Sheet

To help you track information for each form processed, the FormIDEA add-in logs the processing status and form registration information into a worksheet in you formbook, called report sheet.

The information logged includes:

•	Path: 	Form location

•	Name: 	Form name

•	Status:	Processing status

•	Registration Info:	Form registration information

•	Registration Settings:	Form registration settings

•	Date:	Processing date

•	Time:	Processing time

The Form Registration Info tells you how FormIDEA matches each input document form with the reference to capture data images on the input form. The registration result and parameters used for form matching include the form Similarity, Skew, Offset, Scale and Orientation to the target reference.

The Status indicates whether the input form is accepted or not. The possible value of the processing status is Succeeded (form similarity is above the threshold), Rejected (form similarity is below the threshold), or Failed (other input errors, e.g. unsupported image file formats).

The Registration Settings record the form registration options you set in the Form Document Server, which include Max_Skew  (maximum skew), Auto_Orient (correct upside down), Auto_Scale (scale match), and S_Threshold (warning level). This information may help you to analyze the processing results and determine if you need to make any necessary changes in these options.