Programmer's ROM - The Computer Language Library

home *** CD-ROM | disk | FTP | other *** search

/ Programmer's ROM - The Computer Language Library / programmersrom.iso / ada / metric / sinstool.doc < prev next >

Wrap

Text File | 1988-05-03 | 245.2 KB | 9,404 lines

--:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: --testtool.ug --:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: Ada* Test and Analysis Tools ( ATEST ) Version 1.0 User's Guide by Mary Koppes Intermetrics, Inc. * Ada is a trademark of the U.S. Department of Defense (AJPO) Ada Test and Evaluation Tools User's Guide Page: i PREFACE This program was developed by Intermetrics, Inc. under Contract Number N66001-85-C-0032 for: Naval Ocean Systems Center 271 Catalina Boulevard San Diego, California 92152. Ada Test and Evaluation Tools User's Guide Page: ii CONTENTS 1.0 INTRODUCTION ...................................... 1 2.0 ATEST SYSTEM OVERVIEW ............................. 2 3.0 USER INTERFACE INFORMATION ........................ 3 4.0 ATEST TOOL OVERVIEWS .............................. 4 4.1 Source Instrumenter ............................. 4 4.2 Path Analyzer ................................... 5 4.3 Performance Analyzer ............................ 6 4.4 Self-Metric Analysis and Report Tool ............ 7 4.5 Automatic Path Analyzer ......................... 8 5.0 TEST TOOL EXECUTION ............................... 9 6.0 RESTRICTIONS AND LIMITATIONS ...................... 10 APPENDIX A - GLOSSARY .................................. A-1 ATEST User's Guide Page: 1 1.0 INTRODUCTION The purpose of this document is to provide the user of the Ada Test and Evaluation Tools (ATEST) with an overview of the entire system. This document presents concise instructions to a novice user on the required steps of testing an Ada program using the ATEST system. The ATEST system is intended for the test and evaluation of Ada programs. The tools specifically support: evaluation of Ada program performance characteristics, evaluation of logic paths exercised during Ada program execution, analysis of the statistical behavior( maximum, minimum, and average values ) of specified variables, and symbolic debugging of Ada programs. This document describes each of these tools through a presentation of their capabilities, their inputs, and their outputs. This presentation is structured in the following way. A general overview of the entire suite of ATEST tools is first presented in Section 2.0 . The system overview is followed in Section 3.0 by a discussion of the ATEST tools user interface. This user interface controls access to the ATEST tools and provides assistance for the use of the Source Instrumenter and the remaining tools. In Section 4.0, each of the ATEST tools is described. Information is provided on both the operational sequence of steps required to effectively use the tools and the information produced by the ATEST tools. Section 5.0 provides general information on test tool execution. The restrictions and limitations of the ATEST tools are provided in Section 6.0. ATEST User's Guide Page: 2 2.0 ATEST SYSTEM OVERVIEW The Ada Test and Evaluation Tools consist of the following tools: - Source Instrumenter - Path Analyzer - Automatic Path Analyzer - Performance Analyzer - Self-Metric Instrumentation and Analysis - Symbolic Debugger The first release of the Ada Test Tools will be composed of the Source Instrumenter, the Path Analyzers, the Performance Analyzer and the Self-Metric Instrumentation and Analysis Tool. (The Symbolic Debugger is scheduled to be released at a later date). The purpose of these tools is to test and evaluate programs written in the Ada language. They collect statistics about a program's run time capabilities and report the results in a form usable for making decisions about a program's execution paths and efficiency. Such information can prove to be invaluable to programmers who are debugging their software. Each of the Analysis and Report Generators consists of three elements: 1. The Source Instrumenter parses the target Ada program and inserts "hooks" that transfer control to the Run Time Monitor. These hooks are inserted at entry to and exit from each instrumented Ada program unit. The user may also choose to insert these "hooks" at each statement in the target program. 2. The Run Time Monitor gains control of the execution at each breakpoint in each instrumented program unit and records execution information about the program in a Logfile which the user specifies. 3. The Report Generator analyzes the data in the logfile and produces a report which can be used by the programmer to assess the efficiency of the instrumented program. ATEST User's Guide Page: 3 3.0 USER INTERFACE INFORMATION The Tool Set components are integrated into a user interface shell which is controlled by a "shell" program, Test_Tools. Test_Tools provides information to the Source Instrumenter and the Report Generators. It allows the user to enter a Tool Session during which tool selection and tool parameter specification occur. The mechanism of the underlying system is transparent to the user since all user interaction is through the Test_Tools interface. Test_Tools will accept and parse a valid Ada command line. For example: PATH (Log_File_Name => Test.Log, Report_File_Name => Test.Rpt) The user may use either positional or named association. However, once named association is used, all following parameters must be entered using named association. The user will be prompted for any required parameters that were not included on the command line. Upon completion of any single tool, the user will be asked to select another tool. This will continue until the user enters 'quit', which will end the complete tool session. The Test_Tools shell is executed on a system in the same manner as any other Ada program. On the VAX under DEC Ada and the VMS operating system the user enters: RUN TEST_TOOLS <cr> A menu which lists the possible choices will be displayed for the user. The following options are available: PATH - produce a path analysis report from a log file. PROFILE - produce a timing report from a log file. SMART - produce a self metric report from a log file. AUTOPATH - produce a path analysis report from multiple log files. SOURCE_INSTRUMENT - insert breakpoints in an Ada program to gather information about the execution. other possible commands are: QUIT - exit to the system. ? - a list of the tools. ATEST User's Guide Page: 4 4.0 ATEST TOOL OVERVIEWS In this section, each of the ATEST tools is described and its operation is outlined. This section is not intended as an exhaustive enumeration of each tool's options and capabilities. Further information on all of these tools can be found in the appropriate user's guide. 4.1 Source_Instrumenter (SI) Source_Instrumenter parses a target Ada program and inserts "hooks" in the code to allow transfer of control to the Run Time Monitor. When the program is executed, dynamic run time statistics are gathered and placed in a logfile which can later be analyzed by the individual report generators. These "hooks" will normally be placed at every entry and exit of a program unit as well as at every decision point. Optionally, they may be placed at each Ada statement. All programs units that need to be tested with the Test and Analysis Tool set must be instrumented prior to compilation, linkage and execution. It is suggested that the user choose to instrument at the statement level for the Path Analyzer and the Automatic Path Analyzer only. The default is to instrument at subprogram entry, exit and decision points. There are three parameters used by Source Instrumenter: 1) Source_File - The input Ada program. 2) Source_Listing - The output listing of the Ada source code. 3) Instrumented_Source - The output instrumented source file. If the user doesn't specify the Source_Listing or the Instrumented_Source, they will be derived from the Source_File. Example: Source_File = SORT.ADA --> Source_Listing = SORT.LST Instrumented_Source = SORT.INS Source_File = SORT --> Source_Listing = SORT.LST Instrumented_Source = SORT.INS For further information about the Source Instrumenter, refer to the Source Instrumenter User's Guide. ATEST User's Guide Page: 5 4.2 Path Analyzer (Path) Path will produce a path analysis report which lists the history of execution and an execution summary, from the log file supplied. This information can be used by the programmer during testing to determine whether the program is following the expected path(s). These statistics will make the programmer aware of sections of code that have high execution frequencies as well as those that are not being executed. The user can then optimize his code in the appropriate places. There are two parameters required by Path: 1) Log_File_Name - The input log file. 2) Report_File_Name - The output report file. If the user doesn't specify the Report_File_Name it will be derived from the Log_File_Name, with '.RPT' appended. Example: Log_File_Name = SORT.LOG --> Report_File_Name = SORT.RPT Log_File_Name = SORT --> Report_File_Name = SORT.RPT For further information about the Path Analyzer, refer to the Path Analyzer User's Guide. ATEST User's Guide Page: 6 4.3 Performance Analyzer (Profile) Profile will produce a timing report which lists the amount of time spent in each program unit from the log file supplied. Net and cumulative times are reported for each program unit. This information enables the user to identify program units that require a high percentage of total execution time. There are two parameters required by Profile: 1) Log_File_Name - The input log file. 2) Report_File_Name - The output report file. If the user doesn't specify the Report_File_Name it will be derived from the Log_File_Name, with '.RPT' appended. Example: Log_File_Name = SORT.LOG --> Report_File_Name = SORT.RPT Log_File_Name = SORT --> Report_File_Name = SORT.RPT For further information about the Performance Analyzer, refer to the Performance Analyzer User's Guide. ATEST User's Guide Page: 7 4.4 Self-Metric Analysis and Report Tool (Smart) Smart will produce a report which provides the following dynamic run time statistics about the target program: 1. Change of value trace of the user-specified program variables. 2. Program loop execution frequencies. 3. The maximum, minimum and average values of user specified variables. This information can then be used by the programmer to evaluate the usage of the program variables and possibly optimize the variable usage. There are two parameters required by Smart: 1) Log_File_Name - The input log file. 2) Report_File_Name - The output report file. If the user doesn't specify the Report_File_Name it will be derived from the Log_File_Name, with '.RPT' appended. Example: Log_File_Name = SORT.LOG --> Report_File_Name = SORT.RPT Log_File_Name = SORT --> Report_File_Name = SORT.RPT For further information about the Self-Metric Analysis and Report Tool, refer to the Smart User's Guide. ATEST User's Guide Page: 8 4.5 Automatic Path Analyzer (Autopath) Autopath will produce a path analysis report which lists a history of execution and execution summary, from the log files supplied. The user will be prompted for the log files. At least one log file must be entered. The user will continue to be prompted for log file names until a carriage return is entered with no other input. The user will then be prompted for the report file name. If the user doesn't specify the Report_File_Name it will default to "AUTOPATH.RPT". For further information about the Automatic Path Analyzer, refer to the Autopath User's Guide. ATEST User's Guide Page: 9 5.0 TEST TOOL EXECUTION To execute a Test Tool the following tasks must be performed: 1. The source code for the Ada program targeted for the test must be instrumented. 2. The instrumented source must be compiled, linked and executed to produce an Execution Logfile. 3. The desired report generator must be executed to analyze the contents of the logfile and produce a report which can be inspected by the user. The Source Instrumenter can be executed from within the Test_Tools "shell". This will produce a new source file. The user must leave Test_Tools to compile the new source and link the target program with the Run Time Monitor. The Source Instru- menter adds "with" and "use" clauses to the source file to include the Run Time Monitor and other library units necessary to collect program execution information. The Ada Environment will automatically link in the required library units to form the executable program. The user then executes his program normally. During the execution, the Run Time Monitor will gain control of the program and prompt the user for the following information: 1. Tool_Name : The user must enter a valid tool name 2. Logfile_Name : The user should enter a legal filename for for the computer he is using. 3. Test_Identification: The user is asked to enter a unique test identification which can be any valid Ada string. After this input, the target program resumes its normal execution and produces an execution logfile. For more detailed information about the Run Time Monitor, refer to the user's guides for the report generators. Next, the user must re-enter the Test_Tools shell to invoke the Analysis and Report tool which corresponds to the Tool_Name that was specified to the Run Time Monitor. The tool will ana- lyze the logfile data and produce a report. This report is in a readable format so the user may inspect it and use the information to evaluate the performance of his program. ATEST User's Guide Page: 10 6.0 RESTRICTIONS AND LIMITATIONS 1) No specific limits are imposed either by the Runtime Monitor or the ATEST report generators on either the size of the target Ada program or the number of program units that can be instrumented. All arrays and stacks maintained by ATEST that are a function of the total number of program units or the number of program units active at a given time are allocated dynamically and deallocated when they are no longer needed. 2) Dynamic execution data may only be reported for program units that have been instrumented by the source instrumenter. 3) Regardless of whether or not the Ada compiler system being used for program development provides an interface to programming languages other than Ada, only Ada program units may be instrumented. 4) The runtime execution monitor imposes a certain amount of overhead on the execution time of each instrumented Ada program unit. This overhead will vary from system to system. Therfore, an instrumented program will execute slower than the same program uninstrumented. In a tasking environment if the execution of some program units is time dependent then the additional overhead imposed by the run-time execution monitor may alter the normal flow of control of the program. 5) When statement level breakpoints are specified, all variables being traced that are visible within the scope of the executing breakpoint are recorded in the execution log file. Thus, depending on the number of variables traced, the total number of breakpoints executed, and the logic of the instrumented program, the execution log file may become extremely large. ATEST User's Guide Page: A - 1 APPENDIX A GLOSSARY ATEST User's Guide Page: A - 2 Execution Log File A sequential file created by the Run Time Monitor during execution of an Ada program that has been instrumented by the Source Instrumenter. The terms "Execution Log File" and "Log File" are interchangeable. Breakpoint A breakpoint is a transfer of control to the Run Time Monitor. Each breakpoint within a compilation unit is assigned a unique breakpoint number by the Source Instrumenter. In general, breakpoints correspond to executable statements. However, not all executable statements are assigned breakpoints. Log File A sequential file created by the Run Time Monitor during execution of an Ada program that has been instrumented by the Source Instrumenter. The terms "Execution Log File" and "Log File" are interchangeable. Overhead The term "overhead" refers to the additional execution time imposed on the program under test by the Run Time Monitor. Only a portion of this overhead can be measured with any degree of accuracy. If the program under test terminates normally the Profile Report Generator will, if requested by the user, adjust execution times for that portion of the overhead that was measured by the Run Time Monitor. Program Unit Ada programs may contain four different types of program units: procedures, functions, tasks and generics. However, the Ada language also provides the ability to include executeable code within a package body that is not included within any of the program units contained in that package. This code is executed during package initialization when the package is elaborated. Profile reports timing information for this "initialization" code the same as for Ada program units. Therefore, for reporting purposes, initialization code within a package body is also treated as a program unit. ATEST User's Guide Page: A - 3 Report Time The time that the ATEST Report Generator was executed. Run Time Monitor A runtime kernel that gains control upon entry to and exit from each instrumented Ada program unit and logs timing information about the program's execution. The runtime execution monitor must be linked with the instrumented Ada program units. Source Instrumenter An Ada program that parses the target Ada program and inserts "hooks" that transfer control to the Run Time Monitor upon entry to and exit from each instrumented Ada program unit. Each Ada program unit for which timing information is to be recorded must be instrumented by the Source Instrumenter prior to compilation. Test ID A unique test identification specified by the user when the program under test is executed. The Test ID is stored in the log file by the Run Time Monitor and included in the Test Configuration Report by the Profile report generator. Test Time The date and time that the program under test began execution. --:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: --autopath.ug --:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: Ada* Automatic Path Analyzer User's Guide by Julie Trost * Ada is a trademark of the U.S. Department of Defense (AJPO) Ada Automatic Path Analyzer User's Guide CONTENTS 1.0 DESCRIPTION ....................................... 1 2.0 COMMAND FORMAT .................................... 2 2.1 Source Program Instrumentation .................. 2 2.2 Compilation and Linking ......................... 2 2.3 Target Program Execution ........................ 3 2.4 Report Generation ............................... 5 3.0 OUTPUT FORMAT ..................................... 7 3.1 Test Configuration Report ....................... 7 3.2 History of Execution Report ..................... 8 3.3 Execution Summary Report ........................ 8 4.0 RESTRICTIONS AND LIMITATIONS ...................... 9 APPENDIX A - SAMPLE REPORT FILE ........................ A-1 APPENDIX B - GLOSSARY .................................. B-1 Automatic Path Analyzer User's Guide Page: 1 1.0 DESCRIPTION The Ada Automatic Path Analyzer, Autopath, is one of an integrated set of five tools that provide testing and analysis aids for the development of computer programs implemented in the Ada programming language. Specifically, Autopath provides execution information on an Ada program to enable the analyst to identify program units that are never executed. Autopath consists of three elements: 1) A source instrumenter that parses the target Ada program and inserts "hooks" that transfer control to a runtime kernal at each statement in each instrumented Ada program unit. Each Ada program unit for which execution information is to be recorded must be instrumented prior to compilation. 2) A runtime execution monitor, or kernel, that gains control at each statement in each instrumented Ada program unit and logs information about the program's execution. The runtime execution monitor must be linked with the instrumented Ada program units. 3) A report generator that analyzes the recorded execution information and produces a comprehensive report in an easily understandable format. Reports generated provide detailed quantitative information about the history of execution and a summary of execution for each Ada program unit. Autopath is a software development tool. Its purpose is to aid the programmer or analyst in identifying statements or blocks of code that are not executed or are executed a large number of times. Autopath can aid the programmer or analyst: O during program development in identifying those parts of the program that are not being used. O in analyzing existing programs with performance problems to identify inefficiencies or areas that may benefit from optimization and result in significantly improved overall program performance. O identify untested areas of code. Automatic Path Analyzer User's Guide Page: 2 2.0 COMMAND FORMAT Operation of the Ada Automatic Path Analyzer requires four steps: 1) source program instrumentation 2) compilation and linking 3) target program execution 4) report generation Each of these steps is described in detail in the following paragraphs. 2.1 Source Program Instrumentation Each Ada compilation unit containing program units for which execution trace data is to be recorded must be separately instrumented by the source instrumenter. The Automatic Path Analyzer will provide the most accurate execution information on programs and subprograms that have been instrumented at each statement. However, it may also be used to provide execution analysis of programs and subprograms that have been instrumented at decision points or at entry/exit points for use with the Performance Analyzer, Path Analyzer, or Self Metric Instrumentation and Analysis Tool. Refer to the Source Instrumenter User's Guide for detailed information on using the Source Instrumenter. 2.2 Compilation and Linking When the instrumentation process is completed the instrumented target Ada program must be compiled and linked to prepare it for execution. Each instrumented Ada compilation unit must be separately compiled. When the compilation process is completed the target program must be linked with the Ada Automatic Path Analyzer Run Time Monitor. The user should follow the normal procedures for compiling and linking with the Ada compiler system being used for program development. Automatic Path Analyzer User's Guide Page: 3 2.3 Target Program Execution When linking is completed, the target Ada program may be executed. The user interface and method of executing the target program is not altered by the instrumentation process. Therefore, the user should follow the normal procedures for executing the target program on the host computer system being used for program development. When the first unit of the target program is entered, the Run Time Monitor will prompt the user for the following information: 1. Tool_Name : The user should enter AUTOPATH_TOOL (upper or lower case is acceptable). If the user should enter an invalid tool name, he will be repeatedly prompted until a correct tool name is entered or the program is aborted. 2. Logfile_Name : The user should enter a legal filename for the computer he is using. The default filename may be chosen by entering a carriage return. The default Logfile_Name is AUTOPATH.LOG. If the File already exists, the user is asked if he would like to Enter a new filename, Overwrite the existing file, or Append to the existing file. 3. Test_Identification: The user is asked to enter a unique test identification which can be any valid Ada string. The default Test ID may be selected by entering a carriage return. The default Test ID for Autopath is "Automatic Path Analysis Report". The Test ID is recorded in the log file generated by the Run Time Monitor and printed on the Test Configuration Report generated by the Autopath Report Generator. A sample session with the Automatic Path Analyzer is presented on the following page. Automatic Path Analyzer User's Guide Page: 4 EXAMPLE SESSION Following is a sample session with the Run Time Monitor. (User responses are shown in Bold Face): Enter Tool Type : PATH_TOOL, AUTOPATH_TOOL, SMART_TOOL, or PROFILE_TOOL -----> AUTOPATH_TOOL <cr>* Enter Logfile Name, Null for Default -----> Testa.log <cr> If the Logfile already exists, the following message and prompt are printed: File already exists!!! Do you wish to: E = Enter a New Filename O = Overwrite existing file A = Append to the existing file Enter Option ===> O <cr> The Run Time Monitor will repeatedly prompt the user until a valid option is entered. Enter a unique Test Identification, Null for default. Log File Test Case # 5 <cr> When the Run Time Monitor input is complete, the target program resumes its normal execution. * <cr> represents a carriage return. Automatic Path Analyzer User's Guide Page: 5 2.4 Report Generation During execution, the instrumented Ada program will create a log file containing program execution trace data. The Ada Automatic Path Analyzer Report Generator will analyze the data from the log files supplied and present it in an easily understood format. The user may include as many log files as he desires. The Autopath report generator is one of four report generators included in the Ada Test and Analysis System Report Generator. The report generator is executed as follows: (User input and responses shown in are Bold Face) RUN TEST_TOOLS <cr> The user will be prompted to enter the name of the reporting tool he wishes to run. To execute Autopath, the user should respond with the tool name, Autopath. EXAMPLE Welcome to the Test Tools please enter desired tool enter ? for a list of tools available AUTOPATH <cr> If an incorrect tool name is entered the user will be prompted to reenter the tool name. When a valid tool name has been entered the user will be prompted to enter the name of the log file to be processed and the name of the report file to be generated. EXAMPLE Enter Log_File_Name => TASKLOG.LOG <cr> Enter Report_File_Name => TASKLOG.RPT <cr> If the user doesn't enter a Report_File_Name, it will default to AUTOPATH.RPT. Autopath will display the test configuration data that was recorded in the first log file at the time the program under test was executed. The user is then prompted as to whether or not he wishes to continue. After examining the test configuration data, the user can discontinue processing by entering "N". If the correct log Automatic Path Analyzer User's Guide Page: 6 file has been selected, the user should enter "Y" to continue processing. EXAMPLE Program Under Test: Testlog Test Date: 06/23/85 Test Time: 20:24:59 Test ID: Log File Test Case #5 Do you wish to continue (Y/N) ? Y <cr> If the user enters "N", processing will be discontinued. EXAMPLE SESSION Following is a sample session with the Report Generator (User input and responses shown in are Bold Face): $ RUN TEST_TOOLS <cr> Welcome to the Test Tools please enter desired tool enter ? for a list of tools available ? <cr> The following tools may be executed: path -- the report generator for the path analysis tool profile -- the report generator for the timing tool smart -- the report generator for the self metric tool autopath -- the report generator for the automatic path tool source_instrument -- instrument an Ada source file please enter desired tool Autopath <cr> Enter Log_File_Name => TASKA.LOG <cr> Enter Log_File_Name => TASKB.LOG <cr> Enter Log_File_Name => TASKC.LOG <cr> Enter Log_File_Name => <cr> Enter Report_File_Name => <cr> report_file_name = AUTOPATH.RPT Program Under Test: Testlog Test Date: 06/23/85 Test Time: 20:24:59 Test ID: Log File Test Case #5 Do you wish to continue (Y/N) ? Y <cr> please enter desired tool quit <cr> $ Automatic Path Analyzer User's Guide Page: 7 3.0 OUTPUT FORMAT The report file generated by Autopath contains three reports: a Test Configuration Report, a History Of Execution Report and an Execution Summary Report. Each report provides the user with different information about the program being tested and is generated in 80 column format to enable "browsing" at a terminal when hardcopy is either unavailable or not desired. A sample report file is included in Appendix A. Prior to using the Autopath report generator it is important for the user to understand the information that is being presented and how to interpret it. Each of the execution reports is described below. 3.1 Test Configuration Report The Test Configuration Report includes information necessary to correlate the report with a specific test or execution of the target Ada program. It includes the following information: 1) Program Under Test: This is the name of the main program unit of the program being analyzed. 2) Test Date: The calendar date that the program under test was executed and the execution log file was created. The test date is presented in the form MM/DD/YY. 3) Test Day: The day of the week, Sunday through Saturday, that the program under test was executed and the execution log file was created. 4) Test Time: The time of day that the program under test was executed and the execution log file was created. The test time is presented in the form HH:MM:SS. 5) Log File: The name of the log file that was used in the preparation of the execution timing reports. 6) Test ID: The test ID specified by the user at the time the program under test was executed. Automatic Path Analyzer User's Guide Page: 8 3.2 History Of Execution Report This report lists the order in which the statements executed. A 'Begin' indicates the program unit was entered at the first statement. A 'Resume' indicates the program unit was entered at a statement other than the first statement. An 'End' indicates the last statement of the program unit was executed. Consecutive statements are grouped together in square brackets '[]'. A backslash '/' indicates a backward jump in the code (a decrease in statement numbers). 3.3 Execution Summary Report This report lists the number of times each statement was executed. Statements are listed under the compilation unit name. Consecutive statements that were executed the same number of times are grouped together in square brackets '[]'. Automatic Path Analyzer User's Guide Page: 9 4.0 RESTRICTIONS AND LIMITATIONS 1) No specific limits are imposed either by the Runtime Monitor or the Profile report generator on either the size of the target Ada program or the number of program units that can be instrumented. All arrays and stacks maintained by Path that are a function of the total number of program units or the number of program units active at a given time are allocated dynamically and deallocated when they are no longer needed. 2) Execution data may only be reported for program units that have been instrumented by the source instrumenter. 3) Regardless of whether or not the Ada compiler system being used for program development provides an interface to programming languages other than Ada, only Ada program units may be instrumented. 4) Autopath will not limit the number of log files to be analyzed, however a new log file is created by con- catenating the desired log files. This new file may be quite large, and disk space may be a problem. 5) The numbers recorded in the output file represent the breakpoint number, not the statement number. If a program unit is instrumented at entry/exit, it will have 2 breakpoints. If a program unit is instrumented at entry/exit/decision points, it will have breakpoints throughout the code but not every statement will be represented. If a program unit is instrumented at each statement, breakpoint numbers will correspond to statement numbers. Automatic Path Analyzer User's Guide Page: A - 1 APPENDIX A SAMPLE REPORT Automatic Path Analyzer User's Guide Page: A - 2 Source Instrumenter Output on 07/29/85 at 09:27:03 1 Source File: test_case1.ada Bkpt Number Source Text ------ ----------- procedure TEST_CASE1 is type STORAGE_ARRAY is array(INTEGER range <>) of INTEGER; STORAGE : STORAGE_ARRAY(1 .. 10) := (15, 12, 14, 10, 2, 5, 1, 4, 11, 12); procedure ONE_PASS(S : in out STORAGE_ARRAY) is TEMP : INTEGER; begin 1 for I in S'FIRST .. S'LAST - 1 loop 2 if S(I) > S(I + 1) then 3 TEMP := S(I); S(I) := S(I + 1); S(I + 1) := TEMP; end if; end loop; 4 end ONE_PASS; begin 5 for J in 1 .. STORAGE'LAST - STORAGE'FIRST - 1 loop 6 ONE_PASS(STORAGE); end loop; 7 end TEST_CASE1; Automatic Path Analyzer User's Guide Page: A - 3 Source Instrumenter Output on 07/29/85 at 09:27:03 1 Source File: test_case1.ada Bkpt Number Source Text ------ ----------- procedure TEST_CASE1 is type STORAGE_ARRAY is array(INTEGER range <>) of INTEGER; STORAGE : STORAGE_ARRAY(1 .. 10) := (12, 2, 5, 8, 2, 1, 14, 10, 3, 12); procedure ONE_PASS(S : in out STORAGE_ARRAY) is TEMP : INTEGER; begin 1 for I in S'FIRST .. S'LAST - 1 loop 2 if S(I) > S(I + 1) then 3 TEMP := S(I); S(I) := S(I + 1); S(I + 1) := TEMP; end if; end loop; 4 end ONE_PASS; begin 5 for J in 1 .. STORAGE'LAST - STORAGE'FIRST - 1 loop 6 ONE_PASS(STORAGE); end loop; 7 end TEST_CASE1; Automatic Path Analyzer User's Guide Page: A - 4 Path Analyzer 01.00 MONDAY 07/29/85 09:36:43 Page: 1 Ada Path Analyzer - Test Configuration Report -------------------------------------------------------------------------------- Program Under Test: TEST_CASE1 Test Date: 07/29/85 Test Day: MONDAY Test Time: 09:30:01 Log File: autopath.log Test ID: Path Analysis Report Automatic Path Analyzer User's Guide Page: A - 5 Path Analyzer 01.00 MONDAY 07/29/85 09:36:43 Page: 2 autopath.log Begin TEST_CASE1 [5-6] Begin TEST_CASE1.ONE_PASS [1-3] \ [2-3] \ [2-3] \ [2-3] \ [2-3] \ [2-3] \ [2-3] \ [2-3] \ [2-4] End TEST_CASE1.ONE_PASS Resume TEST_CASE1 [6] Begin TEST_CASE1.ONE_PASS [1-2] [2-3] \ [2-3] \ [2-3] \ [2-3] \ [2-3] \ [2-3] \ [2-3] \ [2] [4] End TEST_CASE1.ONE_PASS Resume TEST_CASE1 [6] Begin TEST_CASE1.ONE_PASS [1-3] \ [2-3] \ [2-3] \ [2-3] \ [2-3] \ [2-3] \ [2] [2] [2] [4] End TEST_CASE1.ONE_PASS Resume TEST_CASE1 [6] Begin TEST_CASE1.ONE_PASS [1-3] \ [2-3] \ [2-3] \ [2-3] \ [2] [2] [2] [2] [2] [4] End TEST_CASE1.ONE_PASS Resume TEST_CASE1 [6] Begin TEST_CASE1.ONE_PASS [1-2] [2-3] \ [2-3] \ [2] [2] [2] [2] [2] [2] [4] End TEST_CASE1.ONE_PASS Resume TEST_CASE1 [6] Begin TEST_CASE1.ONE_PASS [1-3] \ [2] [2] [2] [2] [2] [2] [2] [2] [4] End TEST_CASE1.ONE_PASS Resume TEST_CASE1 [6] Begin TEST_CASE1.ONE_PASS [1-2] [2] [2] [2] [2] [2] [2] [2] [2] [4] End TEST_CASE1.ONE_PASS Resume TEST_CASE1 [6] Begin TEST_CASE1.ONE_PASS [1-2] [2] [2] [2] [2] [2] [2] [2] [2] [4] End TEST_CASE1.ONE_PASS Resume TEST_CASE1 [7] End TEST_CASE1 Begin TEST_CASE1 [5-6] Begin TEST_CASE1.ONE_PASS [1-3] \ [2-3] \ [2-3] \ [2-3] \ [2-3] \ [2] [2-3] \ [2-3] \ [2-4] Automatic Path Analyzer User's Guide Page: A - 6 Path Analyzer 01.00 MONDAY 07/29/85 09:36:43 Page: 3 autopath.log End TEST_CASE1.ONE_PASS Resume TEST_CASE1 [6] Begin TEST_CASE1.ONE_PASS [1-2] [2] [2-3] \ [2-3] \ [2] [2-3] \ [2-3] \ [2] [2] [4] End TEST_CASE1.ONE_PASS Resume TEST_CASE1 [6] Begin TEST_CASE1.ONE_PASS [1-2] [2-3] \ [2-3] \ [2] [2] [2-3] \ [2] [2] [2] [4] End TEST_CASE1.ONE_PASS Resume TEST_CASE1 [6] Begin TEST_CASE1.ONE_PASS [1-2] [2-3] \ [2] [2] [2-3] \ [2] [2] [2] [2] [4] End TEST_CASE1.ONE_PASS Resume TEST_CASE1 [6] Begin TEST_CASE1.ONE_PASS [1-3] \ [2] [2] [2-3] \ [2] [2] [2] [2] [2] [4] End TEST_CASE1.ONE_PASS Resume TEST_CASE1 [6] Begin TEST_CASE1.ONE_PASS [1-2] [2] [2] [2] [2] [2] [2] [2] [2] [4] End TEST_CASE1.ONE_PASS Resume TEST_CASE1 [6] Begin TEST_CASE1.ONE_PASS [1-2] [2] [2] [2] [2] [2] [2] [2] [2] [4] End TEST_CASE1.ONE_PASS Resume TEST_CASE1 [6] Begin TEST_CASE1.ONE_PASS [1-2] [2] [2] [2] [2] [2] [2] [2] [2] [4] End TEST_CASE1.ONE_PASS Resume TEST_CASE1 [7] End TEST_CASE1 Automatic Path Analyzer User's Guide Page: A - 7 Path Analyzer 01.00 MONDAY 07/29/85 09:36:43 Page: 4 autopath.log Statement Execution Count --------- --------------- TEST_CASE1 [1] 16 [2] 144 [3] 48 [4] 16 [5] 2 [6] 16 [7] 2 Automatic Path Analyzer User's Guide Page: B - 1 GLOSSARY Execution Log File A sequential file created by the Run Time Monitor during execution of an Ada program that has been instrumented by the Source Instrumenter. The terms "Execution Log File" and "Log File" are interchangeable. Log File A sequential file created by the Run Time Monitor during execution of an Ada program that has been instrumented by the Source Instrumenter. The terms "Execution Log File" and "Log File" are interchangeable. Program Unit Ada programs may contain four different types of program units: procedures, functions, tasks and generics. However, the Ada language also provides the ability to include executeable code within a package body that is not included within any of the program units contained in that package. This code is executed during package initialization when the package is elaborated. Profile reports timing information for this "initialization" code the same as for Ada program units. Therefore, for reporting purposes, initialization code within a package body is also treated as a program unit. Report Time The time that the Profile Report Generator was executed. Run Time Monitor A runtime kernel that gains control upon entry to and exit from each instrumented Ada program unit and logs timing information about the program's execution. The runtime execution monitor must be linked with the instrumented Ada program units. Source Instrumenter An Ada program that parses the target Ada program and inserts "hooks" that transfer control to the Run Time Monitor upon entry to and exit from each instrumented Ada program unit. Each Ada program unit for which timing information is to be recorded must be instrumented by the Source Instrumenter prior to compilation. Automatic Path Analyzer User's Guide Page: B - 2 Test ID A unique test identification specified by the user when the program under test is executed. The Test ID is stored in the log file by the Run Time Monitor and included in the Test Configuration Report by the Profile report generator. Test Time The date and time that the program under test began execution. --:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: --path.ug --:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: Ada* Path Analyzer User's Guide by Julie Trost * Ada is a trademark of the U.S. Department of Defense (AJPO) Ada Path Analyzer User's Guide CONTENTS 1.0 DESCRIPTION ....................................... 1 2.0 COMMAND FORMAT .................................... 2 2.1 Source Program Instrumentation .................. 2 2.2 Compilation and Linking ......................... 2 2.3 Target Program Execution ........................ 3 2.4 Report Generation ............................... 5 3.0 OUTPUT FORMAT ..................................... 7 3.1 Test Configuration Report ....................... 7 3.2 History of Execution Report ..................... 8 3.3 Execution Summary Report ........................ 8 4.0 RESTRICTIONS AND LIMITATIONS ...................... 9 APPENDIX A - SAMPLE REPORT FILE ........................ A-1 APPENDIX B - GLOSSARY .................................. B-1 Path Analyzer User's Guide Page: 1 1.0 DESCRIPTION The Ada Path Analyzer, Path, is one of an integrated set of five tools that provide testing and analysis aids for the development of computer programs implemented in the Ada programming language. Specifically, Path provides execution information on an Ada program to enable the analyst to identify program units that are never executed. Path consists of three elements: 1) A source instrumenter that parses the target Ada program and inserts "hooks" that transfer control to a runtime kernal at each statement in each instrumented Ada program unit. Each Ada program unit for which execution information is to be recorded must be instrumented prior to compilation. 2) A runtime execution monitor, or kernel, that gains control at each statement in each instrumented Ada program unit and logs information about the program's execution. The runtime execution monitor must be linked with the instrumented Ada program units. 3) A report generator that analyzes the recorded execution information and produces a comprehensive report in an easily understandable format. Reports generated provide detailed quantitative information about the history of execution and a summary of execution for each Ada program unit. Path is a software development tool. Its purpose is to aid the programmer or analyst in identifying statements or blocks of code that are not executed or are executed a large number of times. Path can aid the programmer or analyst: O during program development in identifying those parts of the program that are not being used. O in analyzing existing programs with performance problems to identify inefficiencies or areas that may benefit from optimization and result in significantly improved overall program performance. O identify untested areas of code. Path Analyzer User's Guide Page: 2 2.0 COMMAND FORMAT Operation of the Ada Path Analyzer requires four steps: 1) source program instrumentation 2) compilation and linking 3) target program execution 4) report generation Each of these steps is described in detail in the following paragraphs. 2.1 Source Program Instrumentation Each Ada compilation unit containing program units for which execution trace data is to be recorded must be separately instrumented by the source instrumenter. The Path Analyzer will provide the most accurate execution information on programs and subprograms that have been instrumented at each statement. However, it may also be used to provide execution analysis of programs and subprograms that have been instrumented at decision points or at entry/exit points for use with the Performance Analyzer, Automatic Path Analyzer, or Self Metric Instrumentation and Analysis Tool. Refer to the Source Instrumenter User's Guide for detailed information on using the Source Instrumenter. 2.2 Compilation and Linking When the instrumentation process is completed the instrumented target Ada program must be compiled and linked to prepare it for execution. Each instrumented Ada compilation unit must be separately compiled. When the compilation process is completed the target program must be linked with the Ada Path Analyzer Run Time Monitor. The user should follow the normal procedures for compiling and linking with the Ada compiler system being used for program development. Path Analyzer User's Guide Page: 3 2.3 Target Program Execution When linking is completed, the target Ada program may be executed. The user interface and method of executing the target program is not altered by the instrumentation process. Therefore, the user should follow the normal procedures for executing the target program on the host computer system being used for program development. When the first unit of the target program is entered, the Run Time Monitor will prompt the user for the following information: 1. Tool_Name : The user should enter PATH_TOOL (upper or lower case is acceptable). If the user should enter an invalid tool name, he will be repeatedly prompted until a correct tool name is entered or the program is aborted. 2. Logfile_Name : The user should enter a legal filename for the computer he is using. The default filename may be chosen by entering a carriage return. The default Logfile_Name is PATH.LOG. If the File already exists, the user is asked if he would like to Enter a new filename, Overwrite the existing file, or Append to the existing file. 3. Test_Identification: The user is asked to enter a unique test identification which can be any valid Ada string. The default Test ID may be selected by entering a carriage return. The default Test ID for Path is "Path Analysis Report". The Test ID is recorded in the log file generated by the Run Time Monitor and printed on the Test Configuration Report generated by the Path Report Generator. A sample session with the Ada Path Analyzer is presented on the following page. Path Analyzer User's Guide Page: 4 EXAMPLE SESSION Following is a sample session with the Run Time Monitor. (User responses are shown in Bold Face): Enter Tool Type : PATH_TOOL, AUTOPATH_TOOL, SMART_TOOL, or PROFILE_TOOL -----> PATH_TOOL <cr>* Enter Logfile Name, Null for Default -----> Testa.log <cr> If the Logfile already exists, the following message and prompt are printed: File already exists!!! Do you wish to: E = Enter a New Filename O = Overwrite existing file A = Append to the existing file Enter Option ===> O <cr> The Run Time Monitor will repeatedly prompt the user until a valid option is entered. Enter a unique Test Identification, Null for default. Log File Test Case #5 <cr> When the Run Time Monitor input is complete, the target program resumes its normal execution. * <cr> represents a carriage return. Path Analyzer User's Guide Page: 5 2.4 Report Generation During execution, the instrumented Ada program will create a log file containing program execution trace data. The Ada Path Analyzer Report Generator, analyzes the data and presents it in an easily understood format. The Path report generator is one of four report generators included in the Ada Test and Analysis System Report Generator. The report generator is executed as follows: (User input and responses shown in are Bold Face) RUN TEST_TOOLS <cr> The user will be prompted to enter the name of the reporting tool he wishes to run. To execute Path, the user should respond with the tool name, Path. EXAMPLE Welcome to the Test Tools please enter desired tool enter ? for a list of tools available PATH <cr> If an incorrect tool name is entered the user will be prompted to reenter the tool name. When a valid tool name has been entered the user will be prompted to enter the name of the log file to be processed and the name of the report file to be generated. EXAMPLE Enter Log_File_Name => TASKLOG.LOG <cr> Enter Report_File_Name => TASKLOG.RPT <cr> If the user doesn't enter a Report_File_Name, it will be derived from the Log_File_Name, with '.RPT' appended. Path will display the test configuration data that was recorded in the log file at the time the program under test was executed. The user is then prompted as to whether or not he wishes to continue. After examining the test configuration data, the user can discontinue processing by entering "N". If the correct log file has been selected, the user should enter "Y" to continue processing. Path Analyzer User's Guide Page: 6 EXAMPLE Program Under Test: Testlog Test Date: 06/23/85 Test Time: 20:24:59 Test ID: Log File Test Case #5 Do you wish to continue (Y/N) ? Y <cr> If the user enters "N", processing will be discontinued. EXAMPLE SESSION Following is a sample session with the Report Generator (User input and responses shown in are Bold Face): $ RUN TEST_TOOLS <cr> Welcome to the Test Tools please enter desired tool enter ? for a list of tools available ? <cr> The following tools may be executed: path -- the report generator for the path analysis tool profile -- the report generator for the timing tool smart -- the report generator for the self metric tool autopath -- the report generator for the automatic path tool source_instrument -- instrument an Ada source file please enter desired tool Path <cr> Enter Log_File_Name => TASKLOG.LOG <cr> Enter Report_File_Name => <cr> log_file_name = TASKLOG.LOG report_file_name = TASKLOG.RPT Program Under Test: Testlog Test Date: 06/23/85 Test Time: 20:24:59 Test ID: Log File Test Case #5 Do you wish to continue (Y/N) ? Y <cr> please enter desired tool quit <cr> $ Path Analyzer User's Guide Page: 7 3.0 OUTPUT FORMAT The report file generated by PATH contains three reports: a Test Configuration Report, a History Of Execution Report and an Execution Summary Report. Each report provides the user with different information about the program being tested and is generated in 80 column format to enable "browsing" at a terminal when hardcopy is either unavailable or not desired. A sample report file is included in Appendix A. Prior to using the Path report generator it is important for the user to understand the information that is being presented and how to interpret it. Each of the execution reports is described below. 3.1 Test Configuration Report The Test Configuration Report includes information necessary to correlate the report with a specific test or execution of the target Ada program. It includes the following information: 1) Program Under Test: This is the name of the main program unit of the program being analyzed. 2) Test Date: The calendar date that the program under test was executed and the execution log file was created. The test date is presented in the form MM/DD/YY. 3) Test Day: The day of the week, Sunday through Saturday, that the program under test was executed and the execution log file was created. 4) Test Time: The time of day that the program under test was executed and the execution log file was created. The test time is presented in the form HH:MM:SS. 5) Log File: The name of the log file that was used in the preparation of the execution timing reports. 6) Test ID: The test ID specified by the user at the time the program under test was executed. Path Analyzer User's Guide Page: 8 3.2 History Of Execution Report This report lists the order in which the statements executed. A 'Begin' indicates the program unit was entered at the first statement. A 'Resume' indicates the program unit was entered at a statement other than the first statement. An 'End' indicates the last statement of the program unit was executed. Consecutive statements are grouped together in square brackets '[]'. A backslash '/' indicates a backward jump in the code (a decrease in statement numbers). 3.3 Execution Summary Report This report lists the number of times each statement was executed. Statements are listed under the compilation unit name. Consecutive statements that were executed the same number of times are grouped together in square brackets '[]'. Path Analyzer User's Guide Page: 9 4.0 RESTRICTIONS AND LIMITATIONS 1) No specific limits are imposed either by the Runtime Monitor or the Profile report generator on either the size of the target Ada program or the number of program units that can be instrumented. All arrays and stacks maintained by Path that are a function of the total number of program units or the number of program units active at a given time are allocated dynamically and deallocated when they are no longer needed. 2) Execution data may only be reported for program units that have been instrumented by the source instrumenter. 3) Regardless of whether or not the Ada compiler system being used for program development provides an interface to programming languages other than Ada, only Ada program units may be instrumented. 4) The numbers recorded in the output file represent the breakpoint number not the statement number. If a program unit is instrumented at entry/exit, it will have 2 breakpoints. If a program unit is instrumented at entry/exit/decision points, it will have breakpoints throughout the code but not every statement will be represented. If a program unit is instrumented at each statement, breakpoint numbers will correspond to statement numbers. Path Analyzer User's Guide Page: A - 1 APPENDIX A SAMPLE REPORT Path Analyzer User's Guide Page: A - 2 Source Instrumenter Output on 07/29/85 at 09:27:03 1 Source File: test_case1.ada Bkpt Number Source Text ------ ----------- procedure TEST_CASE1 is type STORAGE_ARRAY is array(INTEGER range <>) of INTEGER; STORAGE : STORAGE_ARRAY(1 .. 10) := (15, 12, 14, 10, 2, 5, 1, 4, 11, 12); procedure ONE_PASS(S : in out STORAGE_ARRAY) is TEMP : INTEGER; begin 1 for I in S'FIRST .. S'LAST - 1 loop 2 if S(I) > S(I + 1) then 3 TEMP := S(I); S(I) := S(I + 1); S(I + 1) := TEMP; end if; end loop; 4 end ONE_PASS; begin 5 for J in 1 .. STORAGE'LAST - STORAGE'FIRST - 1 loop 6 ONE_PASS(STORAGE); end loop; 7 end TEST_CASE1; Path Analyzer User's Guide Page: A - 3 Path Analyzer 01.00 MONDAY 07/29/85 09:36:10 Page: 1 Ada Path Analyzer - Test Configuration Report -------------------------------------------------------------------------------- Program Under Test: TEST_CASE1 Test Date: 07/29/85 Test Day: MONDAY Test Time: 09:30:01 Log File: path1.log Test ID: Path Analysis Report Path Analyzer User's Guide Page: A - 4 Path Analyzer 01.00 MONDAY 07/29/85 09:36:10 Page: 2 path1.log Begin TEST_CASE1 [5-6] Begin TEST_CASE1.ONE_PASS [1-3] \ [2-3] \ [2-3] \ [2-3] \ [2-3] \ [2-3] \ [2-3] \ [2-3] \ [2-4] End TEST_CASE1.ONE_PASS Resume TEST_CASE1 [6] Begin TEST_CASE1.ONE_PASS [1-2] [2-3] \ [2-3] \ [2-3] \ [2-3] \ [2-3] \ [2-3] \ [2-3] \ [2] [4] End TEST_CASE1.ONE_PASS Resume TEST_CASE1 [6] Begin TEST_CASE1.ONE_PASS [1-3] \ [2-3] \ [2-3] \ [2-3] \ [2-3] \ [2-3] \ [2] [2] [2] [4] End TEST_CASE1.ONE_PASS Resume TEST_CASE1 [6] Begin TEST_CASE1.ONE_PASS [1-3] \ [2-3] \ [2-3] \ [2-3] \ [2] [2] [2] [2] [2] [4] End TEST_CASE1.ONE_PASS Resume TEST_CASE1 [6] Begin TEST_CASE1.ONE_PASS [1-2] [2-3] \ [2-3] \ [2] [2] [2] [2] [2] [2] [4] End TEST_CASE1.ONE_PASS Resume TEST_CASE1 [6] Begin TEST_CASE1.ONE_PASS [1-3] \ [2] [2] [2] [2] [2] [2] [2] [2] [4] End TEST_CASE1.ONE_PASS Resume TEST_CASE1 [6] Begin TEST_CASE1.ONE_PASS [1-2] [2] [2] [2] [2] [2] [2] [2] [2] [4] End TEST_CASE1.ONE_PASS Resume TEST_CASE1 [6] Begin TEST_CASE1.ONE_PASS [1-2] [2] [2] [2] [2] [2] [2] [2] [2] [4] End TEST_CASE1.ONE_PASS Resume TEST_CASE1 [7] End TEST_CASE1 Path Analyzer User's Guide Page: A - 5 Path Analyzer 01.00 MONDAY 07/29/85 09:36:10 Page: 3 path1.log Statement Execution Count --------- --------------- TEST_CASE1 [1] 1 [2] 72 [3] 29 [4] 8 [5] 1 [6] 8 [7] 1 Path Analyzer User's Guide Page: B - 1 GLOSSARY Execution Log File A sequential file created by the Run Time Monitor during execution of an Ada program that has been instrumented by the Source Instrumenter. The terms "Execution Log File" and "Log File" are interchangeable. Log File A sequential file created by the Run Time Monitor during execution of an Ada program that has been instrumented by the Source Instrumenter. The terms "Execution Log File" and "Log File" are interchangeable. Program Unit Ada programs may contain four different types of program units: procedures, functions, tasks and generics. However, the Ada language also provides the ability to include executeable code within a package body that is not included within any of the program units contained in that package. This code is executed during package initialization when the package is elaborated. Profile reports timing information for this "initialization" code the same as for Ada program units. Therefore, for reporting purposes, initialization code within a package body is also treated as a program unit. Report Time The time that the Profile Report Generator was executed. Run Time Monitor A runtime kernel that gains control upon entry to and exit from each instrumented Ada program unit and logs timing information about the program's execution. The runtime execution monitor must be linked with the instrumented Ada program units. Source Instrumenter An Ada program that parses the target Ada program and inserts "hooks" that transfer control to the Run Time Monitor upon entry to and exit from each instrumented Ada program unit. Each Ada program unit for which timing information is to be recorded must be instrumented by the Source Instrumenter prior to compilation. Path Analyzer User's Guide Page: B - 2 Test ID A unique test identification specified by the user when the program under test is executed. The Test ID is stored in the log file by the Run Time Monitor and included in the Test Configuration Report by the Profile report generator. Test Time The date and time that the program under test began execution. --:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: --profile.ug --:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: Ada* Performance Analyzer ( PROFILE ) Version 1.0 User's Guide by Jeff England Intermetrics, Inc. * Ada is a trademark of the U.S. Department of Defense (AJPO) Performance Analyzer User's Guide Page: i PREFACE This program was developed by Intermetrics, Inc. under Contract Number N66001-85-C-0032 for: Naval Ocean Systems Center 271 Catalina Boulevard San Diego, California 92152. Performance Analyzer User's Guide Page: ii CONTENTS 1.0 DESCRIPTION ....................................... 1 2.0 COMMAND FORMAT .................................... 2 2.1 Source Program Instrumentation .................. 2 2.2 Compilation and Linking ......................... 5 2.3 Target Program Execution ........................ 6 2.4 Report Generation ............................... 8 3.0 OUTPUT FORMAT ..................................... 11 3.1 Test Configuration Report ....................... 11 3.2 Total Execution Times Report .................... 12 3.3 Net Execution Times Report ...................... 15 3.4 Cumulative Execution Times Report ............... 16 3.5 Call Summary Report ............................. 17 4.0 INTERPRETING THE REPORTS .......................... 19 4.1 Execution Times ................................. 19 4.2 Percent of Total Execution Time ................. 19 4.2 Tasks ........................................... 20 4.3 Net Times vs Cumulative Times ................... 20 4.4 Expected Errors ................................. 20 4.5 Clock Faults .................................... 21 4.6 Dangling Units .................................. 22 5.0 RESTRICTIONS AND LIMITATIONS ...................... 23 APPENDIX A - SAMPLE REPORT AND ANALYSIS ................ A-1 APPENDIX B - GLOSSARY .................................. B-1 Performance Analyzer User's Guide Page: 1 1.0 DESCRIPTION The Ada Performance Analyzer, Profile, is one of an integrated set of five tools that provide testing and analysis aids for the development of computer programs implemented in the Ada programming language. Specifically, Profile provides timing information on an Ada program to enable the analyst to identify program units that require a high percentage of total system execution time. Profile consists of three elements: 1) A source instrumenter that parses the target Ada program and inserts "hooks" that transfer control to a runtime kernal upon entry to and exit from each instrumented Ada program unit. Each Ada program unit for which timing information is to be recorded must be instrumented prior to compilation. 2) A runtime execution monitor, or kernel, that gains control upon entry to or exit from each instrumented Ada program unit and logs timing information about the program's execution. The runtime execution monitor must be linked with the instrumented Ada program units. 3) A report generator that analyzes the recorded timing information and produces a comprehensive timing and execution profile in an easily understandable format. Reports generated provide detailed quantitative information about the percentage of total program execution time spent in each instrumented Ada program unit. Profile is a software development tool. Its purpose is not so much to time the execution of the program being analyzed, but rather to aid the programmer or analyst in identifying those program units that require a high percentage of total program execution time. It has been said that in most programs 90% of the total program execution time is usually spent executing 10% of the program's code. Profile can aid the programmer or analyst: O during program development in identifying those parts of the program that may later result in performance problems. O in analyzing existing programs with performance problems to identify inefficiencies or areas that may benefit from optimization and result in significantly improved overall program performance. Performance Analyzer User's Guide Page: 2 2.0 COMMAND FORMAT Operation of the Ada Performance Analyzer consists of four steps: 1) source program instrumentation 2) compilation and linking 3) target program execution 4) report generation Each of these steps is described in detail in the following paragraphs. 2.1 Source Program Instrumentation Each Ada compilation unit containing program units for which execution trace and timing data is to be recorded must be separately instrumented by the source instrumenter. The Performance Analyzer will provide the most accurate timing information on programs and subprograms that have been instrumented at entry/exit only. However, it may also be used to provide timing analysis of programs and subprograms that have been instrumented at decision points or at every statement for use with the Path Analyzer, Automatic Path Analyzer, or Self Metric Instrumentation and Analysis Tool. The Source Instrumenter is one of five tools included in the Ada Test and Analysis System shell, TEST_TOOLS, and is executed as follows: (User input and responses are shown in Bold Face) RUN TEST_TOOLS <cr> The Test_Tools prompt will then be displayed and the user will be asked to enter the name of the tool he wishes to run. To execute the Source Instrumenter the user should respond with the tool name, Source_Instrument. EXAMPLE Welcome to the Test Tools please enter desired tool enter ? for a list of tools available SOURCE_INSTRUMENT <cr> Performance Analyzer User's Guide Page: 3 If "?" is entered then Test_Tools will display a list of the tools available for execution and the user will be reprompted to enter a tool name. If "Q" is entered then execution of the Test_Tools shell will be terminated. EXAMPLE Welcome to the Test Tools please enter desired tool enter ? for a list of tools available ? <cr> The following tools may be executed path -- the report generator for the path analysis tool profile -- the report generator for the timing tool smart -- the report generator for the self metric tool autopath -- the report generator for the automatic path tool source_instrument -- instrument an Ada source file please enter desired tool It is only necesary to enter the first two characters of the tool name, i.e., PA, PR, SM, AP or SO. If an incorrect tool name is entered the user will be prompted to reenter the tool name. When a valid tool name has been entered the user will be prompted to enter file names for the Ada source file to be instrumented, the listing file to be generated, and the instrumented source file to be generated. EXAMPLE Enter Source_File => Test5.ADA <cr> Enter Source_Listing_File => <cr> Enter Instrumented_Source_File => <cr> source_file = TEST5.ADA source_listing = TEST5.LST instrumented_source = TEST.INS If the user does not enter a Source_Listing_File or an Instrumented_Source_File then default file names will be derived from the first qualifier of Source_File_Name and the extentions ".LST" and ".INS", respectively. Performance Analyzer User's Guide Page: 4 When all file names have been entered the Source Instrumenter will display a menu of instrumenting options and query the user to select the options to be used when instrumenting the specified source file. It is recommended that the predefined instrumenting defaults for the preformance analyzer, option 2, be selected. EXAMPLE Source Instrumenter Version 1.0 Instrumenting Options are: 1 - Path/Autopath Analyzer Defaults 2 - Performance Analyzer Defaults 3 - Self Metric Defaults 4 - User Supplied Options Enter option (1, 2, 3, 4, ?, or <cr> for default of 1): 2 <cr> When instrumenting is completed the Source Instrumenter will return control to the TEST_TOOLS shell. At this point the user may instrument another Ada source file or terminate the shell by entering "Q" or "QUIT". EXAMPLE please enter desired tool Q <cr> $ The Source Instrumenter also provides a capability to explicitly select specific program units within a compilation unit for instrumentation. Refer to the Source Instrumenter User's Guide for more detailed information on using this option. Performance Analyzer User's Guide Page: 5 2.2 Compilation and Linking When the instrumentation process is completed the instrumented target Ada program must be compiled and linked to prepare it for execution. Each instrumented Ada compilation unit must be separately compiled. When the compilation process is completed the target program must be linked with the Run Time Monitor. The user should follow the normal procedures for compiling and linking with the Ada compiler system being used for program development. These procedures may vary from system to system. The procedures for compiling and linking the instrumented on a DEC VAX under VMS Version 4.0 and using the DEC VAX Ada Compiler System Version 1.0 are shown below. However this example assumes that the user has already performed all operations necessary to set up an Ada program library. EXAMPLE $ Ada Test5.INS <cr> $ ACS Link Test5 <cr> %ACS-I-CL_LINKING, Invoking the VAX/VMS Linker $ Performance Analyzer User's Guide Page: 6 2.3 Target Program Execution When linking is completed, the target Ada program may be executed. The user interface and method of executing the target program is not altered by the instrumentation process. Therefore, the user should follow the normal procedures for executing the target program on the host computer system being used for program development. When the first unit of the target program is entered, the Run Time Monitor will prompt the user for the following information: 1. Tool_Name : The user should enter PROFILE_TOOL (upper or lower case is acceptable). If the user should enter an invalid tool name, he will be repeatedly prompted until a correct tool name is entered or the program is aborted. 2. Logfile_Name : The user should enter a legal filename for the computer he is using. The default filename may be chosen by entering a carriage return. The default Logfile_Name is PROFILE.LOG. If the File already exists, the user is asked if he would like to Enter a new filename, Overwrite the existing file, or Append to the existing file. 3. Test_Identification: The user is asked to enter a unique test identification which can be any valid Ada string. The default Test ID may be selected by entering a carriage return. The default Test ID for Profile is "Performance Analysis Report". The Test ID is recorded in the log file generated by the Run Time Monitor and printed on the Test Configuration Report generated by the Profile Report Generator. An example session with the Run Time Monitor is presented on the following page. Performance Analyzer User's Guide Page: 7 EXAMPLE SESSION Following is a sample session with the Run Time Monitor. (User responses are shown in Bold Face): $ RUN TEST5 <cr> Enter Tool Type : PATH_TOOL, AUTOPATH_TOOL, SMART_TOOL, or PROFILE_TOOL -----> PROFILE_TOOL <cr> Enter Logfile Name, Null for Default -----> Test5.log <cr> If the Logfile already exists, the following message and prompt are printed: File already exists!!! Do you wish to: E = Enter a New Filename O = Overwrite existing file A = Append to the existing file Enter Option ===> O <cr> The Run Time Monitor will repeatedly prompt the user until a valid option is entered. Enter a unique Test Identification, Null for default. 5 Disks, 3 Tasks and a Bubble Sort <cr> When the Run Time Monitor input is complete, the target program resumes its normal execution. Performance Analyzer User's Guide Page: 8 2.4 Report Generation During execution, the instrumented Ada program will create a log file containing program execution trace and timing data. The Ada Performance Analyzer Report Generator, Profile, analyzes the data and presents it in an easily understood format. The Profile report generator is one of five tools included in the Ada Test and Analysis System shell, TEST_TOOLS. The report generator is executed as follows: (User input and responses are shown in Bold Face) RUN TEST_TOOLS <cr> The Test_Tools prompt will then be displayed and the user will be asked to enter the name of the tool he wishes to run. To execute Profile, the user should respond with the tool name, Profile. EXAMPLE Welcome to the Test Tools please enter desired tool enter ? for a list of tools available PROFILE <cr> It is only necesary to enter the first two characters of the tool name, i.e., PA, PR, SM, AP or SO. If an incorrect tool name is entered the user will be prompted to reenter the tool name. When a valid tool name has been entered the user will be prompted to enter the name of the log file to be processed and the name of the report file to be generated. EXAMPLE Enter Log_File_Name => TEST5.LOG <cr> Enter Report_File_Name => TEST5.RPT <cr> If the user does not enter a Report_File_Name, then a default file name be derived from the first qualifier of Log_File_Name and extention ".RPT". When valid log file and report file names have been entered, Profile will display the test configuration data that was recorded in the log file at the time the program under test was executed. The user is then prompted as to whether or not he wishes to continue. Performance Analyzer User's Guide Page: 9 After examining the test configuration data, the user can discontinue processing by entering "N". If the correct log file has been selected then the user should enter "Y" to continue processing. EXAMPLE Profile 1.0 Ada Performance Analyzer Program Under Test: Test5 Test Date: 06/30/85 Test Time: 20:24:59 Test ID: 5 Disks, 3 Tasks and a Sort Do you wish to continue (Y/N)? Y <cr> If the user enters "N" then processing will be discontinued. If "Y" is entered then processing of the log file will begin. When processing is completed, Profile will display the Test Progam Start Time, Stop Time, and Total Test Execution Time at the user's console. EXAMPLE Test Program Start Time: 15:52:16.04 Test Program Stop Time: 15:53:43.64 Total Test Execution Time: 00:01:27.60 The Run Time Monitor automatically adjusts execution times for the overhead it imposes on the program under test. If the program terminated abnormally then Test Program Stop Time and Total Test Execution Time will already be adjusted for overhead and the message <program name> terminated abnormally will be displayed at the user's console. If the program terminated normally then Test Program Stop Time and Total Test Execution Time will be actual times and the estimated Profile overhead and adjusted execution time will also be displayed at the user's console. EXAMPLE Estimated Profile Overhead: 00:01:03.10 Adjusted Test Execution Time: 00:00:24.50 Performance Analyzer User's Guide Page: 10 Shortly after test program execution times are displayed at the user's console generation of the report file will be completed and ready for analysis by the user. At this point control will be returned to the Test_Tools shell where the user may select another tool for execution or terminate the shell by entering "Q" for Quit. An example session with the Profile Report Generator is presented below. EXAMPLE SESSION Following is a sample session with the Report Generator (User input and responses are shown in Bold Face): $ RUN TEST_TOOLS <cr> Welcome to the Test Tools please enter desired tool enter ? for a list of tools available PR <cr> Enter Log_File_Name => TEST5.LOG <cr> Enter Report_File_Name => <cr> log_file_name = TEST5.LOG report_file_name = TEST5.RPT Profile 1.0 Ada Performance Analyzer Program Under Test: Test5 Test Date: 06/30/85 Test Time: 15:52:16 Test ID: 5 Disks, 3 Tasks and a Sort Do you wish to continue (Y/N)? Y <cr> Test Program Start Time: 15:52:16.04 Test Program Stop Time: 15:53:43.64 Total Test Execution Time: 00:01:27.60 Estimated Profile Overhead: 00:01:03.10 Adjusted Test Execution Time: 00:00:24.50 please enter desired tool quit <cr> $ Performance Analyzer User's Guide Page: 11 3.0 OUTPUT FORMAT The report file generated by PROFILE contains five reports: a Test Configuration Report, three different execution timing reports, and a Call Summary Report. Each report provides the user with different information about the program being tested and is generated in 80 column format to enable "browsing" at a terminal when hardcopy is either unavailable or not desired. A sample report file is included in Appendix A. Prior to using the Profile report generator it is important for the user to understand the information that is being presented and how to interpret it. Each of the execution timing reports is described below. A more general discussion on evaluating information presented in these reports is included in section 4.0. 3.1 Test Configuration Report The Test Configuration Report includes information necessary to correlate the report with a specific test or execution of the target Ada program. It includes the following information: 1) Program Under Test: This is the name of the main program unit of the program being analyzed. 2) Test Date: The calendar date that the program under test was executed and the execution log file was created. The test date is presented in the form MM/DD/YY. 3) Test Day: The day of the week, Sunday through Saturday, that the program under test was executed and the execution log file was created. 4) Test Time: The time of day that the program under test was executed and the execution log file was created. The test time is presented in the form HH:MM:SS. 5) Log File: The name of the log file that was used in the preparation of the execution timing reports. 6) Test ID: The test ID specified by the user at the time the program under test was executed. Performance Analyzer User's Guide Page: 12 7) Test Program Start Time: The time that the first program unit began execution. Start time is presented in the form HH:MM:SS.SS. 8) Test Program Stop Time: This is the last time recorded in the log file. If the program under test terminated normally then this is the time that the main program unit ended. Stop time is presented in the form HH:MM:SS.SS. 9) Total Test Execution Time: The total elapsed test execution time presented in the form HH:MM:SS.SS. If the program under test terminated abnormally then Test Program Stop Time and Total Test Execution Time will already be adjusted for overhead. If the program terminated normally then the above times will be actual unadjusted execution times and the Test Configuration Report will also include the following: 10) Estimated Profile Overhead: This is the estimated overhead imposed by the Run Time monitor on the execution of the program under test. The estimated overhead is presented in the form HH:MM:SS.FF. 11) Adjusted Test Execution Time: This is the total test execution adjusted for the estimated overhead imposed by the Run Time Monitor. 3.2 Total Execution Times Report This report presents both "net" and "cumulative" execution times for each instrumented Ada program unit. It is sorted in descending order by cumulative execution time. The "Net" execution time for a program unit is the total accumulated time spent in that program unit. It does not include time spent in other program units called by that unit if they have also been instrumented by the source instrumenter. The "Cumulative" execution time for a program unit is the total accumulated time spent in that program unit and all other program units called by that unit whether or not the other program units have been instrumented by the source instrumenter. For Example, assume program A contains the three program units A, B, and C. The main program unit, A, calls both B and C. Only program units A and B have been instrumented. Performance Analyzer User's Guide Page: 13 The "net" execution time reported for A will include all time spent in program units A and C, since C is not instrumented. The "cumulative" execution time reported for A will include all time spent in A and all other program units called by A ( i.e., B and C ). Since B does not call any other program units, B's "net" and "cumulative" execution times will be equal. No execution times will be reported for program unit C since C is not instrumented. REPORT COLUMNS The Total Execution Times Report presents the following information about each instrumented Ada program unit: 1) UNIT NAME ( Program Unit Name ) This is the fully qualified name of the procedure, function, task, generic or package. The field is limited to 29 characters. If the program unit name is longer than 29 characters then multiple lines are used until all characters in the name have been printed. Continuation lines are indented two spaces. If the program unit name has been overloaded within the same compilation unit and cannot be uniquely identified by its name alone then the user must refer to the program unit number (PU#). 2) PU# ( Program Unit Number ) Each instrumented Ada program unit within a compilation unit is assigned a unique Program Unit Number by the Source Instrumenter to provide for unique identification of overloaded program unit names. Normally, it will not be necessary for the user to refer to the listing file generated by the Source Instrumenter. However, if the program under test makes use of "overloading" of program unit names it may be necessary to refer to the Source Instrumenter Listing in order to correlate timing data presented in the reports to a specific program unit. 3) T ( Program Unit Type ) Ada programs may contain four different types of program units: procedures, functions, tasks and generics. However, the Ada language also provides the ability to include executeable code within a package body that is not included within any of the program units contained in that package. This code is executed during package initialization when the package is elaborated. Profile reports timing information for this "initialization" code the same as for Ada program units. Therefore, for reporting purposes, initialization code within a package body is also treated as a program unit. Performance Analyzer User's Guide Page: 14 Program unit types are coded as follows: P Procedure F Function T Task G Generic K pacKage 4) # EXEC ( Number of Executions ) Normally, this is the number of times a program unit was executed. However, if the program unit terminated abnormally then # Exec represents the number of times the program unit began execution. 5) NET ( Net Execution Time ) This is the accumulated net time that a program unit was active. It is the sum of the net times for all executions of the program unit. 6) CUMULATIVE ( Cumulative Execution Time ) This is the accumulated cumulative time that a program unit was active. It is the sum of the cumulative times for all executions of the program unit and all other program units called by that program unit. 7) %T ( Percent of Total Execution Time ) This is the program unit's execution time as a percentage of total program execution time. It is displayed as a two digit integer in the range 0 to 99. Special characters, discussed in section 4, are used to represent percentages in excess of 99%. 8) %E ( Percent of Expected Error ) %E is the expected error in the reported time as a percentage of the reported time. %E is reported for both NET and CUMULATIVE execution times. The accuracy of the times reported is a function of the total length of execution time of the program unit, the number of times the unit executed, the duration of the test, and the basic clock period (System.Tick). In General, the larger %T for a program unit's reported execution time, the smaller %E will be. Performance Analyzer User's Guide Page: 15 3.3 Net Execution Time Report This report includes the "net" time spent in each instrumented Ada program unit. The time reported for each unit does not include time spent in other program units called by that unit if they have also been instrumented by the Source Instrumenter. The Net Execution Times Report is sorted by Program Unit Name. REPORT COLUMNS The Net Execution Times Report presents the following information about each instrumented Ada program unit: 1) UNIT NAME ( Program Unit Name ) This is the fully qualified name of the procedure, function, task, generic or package. The field is limited to 29 characters. If the program unit name is longer than 29 characters then multiple lines are used until all characters in the name have been printed. Continuation lines are indented two spaces. 2) PU# ( Program Unit Number ) This is the Program Unit Number assigned by the Source Instrumenter to provide for unique identification of overloaded Ada program unit names. 3) # EXEC ( Number of Executions ) Normally, this is the number of times the program unit was executed. However, if the program unit terminated abnormally then # Exec represents the number if times the program unit began execution. 4) MAXIMUM ( Maximum Net Execution Time ) This is the maximum net execution time reported for a single execution of the program unit. 5) MINIMUM ( Minimum Net Execution Time ) This is the minimum net execution time reported for a single execution of the program unit. 6) AVERAGE ( Average Net Execution Time ) This is the average net execution time reported for all executions of the program unit. 7) %E ( Percent of Expected Error ) %E is the expected error in the total accumulated net execution time as a percentage of the unit's total accumulated net execution time. This is the same %E reported for NET on the Total Execution Times Report. Performance Analyzer User's Guide Page: 16 3.4 Cumulative Execution Times Report This report includes the "cumulative" time spent in each instrumented Ada program unit. The time reported for each unit also includes all time spent in other program units called by that unit whether or not the other program units have been instrumented by the source instrumenter. The Cumulative Execution Times Report is sorted by Program Unit Name. REPORT COLUMNS The Cumulative Execution Times Report presents the following information about each instrumented Ada program unit: 1) UNIT NAME ( Program Unit Name ) This is the fully qualified name of the procedure, function, task, generic or package. The field is limited to 29 characters. If the program unit name is longer than 29 characters then multiple lines are used until all characters in the name have been printed. Continuation lines are indented two spaces. 2) PU# ( Program Unit Number ) This is the Program Unit Number assigned by the Source Instrumenter to provide for unique identification of overloaded Ada program unit names. 3) # EXEC ( Number of Executions ) Normally, this is the number of times the program unit was executed. However, if the program unit terminated abnormally then # Exec represents the number of times the program unit began execution. 4) MAXIMUM ( Maximum Cumulative Execution Time ) This is the maximum cumulative execution time reported for a single execution of the program unit. 5) MINIMUM ( Minimum Cumulative Execution Time ) This is the minimum cumulative execution time reported for a single execution of the program unit. 6) AVERAGE ( Average Cumulative Execution Time ) This is the average cumulative execution time reported for all executions of the program unit. 7) %E ( Percent of Expected Error ) %E is the expected error in the total accumulated cumulative execution time as a percentage of the unit's total accumulated cumulative execution time. This is the same %E reported for CUMULATIVE on the Total Execution Times Report. Performance Analyzer User's Guide Page: 17 3.5 Call Summary Report This report provides a summary of all calls by instrumented Ada program units to other instrumented Ada program units. The call summary report can aid the user in evaluating program unit timing information presented in the execution timing reports. REPORT COLUMNS The Call Summary Report presents the following information about each instrumented Ada program unit: 1) UNIT NAME ( Program Unit Name ) This is the fully qualified name of the procedure, function, task, generic or package. The field is limited to 29 characters. If the program unit name is longer than 29 characters then multiple lines are used until all characters in the name have been printed. Continuation lines are indented two spaces. 2) PU# ( Program Unit Number ) This is the Program Unit Number assigned by the Source Instrumenter to provide for unique identification of overloaded Ada program unit names. 3) T ( Program Unit Type ) Ada program unit types are coded as follows: P Procedure F Function T Task G Generic K pacKage 4) EXECS ( Number of Executions ) Normally, this is the number of times a program unit was executed. However, if the program unit terminated abnormally then # Exec represents the number of times the program unit began execution. 5) ABEND ( Abnormal End ) ABEND indicates the recursion level of each program unit at the time the program under test terminated execution. Normally, the recursion level will be zero for each program unit at the time the program terminated. However, if the program under test terminated abnormally then ABEND can be used to identify "dangling" program units and their recursion level at the time execution ended. Performance Analyzer User's Guide Page: 18 6) MAX RL ( Maximum Recursion Level ) For non-task program units MAX RL is the maximum level of recursion that was reached during execution of the program. For tasks, MAX RL represents the maximum number of tasks of a given type that were in execution concurrently. Depending on compiler implementation, it is possible that a recursive procedure may require significantly more execution time than the same procedure implemented without recursion. This may be significant if both the number of executions and the recursion level are high. 7) SONS Sons represents the number of direct calls to other instrumented Ada program units. It is the total for all executions of the program unit and does not include calls to uninstrumented program units. 8) GRANDSONS Grandsons represents the number of indirect calls to other instrumented Ada program units, i.e., the number of instrumented units called by the units sons. It is the total for all executions of the program unit and does not include calls to uninstrumented program units. Performance Analyzer User's Guide Page: 19 4.0 INTERPRETING THE REPORTS It is important for the user to remember that Profile is not a CPU monitor and to understand how timing estimates are calculated in order to properly analyze and interpret data presented in the reports. 4.1 Execution Times The Run Time Monitor records the time of day each time an instrumented program unit begins and ends execution. It also keeps track of the accumulated time, or overhead, that it requires to log this information. All times that the Run Time Monitor records are automatically adjusted for this overhead. Although execution times that have been adjusted for overhead are closer to the true execution times of the same program uninstrumented, not all overhead can be measured by the Run Time Monitor. Consequently, adjusted execution times for an instrumented program will still be greater than actual execution times for the uninstrumented program. However, the user should keep in mind that the purpose of Profile is not to time the actual execution times of the program being tested. It is a program development "tool" to aid the user in identifying program units that require a high percentage of program execution time. Therefore, the greatest reliance on timing information presented in the various execution timing reports should be on Percentage of Total Execution Time (%T). 4.2 Percent of Total Execution Time (%T) Percent of total execution time is the program unit's execution time as a percentage of total program execution time. It is displayed as a two digit integer in the range 0 to 99. Special characters, as described below, are used to represent percentages in excess of 99%. %T is calculated and presented for both NET and CUMULATIVE execution times for each program unit. Normally, the main program unit will have a cumulative execution time of 100%. However, if the program contains tasks it is possible that the tasks will begin execution prior to the main program unit. WITH'd packages may also contain code that is executed during elaboration. In both of these situations the main program will have a %T of less than 100%. Performance Analyzer User's Guide Page: 20 %T is represented as follows: Percentage Represented By ------------ ------------------------------------ 0.0 - 99.49 "nn" where nn is the nearest integer 99.5 - 99.99 " *" > 99.99 "**" 4.3 Tasks Tasks are considered to run concurrently with other program units. Both "net" and "cumulative" execution times for tasks are based solely on when the task begins and when it ends execution. While there may be some form of time sharing or interleaving of execution, Profile is not a CPU monitor and is not able to determine when a task, or for that matter any other program unit, actually has control of the CPU. Therefore, times are reported for tasks as though each task was running in a dedicated processor. The execution times for Tasks are not included in the cumulative execution times of the program units that activate them. 4.4 Net Times vs Cumulative Times In general, the sum of the net times of all instrumented units in a program will be equal to the cumulative execution time of the main program unit. Slight variations may occur due to roundoff errors. However, as mentioned before, Tasks are considered to run concurrently with other program units. Consequently, the net times for tasks are not included in the cumulative times of their parent units. 4.5 Expected Errors It is possible for an Ada program unit with an execution time of less than the basic clock period that has executed one or more times to report maximum, minimum, average, and total execution times of 00:00:00.00. It is also possible for the same program unit to report a total execution time equal to the basic clock period multiplied by the number of times the unit executed. Performance Analyzer User's Guide Page: 21 Expected error is an estimation of the maximum expected error in the reported time of a program unit and is given as as a percentage of the reported time. Expected error is reported for both NET and CUMULATIVE accumulated execution times. The accuracy of the times reported is a function of the length of execution time of the program unit, the number of times the unit executed, the duration of the test, and the basic clock period (System.Tick). In general, the smaller a unit's percentage of total program execution time, the larger the expected error. However, the maximum error can be no greater than the 50% of the number of clock accesses, or starts and stops, for the program unit. No expected errors are reported for "dangling units" or units that have been flagged for clock faults. The Expected Error column of the execution times reports will contain "C" if a clock fault occurs and "D" if the unit is a dangling unit. "CD" indicates that the unit was flagged for a clock fault and was also a dangling unit. 4.6 Clock Faults A Clock Fault is not the result of a program error but rather and indication of a condition that may result in the incorrect reporting of execution times for one or more program units. A Clock Fault occurs when a program unit ends execution and its starting time is not found on the "top" of an internal stack maintained by the report generator. If a program unit is called by a task, a Clock fault may or may not occur when the program unit terminates. If a Clock Fault occurs then all program units that began execution after the unit in which the fault occurred and were still active when the unit terminated are flagged. Although execution times for flagged units will, in most cases, still be reported accurately, the confidence level in the accuracy of the times is significantly reduced. Therefore, no Expected Error (%E) is reported for program units that have been flagged with a Clock Fault. If a Clock fault occurs then the Expected Error column of the execution times reports will contain "C". Performance Analyzer User's Guide Page: 22 4.7 Dangling Units A "dangling unit" is a program unit that was still "active" when the end of the log file was encountered by the Profile report generator. This condition normally indicates that the program under test terminated abnormally. However, it can also occur if a portion of the log file is destroyed after generation by the Run Time Monitor and before processing by the report generator. Although execution times for dangling units will still be reported, the confidence level in the accuracy of the times is significantly reduced. Therefore, no Expected Error (%E) is reported for dangling program units. The Expected Error column of the execution times reports for dangling units will contain "D". Performance Analyzer User's Guide Page: 23 5.0 RESTRICTIONS AND LIMITATIONS 1) No specific limits are imposed either by the Runtime Monitor or the Profile report generator on either the size of the target Ada program or the number of program units that can be instrumented. All arrays and stacks maintained by Profile that are a function of the total number of program units or the number of program units active at a given time are allocated dynamically and deallocated when they are no longer needed. 2) Execution and timing data may only be reported for program units that have been instrumented by the source instrumenter. 3) Regardless of whether or not the Ada compiler system being used for program development provides an interface to programming languages other than Ada, only Ada program units may be instrumented. 4) The runtime execution monitor imposes a certain amount of overhead on the execution time of each instrumented Ada program unit. This overhead will vary from system to system. If the program under test terminates normally then Profile will report the amount of overhead that was measured and adjusted for the Run Time Monitor. However, not all of the overhead can be compensated for without greatly reducing the accuracy of the reported times. 5) In addition to overhead imposed by the Run Time Monitor, execution time of the program under test is affected by many factors such as process scheduling algorithms of the host computer system, the output device assigned to log files, virtual memory schema, and the number of other processes active during the performance analysis. Profile is NOT a CPU monitor and cannot determine actual CPU times for the program under test. All times reported are elapsed wall clock times which are the result of starting and ending times for each instrumented Ada program unit. Therefore, the most accurate timing data will be obtained when the instrumented test program is executed stand alone. 6) Timing information reported for Ada program units is limited by the real-time clock interval for the host computer. Performance Analyzer User's Guide Page: 24 7) It is not possible to determine the names of objects of task types in the Run Time monitor; only the type of the task and the number of times a task of that type has executed. Therefore, PROFILE reports the TOTAL, MAX, MIN, and AVERAGE execution times for all tasks of each type. 8) It is not possible to detect that a unit has been entered until it actually begins execution. Therefore, any initialization time required for a program unit prior to beginning execution is "charged" to the calling unit. Performance Analyzer User's Guide Page: A - 1 APPENDIX A SAMPLE REPORT AND ANALYSIS This appendix contains Performance Analysis Reports generated by Profile during testing of a real Ada program. In addition, an analysis of each report is given in order to guide the user in interpreting the information presented. Performance Analyzer User's Guide Page: A - 2 The reports in this appendix have been prepared using the the program Profile as an example. Profile was instrumented and analyzed while it performed a performance analysis of another instrumented program. Before we analyze the reports it might be helpful to have a brief overview of the structure of the program. The package PROFILE_PKG contains 1 procedure, PROFILE. The procedure PROFILE contains 14 other program units: CALCULATE_EXPECTED_ERROR PRINT_NET_EXECUTION_TIMES PERCENT_OF_ERROR PRINT_TOTAL_EXECUTION_TIMES GET_UNIT_CLOCK PRINT_UNIT_STATS NEW_REPORT_LINE READ_LOG_FILE PERCENT_OF_TOTAL SORT PRINT_CALL_SUMMARY_REPORT ORDER PRINT_CUM_EXECUTION_TIMES UPDATE_CURRENT_CLOCK In addition to the predefined library packages STANDARD, CALENDAR, and TEXT_IO, PROFILE also directly with's the following other packages: CLOCKS READ_LOG TIME_LIBRARY_2 DYNARRAY_PKG REPORT_LIBRARY STACK_PKG HEAP_SORT TIME_LIBRARY_1 STRING_PKG TYPE_DEFINITIONS IMPLEMENTATION_DEPENDENCIES Although other packages are with'd by the packages above, they are not currently of interest. For the purposes of this analysis two compilation units were instrumented: the package body PROFILE_PKG and the package body CLOCKS. The package clocks also contains the following program units: CREATE_CLOCKS PAUSE_UNIT START_UNIT DESTROY_CLOCKS PREVIOUS_UNIT STOP_UNIT MORE_TASKS RESTART_UNIT TASK_OF MORE_UNITS The Source Instrumenter defaults for performance analysis were selected for both PROFILE_PKG and CLOCKS (instrument all program units in the compilation unit for entry and exit only). Performance Analyzer User's Guide Page: A - 3 ANALYSIS 1. Test Configuration Report This report tells us the test configuration information that was recorded in the log file at the time it was generated. This enables us to verify that we have used the correct log file. In addition it tells us that the overhead imposed by the Run Time Monitor during execution of the target program, PROFILE, accounted for nearly 50% of the total execution time. This is not necessarily typical. As discussed earlier overhead is a function of many factors and execution time may vary anywhere from about 1.1:1 up to about 20:1, depending on the program being tested. Performance Analyzer User's Guide Page: A - 4 2. Total Execution Times Report This report tells us the most about where profile is spending its time. A quick look at the "Percent of Total Execution Time" column (%T) for cumulative execution times tells us something we already know, that the top level procedure, PROFILE, is the main program unit and had a cumulative execution time of 100% (indicated by "*"). However, the line directly below that tells us that the program unit READ_LOG_FILE, although it was executed only 1 time, had a cumulative execution time of 80%. But, looking to the left we see that it had a net execution time of only 23%. Performance Analyzer User's Guide Page: A - 5 Although 23% is a large portion of total program execution time, remember that "net" execution time also includes time spent in other called program units that have not been instrumented. In this case it may not be out of line since it includes times for all READ_LOG procedures, which performs a great deal of I/O. The conclusion that we draw here is that 57% (80% - 23%) of total program execution time is spent in other instrumented program units that are called by READ_LOG_FILE. Other instrumented program units that are called by READ_LOG_FILE include the procedures UPDATE_CURRENT_CLOCK, GET_UNIT_CLOCK and all of the procedures in the package CLOCKS. Totaling up all of the program units in the package CLOCKS we see that together they used approximately 40% of total program execution time, but individually the most time used by any single unit was only 9%. Perhaps inspection of the CLOCKS package would reveal some areas of the code that could be more efficient. Certainly the total amount of time spent in that package warrants a closer examination. The procedure GET_UNIT_CLOCK and the procedure UPDATE_CURRENT_CLOCK together used approximately 40% of total program execution time and also warrant closer investigation. They use the dynamic array management package DYNARRAY which has not been instrumented. Before any definite conclusions are drawn from the Total Execution Times Report, we will continue on to the other reports to determine whether or not they reveal more about the program. Performance Analyzer User's Guide Page: A - 6 3. Net Execution Times Report A quick glance at this report shows us that several program units that have high execution counts also have a wide variation between maximum and minimum execution times. Looking at the code for one of these units that is executed 232 times, MORE_UNITS, it is obvious that a call to the uninstrumented function STACK_PKG.IS_EMPTY could be avoided on each execution if a local counter were maintained. Further investigation could reveal that the search algorithm used by GET_UNIT_CLOCK could be more efficient to reduce the number of accesses to the package DYNARRAY. Performance Analyzer User's Guide Page: A - 7 4. Cumulative Execution Times Report This report confirms what we saw on the Net Execution Times Report: a wide variation between maximum and minimum times for program units that had a high frequency of execution. Performance Analyzer User's Guide Page: A - 8 5. Call Summary Report The first thing we see when we look at this report is that the program terminated normally. The Anormal End (ABEND) level of zero for each program unit indicates that all program units had terminated execution and that the log file was properly closed. If the program had terminated abnormally this field would tell us which program units were active and their recursion level at the time of termination. This could be very important in assessing the accuracy of the information presented. Performance Analyzer User's Guide Page: A - 9 We also see that the Maximum Recursion Level (MAX RL) for all instrumented program units was 1, indicating that none of the program units instrumented executed recursively. The main program unit, PROFILE, called 8 other instrumented program units, which in turn made 1109 calls to other instrumented program units. Looking down to READ_LOG_FILE we see that 1001 of PROFILE's 1109 grandsons were called while in Read_Log_File. This is consistent with the statistic that 80% of total program execution time was spent in this unit. CONCLUSIONS We have been able to draw the following conclusions from the information presented about the program PROFILE. 1. Approximately 80% of total program execution time is spent in the procedure READ_LOG_FILE and other program units that it calls. This procedure also has the largest number of direct descendents. This would be the first place to start looking for inefficiencies. 2. The program units PAUSE_UNIT, RESTART_UNIT, MORE_UNITS, and GET_UNIT_CLOCK are not only the most frequently called units but they also all use a substantial amount of total program execution time. Any optimization in these units should result in a sizeable reduction in total program execution time. 3. The next step might be to instrument the package READ_LOG to determine how much of READ_LOG_FILE's time is actually spent in READ_LOG program units. If this were done it would reveal that READ_LOG_FILE's net time is only about 10% of total program execution time. The other 13% (23% - 10%) is actually spent performing I/O. It would also reveal that READ_LOG contains a recursive function call that might warrant close inspection. Minor optimizations in recursive program units can often result in significant performance improvements if they have a high frequency of execution. Performance Analyzer User's Guide Page: B - 1 APPENDIX B GLOSSARY Performance Analyzer User's Guide Page: B - 2 Adjusted Execution Time Actual recorded execution time adjusted for measured overhead imposed by the Run Time Monitor. Not all overhead can be measured. No adjustment is made for that portion of overhead that is not measured. Clock Fault A Clock Fault is an indication of a condition that may result in the incorrect reporting of execution times for one or more program units. Execution times for flagged units will, in most cases, still be reported accurately. However, confidence levels in the accuracy of the reported times for faulted units is greatly reduced. Cumulative Execution Time The "Cumulative" execution time for a program unit is the total accumulated time spent in that program unit and all other program units called by that unit whether or not the other program units have been instrumented by the source instrumenter. Dangling Unit A "dangling unit" is a program unit that was still "active" when the end of the log file was encountered by the Profile report generator. This condition normally indicates that the program under test terminated abnormally. Execution Log File A sequential file created by the Run Time Monitor during execution of an Ada program that has been instrumented by the Source Instrumenter. The terms "Execution Log File" and "Log File" are interchangeable. Expected Error Expected error is an estimation of the maximum expected error in the reported time of a program unit and is given as as a percentage of the reported time. Expected error is a function of the length of execution time of a program unit, the number of times the unit executed, the duration of the test, and the basic clock period (System.Tick). Log File A sequential file created by the Run Time Monitor during execution of an Ada program that has been instrumented by the Source Instrumenter. The terms "Execution Log File" and "Log File" are interchangeable. Performance Analyzer User's Guide Page: B - 3 Net Execution Time The "Net" execution time for a program unit is the total accumulated time spent in that program unit. It does not include time spent in other program units called by that unit if they have also been instrumented by the source instrumenter. Overhead The term "overhead" refers to the additional execution time imposed on the program under test by the Run Time Monitor. Only a portion of this overhead can be measured with any degree of accuracy. If the program under test terminates normally the Profile Report Generator will, if requested by the user, adjust execution times for that portion of the overhead that was measured by the Run Time Monitor. Program Unit Ada programs may contain four different types of program units: procedures, functions, tasks and generics. However, the Ada language also provides the ability to include executeable code within a package body that is not included within any of the program units contained in that package. This code is executed during package initialization when the package is elaborated. Profile reports timing information for this "initialization" code the same as for Ada program units. Therefore, for reporting purposes, initialization code within a package body is also treated as a program unit. Report Time The time that the Profile Report Generator was executed. Run Time Monitor A runtime kernel that gains control upon entry to and exit from each instrumented Ada program unit and logs timing information about the program's execution. The runtime execution monitor must be linked with the instrumented Ada program units. Source Instrumenter An Ada program that parses the target Ada program and inserts "hooks" that transfer control to the Run Time Monitor upon entry to and exit from each instrumented Ada program unit. Each Ada program unit for which timing information is to be recorded must be instrumented by the Source Instrumenter prior to compilation. Performance Analyzer User's Guide Page: B - 4 Test ID A unique test identification specified by the user when the program under test is executed. The Test ID is stored in the log file by the Run Time Monitor and included in the Test Configuration Report by the Profile report generator. Test Time The date and time that the program under test began execution. Profile 1.0 MONDAY 07/22/85 22:17:50 Page: 1 Ada Performance Analyzer - Test Configuration Report ------------------------------------------------------------------------------- Program Under Test: PROFILE_PKG Test Date: 07/22/85 Test Day: MONDAY Test Time: 22:11:18 Log File: Profile.Log Test ID: Performance Analysis of Profile Version 1.0 ------------------------------------------------------------------------------- ========================================= Test Program Start Time: 22:11:45.52 Test Program Stop Time: 22:16:44.95 Total Test Execution Time: 00:04:59.43 Estimated Profile Overhead: 00:02:02.86 Adjusted Test Execution Time: 00:02:56.57 ========================================= Profile 1.0 MONDAY 07/22/85 22:17:50 Page: 2 TOTAL EXECUTION TIMES ------------------------------------------------------------------------------- UNIT NAME PU# T # EXEC NET %T %E CUMULATIVE %T %E ----------------------------- --- - ------ ----------- -- -- ----------- -- -- PROFILE_PKG.PROFILE 001 P 1 00:00:07.10 4 1 00:02:56.57 * 0 PROFILE_PKG.PROFILE. READ_LOG_FILE 004 P 1 00:00:40.19 23 2 00:02:20.53 80 0 PROFILE_PKG.PROFILE. UPDATE_CURRENT_CLOCK 003 P 78 00:00:14.38 8 4 00:00:24.67 14 2 PROFILE_PKG.PROFILE. GET_UNIT_CLOCK 002 P 156 00:00:16.01 9 3 00:00:16.01 9 3 CLOCKS.PAUSE_UNIT 007 P 148 00:00:15.80 9 3 00:00:15.80 9 3 CLOCKS.RESTART_UNIT 005 P 148 00:00:15.72 9 3 00:00:15.72 9 3 CLOCKS.MORE_UNITS 012 F 232 00:00:15.19 9 3 00:00:15.19 9 3 CLOCKS.STOP_UNIT 006 P 78 00:00:08.99 5 6 00:00:09.26 5 6 PROFILE_PKG.PROFILE.SORT 005 P 2 00:00:02.55 1 16 00:00:08.67 5 0 CLOCKS.START_UNIT 004 P 78 00:00:08.49 5 6 00:00:08.49 5 6 PROFILE_PKG.PROFILE. PRINT_TOTAL_EXECUTION_TIMES 011 P 1 00:00:05.10 3 4 00:00:06.29 4 0 PROFILE_PKG.PROFILE.SORT. ORDER 006 F 38 00:00:06.12 3 6 00:00:06.12 3 6 PROFILE_PKG.PROFILE. PRINT_UNIT_STATS 012 P 14 00:00:04.41 2 6 00:00:05.18 3 3 CLOCKS.PREVIOUS_UNIT 011 F 77 00:00:05.14 3 8 00:00:05.14 3 8 PROFILE_PKG.PROFILE. PRINT_NET_EXECUTION_TIMES 013 P 1 00:00:02.03 1 4 00:00:04.61 3 0 PROFILE_PKG.PROFILE. PRINT_CUM_EXECUTION_TIMES 014 P 1 00:00:01.93 1 4 00:00:04.53 3 0 PROFILE_PKG.PROFILE. CALCULATE_EXPECTED_ERROR 007 P 1 00:00:00.92 1 16 00:00:03.02 2 0 PROFILE_PKG.PROFILE. CALCULATE_EXPECTED_ERROR. PERCENT_OF_ERROR 008 F 14 00:00:02.10 1 7 00:00:02.10 1 7 PROFILE_PKG.PROFILE. NEW_REPORT_LINE 009 F 28 00:00:01.98 1 14 00:00:01.98 1 14 PROFILE_PKG.PROFILE. PRINT_CALL_SUMMARY_REPORT 015 P 1 00:00:01.32 1 6 00:00:01.82 1 1 PROFILE_PKG.PROFILE. PERCENT_OF_TOTAL 010 F 14 00:00:00.48 0 20 00:00:00.48 0 20 CLOCKS.TASK_OF 001 F 3 00:00:00.27 0 11 00:00:00.27 0 11 CLOCKS.DESTROY_CLOCKS 003 P 1 00:00:00.15 0 7 00:00:00.15 0 7 CLOCKS.MORE_TASKS 013 F 1 00:00:00.13 0 8 00:00:00.13 0 8 CLOCKS.CREATE_CLOCKS 002 P 1 00:00:00.04 0 20 00:00:00.04 0 20 Profile 1.0 MONDAY 07/22/85 22:17:50 Page: 3 NET EXECUTION TIMES ------------------------------------------------------------------------------- UNIT NAME PU# # EXEC MAXIMUM MINIMUM AVERAGE %E ----------------------------- --- ------ ----------- ----------- ----------- -- CLOCKS.CREATE_CLOCKS 002 1 00:00:00.04 00:00:00.04 00:00:00.04 20 CLOCKS.DESTROY_CLOCKS 003 1 00:00:00.15 00:00:00.15 00:00:00.15 7 CLOCKS.MORE_TASKS 013 1 00:00:00.13 00:00:00.13 00:00:00.13 8 CLOCKS.MORE_UNITS 012 232 00:00:00.28 00:00:00.02 00:00:00.07 3 CLOCKS.PAUSE_UNIT 007 148 00:00:00.77 00:00:00.03 00:00:00.11 3 CLOCKS.PREVIOUS_UNIT 011 77 00:00:00.29 00:00:00.02 00:00:00.07 8 CLOCKS.RESTART_UNIT 005 148 00:00:00.64 00:00:00.03 00:00:00.11 3 CLOCKS.START_UNIT 004 78 00:00:00.47 00:00:00.03 00:00:00.11 6 CLOCKS.STOP_UNIT 006 78 00:00:00.62 00:00:00.03 00:00:00.12 6 CLOCKS.TASK_OF 001 3 00:00:00.09 00:00:00.09 00:00:00.09 11 PROFILE_PKG.PROFILE 001 1 00:00:07.10 00:00:07.10 00:00:07.10 1 PROFILE_PKG.PROFILE. CALCULATE_EXPECTED_ERROR 007 1 00:00:00.92 00:00:00.92 00:00:00.92 16 PROFILE_PKG.PROFILE. CALCULATE_EXPECTED_ERROR. PERCENT_OF_ERROR 008 14 00:00:00.16 00:00:00.14 00:00:00.15 7 PROFILE_PKG.PROFILE. GET_UNIT_CLOCK 002 156 00:00:00.34 00:00:00.03 00:00:00.10 3 PROFILE_PKG.PROFILE. NEW_REPORT_LINE 009 28 00:00:00.17 00:00:00.04 00:00:00.07 14 PROFILE_PKG.PROFILE. PERCENT_OF_TOTAL 010 14 00:00:00.04 00:00:00.03 00:00:00.03 20 PROFILE_PKG.PROFILE. PRINT_CALL_SUMMARY_REPORT 015 1 00:00:01.32 00:00:01.32 00:00:01.32 6 PROFILE_PKG.PROFILE. PRINT_CUM_EXECUTION_TIMES 014 1 00:00:01.93 00:00:01.93 00:00:01.93 4 PROFILE_PKG.PROFILE. PRINT_NET_EXECUTION_TIMES 013 1 00:00:02.03 00:00:02.03 00:00:02.03 4 PROFILE_PKG.PROFILE. PRINT_TOTAL_EXECUTION_TIMES 011 1 00:00:05.10 00:00:05.10 00:00:05.10 4 PROFILE_PKG.PROFILE. PRINT_UNIT_STATS 012 14 00:00:00.43 00:00:00.28 00:00:00.31 6 PROFILE_PKG.PROFILE. READ_LOG_FILE 004 1 00:00:40.19 00:00:40.19 00:00:40.19 2 PROFILE_PKG.PROFILE.SORT 005 2 00:00:01.39 00:00:01.16 00:00:01.27 16 PROFILE_PKG.PROFILE.SORT. ORDER 006 38 00:00:00.20 00:00:00.14 00:00:00.16 6 PROFILE_PKG.PROFILE. UPDATE_CURRENT_CLOCK 003 78 00:00:00.79 00:00:00.05 00:00:00.18 4 Profile 1.0 MONDAY 07/22/85 22:17:50 Page: 4 CUMULATIVE EXECUTION TIMES ------------------------------------------------------------------------------- UNIT NAME PU# # EXEC MAXIMUM MINIMUM AVERAGE %E ----------------------------- --- ------ ----------- ----------- ----------- -- CLOCKS.CREATE_CLOCKS 002 1 00:00:00.04 00:00:00.04 00:00:00.04 20 CLOCKS.DESTROY_CLOCKS 003 1 00:00:00.15 00:00:00.15 00:00:00.15 7 CLOCKS.MORE_TASKS 013 1 00:00:00.13 00:00:00.13 00:00:00.13 8 CLOCKS.MORE_UNITS 012 232 00:00:00.28 00:00:00.02 00:00:00.07 3 CLOCKS.PAUSE_UNIT 007 148 00:00:00.77 00:00:00.03 00:00:00.11 3 CLOCKS.PREVIOUS_UNIT 011 77 00:00:00.29 00:00:00.02 00:00:00.07 8 CLOCKS.RESTART_UNIT 005 148 00:00:00.64 00:00:00.03 00:00:00.11 3 CLOCKS.START_UNIT 004 78 00:00:00.47 00:00:00.03 00:00:00.11 6 CLOCKS.STOP_UNIT 006 78 00:00:00.62 00:00:00.03 00:00:00.12 6 CLOCKS.TASK_OF 001 3 00:00:00.09 00:00:00.09 00:00:00.09 11 PROFILE_PKG.PROFILE 001 1 00:02:56.57 00:02:56.57 00:02:56.57 0 PROFILE_PKG.PROFILE. CALCULATE_EXPECTED_ERROR 007 1 00:00:03.02 00:00:03.02 00:00:03.02 0 PROFILE_PKG.PROFILE. CALCULATE_EXPECTED_ERROR. PERCENT_OF_ERROR 008 14 00:00:00.16 00:00:00.14 00:00:00.15 7 PROFILE_PKG.PROFILE. GET_UNIT_CLOCK 002 156 00:00:00.34 00:00:00.03 00:00:00.10 3 PROFILE_PKG.PROFILE. NEW_REPORT_LINE 009 28 00:00:00.17 00:00:00.04 00:00:00.07 14 PROFILE_PKG.PROFILE. PERCENT_OF_TOTAL 010 14 00:00:00.04 00:00:00.03 00:00:00.03 20 PROFILE_PKG.PROFILE. PRINT_CALL_SUMMARY_REPORT 015 1 00:00:01.82 00:00:01.82 00:00:01.82 1 PROFILE_PKG.PROFILE. PRINT_CUM_EXECUTION_TIMES 014 1 00:00:04.53 00:00:04.53 00:00:04.53 0 PROFILE_PKG.PROFILE. PRINT_NET_EXECUTION_TIMES 013 1 00:00:04.61 00:00:04.61 00:00:04.61 0 PROFILE_PKG.PROFILE. PRINT_TOTAL_EXECUTION_TIMES 011 1 00:00:06.29 00:00:06.29 00:00:06.29 0 PROFILE_PKG.PROFILE. PRINT_UNIT_STATS 012 14 00:00:00.48 00:00:00.32 00:00:00.37 3 PROFILE_PKG.PROFILE. READ_LOG_FILE 004 1 00:02:20.53 00:02:20.53 00:02:20.53 0 PROFILE_PKG.PROFILE.SORT 005 2 00:00:04.87 00:00:03.80 00:00:04.33 0 PROFILE_PKG.PROFILE.SORT. ORDER 006 38 00:00:00.20 00:00:00.14 00:00:00.16 6 PROFILE_PKG.PROFILE. UPDATE_CURRENT_CLOCK 003 78 00:00:00.96 00:00:00.09 00:00:00.32 2 Profile 1.0 MONDAY 07/22/85 22:17:50 Page: 5 CALL SUMMARY REPORT ------------------------------------------------------------------------------- UNIT NAME PU# T EXECS ABEND MAX RL SONS GRANDSONS ----------------------------- --- - ------- ------- ------- --------- --------- CLOCKS.CREATE_CLOCKS 002 P 1 0 1 0 0 CLOCKS.DESTROY_CLOCKS 003 P 1 0 1 0 0 CLOCKS.MORE_TASKS 013 F 1 0 1 0 0 CLOCKS.MORE_UNITS 012 F 232 0 1 0 0 CLOCKS.PAUSE_UNIT 007 P 148 0 1 0 0 CLOCKS.PREVIOUS_UNIT 011 F 77 0 1 0 0 CLOCKS.RESTART_UNIT 005 P 148 0 1 0 0 CLOCKS.START_UNIT 004 P 78 0 1 0 0 CLOCKS.STOP_UNIT 006 P 78 0 1 3 0 CLOCKS.TASK_OF 001 F 3 0 1 0 0 PROFILE_PKG.PROFILE 001 P 1 0 1 8 1109 PROFILE_PKG.PROFILE. CALCULATE_EXPECTED_ERROR 007 P 1 0 1 14 0 PROFILE_PKG.PROFILE. CALCULATE_EXPECTED_ERROR. PERCENT_OF_ERROR 008 F 14 0 1 0 0 PROFILE_PKG.PROFILE. GET_UNIT_CLOCK 002 P 156 0 1 0 0 PROFILE_PKG.PROFILE. NEW_REPORT_LINE 009 F 28 0 1 0 0 PROFILE_PKG.PROFILE. PERCENT_OF_TOTAL 010 F 14 0 1 0 0 PROFILE_PKG.PROFILE. PRINT_CALL_SUMMARY_REPORT 015 P 1 0 1 7 0 PROFILE_PKG.PROFILE. PRINT_CUM_EXECUTION_TIMES 014 P 1 0 1 7 7 PROFILE_PKG.PROFILE. PRINT_NET_EXECUTION_TIMES 013 P 1 0 1 7 7 PROFILE_PKG.PROFILE. PRINT_TOTAL_EXECUTION_TIMES 011 P 1 0 1 21 0 PROFILE_PKG.PROFILE. PRINT_UNIT_STATS 012 P 14 0 1 14 0 PROFILE_PKG.PROFILE. READ_LOG_FILE 004 P 1 0 1 843 158 PROFILE_PKG.PROFILE.SORT 005 P 2 0 1 38 0 PROFILE_PKG.PROFILE.SORT. ORDER 006 F 38 0 1 0 0 PROFILE_PKG.PROFILE. UPDATE_CURRENT_CLOCK 003 P 78 0 1 155 0 --:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: --smart.ug --:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: Ada* Self Metric Analysis and Reporting Tool ( SMART ) Version 1.0 User's Guide by Alexis Wei Intermetrics, Inc. * Ada is a trademark of the U.S. Department of Defense (AJPO) SMART User's Guide Page: i PREFACE This program was developed by Intermetrics, Inc. under Contract Number N66001-85-C-0032 to: Naval Ocean Systems Center 271 Catalina Boulevard San Diego, California 92152. SMART User's Guide Page: ii CONTENTS 1.0 DESCRIPTION ....................................... 1 2.0 COMMAND FORMAT .................................... 2 2.1 Source Program Instrumentation .................. 2 2.2 Compilation and Linking ......................... 5 2.3 Target Program Execution ........................ 6 2.4 Report Generation ............................... 7 3.0 OUTPUT FORMAT ..................................... 10 3.1 Test Configuration Report ....................... 10 3.2 Variable Trace Report ........................... 11 3.3 Loop Summary Report ............................. 12 3.4 Variable Statistic Report ....................... 13 4.0 USING SMART ....................................... 14 4.1 Overhead ........................................ 14 4.2 Tracing Variables ............................... 14 4.3 Report Generation ............................... 14 4.4 Maximum, Minimum, and Average Values ............ 15 4.5 Reporting of Values of Real Types ............... 15 4.6 Guidlines for Source Instrumentation ............ 15 5.0 RESTRICTIONS AND LIMITATIONS ...................... 16 APPENDIX A - SAMPLE REPORT FILE ........................ A-1 APPENDIX B - GLOSSARY .................................. B-1 SMART User's Guide Page: 1 1.0 DESCRIPTION The Ada Self Metric Analysis and Reporting Tool, SMART, is one of an integrated set of five tools that provide testing and analysis aids for the development of computer programs implemented in the Ada programming language. Specifically, SMART provides the ability to: O Trace the value of specified variables at entry to and exit from each program unit or at each "breakpoint", including decision points. O Determine the number of times loops execute. O Determine the maximum, minimum, and average values of specified variables. SMART consists of three elements: 1) A source instrumenter that parses the target Ada program and inserts "hooks" that transfer control to a runtime kernal upon entry to and exit from each instrumented Ada program unit and, optionally, at each breakpoint. Each Ada program unit for which dynamic run time information is to be recorded must be instrumented prior to compilation. 2) A runtime execution monitor, or kernel, that gains control upon execution of each instrumented breakpoint and logs dynamic information about the program's execution. The runtime execution monitor must be linked with the instrumented Ada program units. 3) A report generator that analyzes the recorded information and produces a comprehensive report about the program's execution in an easily understandable format. SMART is a software development tool. Its purpose is to aid the programmer or analyst during program development. SMART User's Guide Page: 2 2.0 COMMAND FORMAT Operation of the Self Metric Analysis and Reporting Tool consists of four steps: 1) source program instrumentation 2) compilation and linking 3) target program execution 4) report generation Each of these steps is described in detail in the following paragraphs. 2.1 Source Program Instrumentation Each Ada compilation unit containing program units for which dynamic runtime information is to be recorded must be separately instrumented by the source instrumenter. The target Ada program may be instrumented to log the value of each user specified variable at entry to and exit from each specified program unit. Optionally, the user may also record the value of each specified variable at each breakpoint, including decision points. The Source Instrumenter is one of five tools included in the Ada Test and Analysis System shell, TEST_TOOLS and is executed as follows: (User input and responses are shown in Bold Face) RUN TEST_TOOLS <cr> The Test_Tools prompt will then be displayed and the user will be asked to enter the name of the tool he wishes to run. To execute the Source Instrumenter the user should respond with the tool name, Source_Instrument. EXAMPLE Welcome to the Test Tools please enter desired tool enter ? for a list of tools available SOURCE_INSTRUMENT <cr> SMART User's Guide Page: 3 If "?" is entered then Test_Tools will display a list of the tools available for execution and the user will be reprompted to enter a tool name. If "Q" is entered then execution of the Test_Tools shell will be terminated. EXAMPLE Welcome to the Test Tools please enter desired tool enter ? for a list of tools available ? <cr> The following tools may be executed path -- the report generator for the path analysis tool profile -- the report generator for the timing tool smart -- the report generator for the self metric tool autopath -- the report generator for the automatic path tool source_instrument -- instrument an Ada source file please enter desired tool It is only necesary to enter the first two characters of the tool name, i.e., PA, PR, SM, AP or SO. If an incorrect tool name is entered the user will be prompted to reenter the tool name. When a valid tool name has been entered the user will be prompted to enter file names for the Ada source file to be instrumented, the listing file to be generated, and the instrumented source file to be generated. EXAMPLE Enter Source_File => Test5.ADA <cr> Enter Source_Listing_File => <cr> Enter Instrumented_Source_File => <cr> source_file = TEST5.ADA source_listing = TEST5.LST instrumented_source = TEST.INS If the user does not enter a Source_Listing_File or an Instrumented_Source_File then default file names will be derived from the first qualifier of Source_File_Name and the extentions ".LST" and ".INS", respectively. SMART User's Guide Page: 4 When all file names have been entered the Source Instrumenter will display a menu of instrumenting options and query the user to select the options to be used when instrumenting the specified source file. It is recommended that the predefined instrumenting defaults for the Self Metric tool, option 3, be selected. EXAMPLE Source Instrumenter Version 1.0 Instrumenting Options are: 1 - Path/Autopath Analyzer Defaults 2 - Performance Analyzer Defaults 3 - Self Metric Defaults 4 - User Supplied Options Enter option (1, 2, 3, 4, ?, or <cr> for default of 1): 3 <cr> When instrumenting is completed the Source Instrumenter will return control to the TEST_TOOLS shell. At this point the user may instrument another Ada source file or terminate the shell by entering "Q" or "QUIT". EXAMPLE please enter desired tool Q <cr> $ The Source Instrumenter also provides a capability to explicitly select specific program units and variables within a compilation unit for instrumentation. Refer to the Source Instrumenter User's Guide for more detailed information on using this option. Some guidelines for determining the source instrumentation options that should be used with SMART are presented in section 4.0. SMART User's Guide Page: 5 2.2 Compilation and Linking When the instrumentation process is completed the instrumented target Ada program must be compiled and linked to prepare it for execution. Each instrumented Ada compilation unit must be separately compiled. When the compilation process is completed the target program must be linked with the Run Time Monitor. The user should follow the normal procedures for compiling and linking with the Ada compiler system being used for program development. These procedures may vary from system to system. The procedures for compiling and linking the instrumented on a DEC VAX under VMS Version 4.0 and using the DEC VAX Ada Compiler System Version 1.0 are shown below. However this example assumes that the user has already performed all operations necessary to set up an Ada program library. EXAMPLE $ Ada Test5.INS <cr> $ ACS Link Test5 <cr> %ACS-I-CL_LINKING, Invoking the VAX/VMS Linker $ 2.3 Target Program Execution When linking is completed, the target Ada program may be executed. The user interface and method of executing the target program is not altered by the instrumentation process. Therefore, the user should follow the normal procedures for executing the target program on the host computer system being used for program development. When the first unit of the target program is entered, the Run Time Monitor will prompt the user for the following information: 1. Tool_Name : The user should enter SMART_TOOL (upper or lower case is acceptable). If the user should enter an invalid tool name, he will be repeatedly prompted until a correct tool name is entered or the program is aborted. SMART User's Guide Page: 6 2. Logfile_Name : The user should enter a legal filename for the computer he is using. The default filename may be chosen by entering a carriage return. The default Logfile_Name is SMART.LOG. If the File already exists, the user is asked if he would like to Enter a new filename, Overwrite the existing file, or Append to the existing file. 3. Test_Identification: The user is asked to enter a unique test identification which can be any valid Ada string. The default Test ID may be selected by entering a carriage return. The default Test ID for SMART is "Self Metric Analysis Report". The Test ID is recorded in the log file generated by the Run Time Monitor and printed on the Test Configuration Report generated by the Smart Report Generator. A sample session with the Run Time Monitor is presented below. EXAMPLE SESSION Following is a sample session with the Run Time Monitor. (User responses are shown in Bold Face): $ RUN TEST5 <cr> Enter Tool Type : PATH_TOOL, AUTOPATH_TOOL, SMART_TOOL, or PROFILE_TOOL -----> SMART_TOOL <cr> Enter Logfile Name, Null for Default -----> Test5.log <cr> If the Logfile already exists, the following message and prompt are printed: File already exists!!! Do you wish to: E = Enter a New Filename O = Overwrite existing file A = Append to the existing file Enter Option ===> O <cr> The Run Time Monitor will repeatedly prompt the user until a valid option is entered. Enter a unique Test Identification, Null for default. Test Case #5 <cr> When the Run Time Monitor input is complete, the target program will resume its normal execution. SMART User's Guide Page: 7 2.4 Report Generation During execution, the instrumented Ada program will create a log file containing dynamic run time information. The SMART Report Generator analyzes the data and presents it in an easily understood format. The SMART report generator is one of five tools included in the Ada Test and Analysis System shell, TEST_TOOLS. The report generator is executed as follows: (User input and responses are shown in Bold Face) RUN TEST_TOOLS <cr> The Test_Tools prompt will then be displayed and the user will be asked to enter the name of the tool he wishes to run. To execute Profile, the user should respond with the tool name, SMART. EXAMPLE Welcome to the Test Tools please enter desired tool enter ? for a list of tools available SMART <cr> It is only necessary to enter the first two characters of the tool name, i.e., "SM". If an incorrect tool name is entered the user will be prompted to reenter the tool name. When a valid tool name has been entered the user will be prompted to enter the name of the log file to be processed and the name of the report file to be generated. EXAMPLE Enter Log_File_Name => TEST5.LOG <cr> Enter Report_File_Name => TEST5.RPT <cr> If the user does not enter a Report_File_Name, then it will be derived from the Log_File_Name and ".RPT" will be appended. When valid log file and report file names have been entered, SMART will display the test configuration data that was recorded in the log file at the time the program under test was executed. The user is then prompted as to whether or not he wishes to continue. SMART User's Guide Page: 8 After examining the test configuration data, the user can discontinue processing by entering "N" or "n". If the correct log file has been selected then the user should enter "Y" or "y" to continue processing. EXAMPLE SMART 1.0 Self Metric Analysis and Reporting Tool Program Under Test: Test5 Test Date: 06/23/85 Test Time: 20:24:59 Test ID: Test Case #234 Do you wish to continue (Y/N)? Y <cr> When processing is completed the report file will be ready for analysis by the user. At this point control will be returned to the Test_Tools shell where the user may select another tool for execution or terminate the shell by entering "Q" or Quit. An example session with the SMART Report Generator is presented on the following page. SMART User's Guide Page: 9 EXAMPLE SESSION Following is a sample session with the Report Generator (User input and responses are shown in Bold Face): $ RUN TEST_TOOLS <cr> Welcome to the Test Tools please enter desired tool enter ? for a list of tools available ? <cr> The following tools may be executed path -- the report generator for the path analysis tool profile -- the report generator for the timing tool smart -- the report generator for the self metric tool autopath -- the report generator for the automatic path tool source_instrument -- instrument an Ada source file please enter desired tool SM <cr> Enter Log_File_Name => TEST5.LOG <cr> Enter Report_File_Name => <cr> log_file_name = TEST5.LOG report_file_name = TEST5.RPT SMART 1.0 Self Metric Analysis and Reporting Tool Program Under Test: TEST5 Test Date: 06/23/85 Test Time: 20:24:59 Test ID: Test Case #234 Do you wish to continue (Y/N)? Y <cr> please enter desired tool quit <cr> $ SMART User's Guide Page: 10 3.0 OUTPUT FORMAT The report file generated by SMART contains four reports: a Test Configuration Report, a Variable Trace Report, a Loop Summary Report, and a Variable Statistic Report. Each report provides the user with different information about the program being tested and is generated in 132 column format. A sample report file is included in Appendix A. Prior to using the SMART report generator it is important for the user to understand the information that is being presented and how to interpret it. Each of the reports is described below. A more general discussion on evaluating information presented in these reports is included in section 4.0. 3.1 Test Configuration Report The Test Configuration Report includes information necessary to correlate the report with a specific test or execution of the target Ada program. It includes the following information: 1) Program Under Test: This is the name of the main program unit of the program being analyzed. 2) Test Date: The calendar date that the program under test was executed and the execution log file was created. The test date is presented in the form MM/DD/YY. 3) Test Day: The day of the week, Sunday through Saturday, that the program under test was executed and the execution log file was created. 4) Test Time: The time of day that the program under test was executed and the execution log file was created. The test time is presented in the form HH:MM:SS. 5) Log File: The name of the log file that was used in the preparation of the execution timing reports. 6) Test ID: The test ID specified by the user at the time the program under test was executed. SMART User's Guide Page: 11 3.2 Variable Trace Report This report traces the values of user specified variables at each breakpoint. A breakpoint may be at entry to or exit from each instrumented Ada program unit or, optionally, at each executeable statement. The following types of variables may be traced: Integer Float Fixed Point String Character Enumerated (including Boolean) Tracing of Task Types, Access Types, Private Types, or implicit tracing of arrays and records is not allowed. However, individual array elements or record components, if of one of the above types, may be traced explicitly. Tracing of variables whose types are generic or variables that are declared in a generic unit is not allowed. REPORT COLUMNS The Variable Trace Report presents the following information about each variable being traced that is visible within the scope of of the breakpoint. 1) COMP UNIT NAME ( Compilation Unit Name ) This is the name of the compilation unit containing the breakpoint. It may or may not correspond to to the compilation unit in which the variable being traced is declared. 2) BKPT# ( Breakpoint Number ) This is the number of the breakpoint that was executed. Breakpoint numbers are assigned by the Ada Source Instrumenter and are identified in the listing file generated by the Source Instrumenter. An asterisk "*" in this column indicates a continuation line as described below. 3) VARIABLE NAME This is the fully qualified name of the Ada variable being traced. This field is limited to 50 characters. If the fully qualified name is longer than 50 characters then excess characters are continued on following lines until the entire name has been printed. Continuation lines contain an asterisk "*" in the BKPT# column. SMART User's Guide Page: 12 4) VALUE This is the value of the variable being traced prior to execution of the specified breakpoint. Values are left justified. Trailing blanks in string variables are indicated by the notation <<N blanks>> where N is the number of trailing blanks. This field is limited to 50 characters. If the value of the variable is longer than 50 characters then excess characters are continued on following lines until the entire value has been printed. Continuation lines contain an asterisk "*" in the BKPT# column. This field is also used to display warning messages such as "Variables of type TASK cannot be traced." 3.3 Loop Summary Report The Loop Summary Report identifies the number of times instrumented loops execute. Loops that are not executed are not reported. If no instrumented loops are executed then this report will not be printed. REPORT COLUMNS The Loop Summary Report presents the following information about each loop breakpoint executed. 1) LOOP COUNT This is the number of times the instrumented loop breakpoint was executed. 2) BKPT# ( Breakpoint number ) This is the number of the loop breakpoint. Breakpoint numbers are assigned by the Ada Source Instrumenter and are identified in the listing file generated by the Source Instrumenter. An asterisk "*" in this column indicates a continuation line as described below. 3) UNIT# ( Program Unit Number ) This is the program unit number assigned by the Ada Source Instrumenter. Program unit numbers are identified in the listing file generated by the Source Instrumenter. If the program unit name is overloaded within the same compilation unit and at the same scoping level then the program unit number may be used to uniquely identify the program unit. SMART User's Guide Page: 13 4) UNIT NAME This is the fully qualified name of the Ada program unit containing the loop breakpoint. This field is limited to 105 characters. If the fully qualified unit name is longer than 105 characters then excess characters are continued on following lines until the entire name has been printed. Continuation lines contain an asterisk "*" in the BKPT# column. 3.4 Variable Statistic Report The Variable Statistic Report reports the maximum, minimum, and average values of all numeric variables that are recorded during execution of the target Ada program. If no numeric variables are recorded then this report will not be printed. REPORT COLUMNS The Variable Statistic Report presents the following information about each loop breakpoint executed. 1) UNIT# ( Program Unit Number ) This is the program unit number assigned by the Ada Source Instrumenter and is identified in the listing file generated by the Source Instrumenter. If the program unit name and variable name are overloaded within the same compilation unit and at the same scoping level then the program unit number may be used to uniquely identify the variable. 2) VARIABLE NAME This is the fully qualified name of the Ada variable being traced. This field is limited to 58 characters. If the fully qualified name is longer than 58 characters then excess characters are continued on following lines until the entire name has been printed. 3) MAX VALUE ( Maximum Value ) This is the maximum value reported for the variable being traced. 4) MIN VALUE ( Minimum Value ) This is the minimum value reported for the variable being traced. 5) AVERAGE VALUE This is the average of all values reported for the variable being traced. SMART User's Guide Page: 14 4.0 USING SMART Although SMART is simple to operate, there are a number of things the user should be aware of before attempting to use the tool. A user who has a basic understanding the processing performed by SMART will be able to use the tool much more effectively as a software development, debugging, and testing aid. 4.1 Overhead Smart imposes a great deal of overhead on the execution of the target Ada program. The least amount of overhead is imposed when the target program is instrumented at entry to and exit from each program unit and at each decision point rather than at every breakpoint. 4.2 Tracing Variables The more variables that are traced the more overhead will be incurred at each breakpoint and the longer the execution time of the target program. Tracing of uninitialized program variables can cause some unpredictable results and should be avoided if possible. For numeric types the max, min, and average values will be distorted as discussed below. Unitialized strings generally are of no consequence unless they contain certain control characters, line feeds, or carriage returns that may cause the log file be unreadable by the report generator. Uninitialized discrete types may, in some cases, result in an exception during execution of the target Ada program. 4.3 Report Generation A tremendous amount of work has to be performed by the SMART report generator for each breakpoint that is executed. The more variables that are being traced the more work that has to be done at each breakpoint. Therefore, the amount of time required to process a logfile and generate the various reports is a function of the number of variables being traced and the number of instrumented breakpoints that are executed by the target Ada program. SMART User's Guide Page: 15 4.4 Maximum, Minimum, and Average Values MAX, MIN and AVERAGE are the maximum, minimum, and AVERAGE values that are recorded during execution of the target Ada program. They may or may not be the actual maximum, minimum, and average values that the variable held during execution. For example, the variable may be visible to, and changed by, a program unit that is executed but not instrumented. MAX, MIN, AND AVERAGE are also subject to severe distortion for unitialized variables. Smart has no way of detecting that a value recorded for a variable is random garbage. Therefore, tracing of uninitialized variables should be avoided 4.5 Reporting of Values of Real Types All values of real types are handled internally by the Run Time Monitor and the SMART Report Generator as type Float. Real values are presented in the Variable Trace Report and the Variable Statistic Report with the following attributes: Fore = 1, Aft = 13, Exponent = 3. 4.5 Guidelines for Source Instrumentation 1. Trace as few variables as possible. 2. Instrument as few program units as possible. 3. When possible, instrument program units at entry and exit only rather than at each statement. 4. When possible, avoid tracing "global" variables. They result in large amounts of data. 5. If possible, avoid tracing variables in program units that that are executed a large number of times. 6. Avoid statement level traces of variables in program units with loops that are executed a large number of times. SMART User's Guide Page: 16 5.0 RESTRICTIONS AND LIMITATIONS 1) No specific limits are imposed either by the Runtime Monitor or the SMART report generator on either the size of the target Ada program or the number of program units that can be instrumented. All arrays and stacks maintained by SMART that are a function of the total number of program units or the number of variables being traced are allocated dynamically. 2) Dynamic execution data may only be reported for program units that have been instrumented by the source instrumenter. 3) Regardless of whether or not the Ada compiler system being used for program development provides an interface to programming languages other than Ada, only Ada program units may be instrumented. 4) The Run Time Monitor imposes a certain amount of overhead on the execution time of each instrumented Ada program unit. Therefore, an instrumented program will execute slower than the same program uninstrumented. In a tasking environment if the execution of some program units is time dependent on the occurrence of certain events, then the additional overhead imposed by the Run Time Monitor may alter the normal flow of control of the program. 5) When statement level breakpoints are specified, all variables being traced that are visible within the scope of the executing breakpoint are recorded in the execution log file. This, depending on the number of variables traced, the total number of breakpoints executed, and the logic of the instrumented program, may result in a very large log file. Ada Self Metric Analysis and Reporting Tool Page: A - 1 APPENDIX A SAMPLE REPORTS Ada Self Metric Analysis and Reporting Tool Page: A - 2 Sample Test Configuration Report ------------------------------------------------------------ Ada Self Metric Analysis and Reporting Tool Page: A - 3 Sample Variable Trace Report ------------------------------------------------------------ Ada Self Metric Analysis and Reporting Tool Page: A - 4 Sample Loop Summary Report ------------------------------------------------------------ Ada Self Metric Analysis and Reporting Tool Page: A - 5 Sample Variable Statistic Report ------------------------------------------------------------ Ada Self Metric Analysis and Reporting Tool Page: B - 1 GLOSSARY Breakpoint A breakpoint is a transfer of control to the Run Time Monitor. Each breakpoint within a compilation unit is assigned a unique breakpoint number by the Source Instrumenter. In general, breakpoints correspond to executeable statements. However, not all executeable statements are assigned breakpoints. Execution Log File A sequential file created by the Run Time Monitor during execution of an Ada program that has been instrumented by the Source Instrumenter. The terms "Execution Log File" and "Log File" are interchangeable. Log File A sequential file created by the Run Time Monitor during execution of an Ada program that has been instrumented by the Source Instrumenter. The terms "Execution Log File" and "Log File" are interchangeable. Overhead The term "overhead" refers to the additional execution time imposed on the program under test by the Run Time Monitor. Program Unit Ada programs may contain four different types of program units: procedures, functions, tasks and generics. However, the Ada language also provides the ability to include executeable code within a package body that is not included within any of the program units contained in that package. This code is executed during package initialization when the package is elaborated. SMART reports dynamic information for this "initialization" code the same as for Ada program units. Therefore, for reporting purposes, initialization code within a package body is also treated as a program unit. Report Time The time that the SMART Report Generator was executed. Ada Self Metric Analysis and Reporting Tool Page: B - 2 Run Time Monitor A runtime kernel that gains control upon entry to and exit from each instrumented Ada program unit and at each user specified breakpoint. The Run Time Monitor logs information about the program's execution to an execution log file for later processing by one or more of the Test_Tools report generator. The runtime execution monitor must be linked with the instrumented Ada program units. Source Instrumenter An Ada program that parses the target Ada program and inserts "hooks" that transfer control to the Run Time Monitor upon entry to and exit from each instrumented Ada program unit and at each user specified breakpoint. Each Ada program unit for which dynamic information is to be recorded must be instrumented by the Source Instrumenter prior to compilation. Test ID A unique test identification specified by the user when the program under test is executed. The Test ID is stored in the log file by the Run Time Monitor and included in the Test Configuration Report by the SMART report generator. Test Time The date and time that the program under test began execution. --:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: --si.ug --:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: Source Instrumenter Version 1.0 User's Guide by Ron Rathgeber Intermetrics, Inc. PREFACE This program was developed by Intermetrics, Inc. under Contract Number N66001-85-C-0032 to: Naval Ocean Systems Center 271 Catalina Boulevard San Diego, California 92152. Source Instrumenter User's Guide Page: ii CONTENTS 1.0 DESCRIPTION ....................................... 1 2.0 COMMAND FORMAT .................................... 2 3.0 INSTRUMENTING OPTIONS ............................. 5 3.1 Statement Trace Level ... ....................... 5 3.2 Variable Tracing .... ........................... 7 4.0 RESTRICTIONS AND LIMITATIONS ...................... 9 APPENDIX A - SAMPLE LISTING FILE ....................... A-1 APPENDIX B - GLOSSARY .................................. B-1 Source Instrumenter User's Guide Page: 1 1.0 DESCRIPTION The Source Instrumenter is one of an integrated set of five tools that provide testing and analysis aids for the development of computer programs implemented in the Ada* programming language. The Source Instrumenter instruments the user's source to add calls to a Run-Time Monitor. These calls provide information about the program as it runs. The Source Instrumenter adds calls to: O Inform the Run_Time Monitor of the names and types of units in the program. O Inform the Run_Time Monitor when a each unit is entered and exited. O Inform the Run-Time Monitor when a statement in the source program has executed. Once a program has been instrumeted, it is compiled and linked through the user's compiler system. When the program is run, the Run-Time Monitor will gather the information about the source program into a log file. This log file can then be processed by one of the report writers which provide a report about the execution of the program. The Source Instrumeter provides for both tracing statement execution and for tracing variables. Breakpoints are placed in the user's source to provide statement tracing information. The Source Instrumenter can place breakpoints at Entry/Exit only, Entry/Exit and Decision Point, or at every statement. When variable tracing is being done, the value of each variable being traced is written to the log file. * Ada is a registered trademark of the U.S. Government(AJPO) Source Instrumenter User's Guide Page: 2 2.0 COMMAND FORMAT The source instrumenter is invoked by running the Test_Tools shell. This shell allows the user to run the Source Instrumenter and the report writers. Compiling, Linking, and Running of the user's program will not be performed by the shell. Once the Test_Tools program has been invoked (on the VAX/VMS the command is "RUN TEST_TOOLS", but on each system this may be different), the following messages will appear: Welcome to the Test Tools please enter desired tool enter ? for a list of tools available enter quit to exit program At this point the user can enter a question mark to get a list of the tools that are available, quit to exit the program, or invoke one of the tools. To invoke one of the tools the user can either enter a procedure call to the tool using standard Ada format including parameters, or he may just enter the tool name and be prompted for each of the input parameters. The procedure specification for the Source Instrumenter is as follows: procedure SOURCE_INSTRUMENT(INPUT_FILE : in FILE_NAME; SOURCE_LISTING_FILE : in FILE_NAME; INSTRUMENTED_FILE_SOURCE: in FILE_NAME); where INPUT_FILE is the user's source. SOURCE_LISTING_FILE is a pretty printed version of the user's source with breakpoints listed. INSTRUMENTED_SOURCE_FILE is the file containing the instrumented source. FILE_NAME is a string. Examples of invoking the Source Instrumenter in both Ada format with parameters and without parameters is shown in Figure 1. In the second example of Figure 1 the user is prompted for the inputs. The user inputs are shown on the lines that begin with enter. The source_file must always be entered but the listing file and Instrumented file will default to the source file name appended by ".lst" and ".ins" respectively if they are left blank when the user is prompted for their names. Source Instrumenter User's Guide Page: 3 Note also that when the Source Instrumenter is invoked in Ada format, named association of parameters may be used as well as positional parameters. Standard Ada rules for postional and named association of parameters are used. Source Instrumenter User's Guide Page: 4 Figure 1 - Example of Source Instrumenter Invocation Welcome to the Test Tools please enter desired tool enter ? for a list of tools available enter quit to exit source_instrument(count.ada, count.lis, count.si) source_file = count.ada source_listing = count.lis instrumented_source = count.si or please enter desired tool source_instrument Enter Source_File => count.ada Enter Source_Listing_File => count.lis Enter Instrumented_Source_File => count.si source_file = count.ada source_listing = count.lis instrumented_source = count.si Source Instrumenter User's Guide Page: 5 3.0 INSTRUMENTING OPTIONS Once the Source Instrumenter has been entered it will display a menu of instrumenting options. This menu allows the user to select how the source should be instrumented. From the menu the user can select the defaults for Path/Autopath Analyzer, Profile, or Smart. The user may also select to provide his own options. The options menu appears as follows: Source Instrumenter Version 1.0 Instrumenting Options are: 1 - Path/Autopath Analyzer Defaults 2 - Performance Analyzer Defaults 3 - Self Metric Defaults 4 - User Supplied Options Enter option (1, 2, 3, 4, ?, or <cr> for default of 1): Normally, the user will just select the default options for the tool he plans to use; however, by providing the user with the ability to select his own options, the user can tailor the instrumenting process to meet his needs. There are two options that the user must specify if he decides to provide his own options. These two options are discussed in the next two sections. 3.1 Statement Trace Level The first option that must be entered is the statement trace level. If the user selects option 4 on the primary options menu (User supplied options), then the following menu will be displayed: Available trace levels are: 1 - Entry/Exit only 2 - Entry/Exit and Decision Point 3 - Every statement 4 - Mixed (Each program unit has its own trace level) Enter option ( default is 2): If Entry/Exit only is selected, then breakpoints (and thus statement information) will be provided only at entrance to and exit from each unit. Note that there will be a breakpoint at each exit point. This means that each exception handler and return statement will also have a breakpoint. Also note that if Source Instrumenter User's Guide Page: 6 the user supplies no exception handler, then one will be added that contains an exiting unit breakpoint. The added exception handler will then re-raise the exception so that it will not alter normal execution of the program. If Entry/Exit and Decision Point is selected then Breakpoints will also be placed at each decision point. Decision Points are defined as: before each "if", "case", "while", and "for"; the first statement of each "then", and "else" branch; the first statement of each case statement branch; and before each "exit" statement. If Every Statement is selected then a breakpoint will be placed before each statement in the user's program. When any of the above three options is selected it applies to the whole file being instrumented. If the user wishes to instrument some units in the same file differently, then option 4 (Mixed) should be selected. When the mixed option is selected the user will be prompted for a trace level for each unit in the file. An example of this is given below: Available trace levels are: 1 - Entry/Exit only 2 - Entry/Exit and Decision Point 3 - Every statement 4 - Mixed (Each program unit has its own trace level) Enter option ( default is 2): 4 Do you want to do type tracing (default is no): no Current Unit Being Instrumented is: procedure count Available trace levels are: 1 - Entry/Exit only 2 - Entry/Exit and Decision Point 3 - Every statement Enter option ( default is 2): 1 Current Unit Being Instrumented is: procedure CHECK_IF_COMMENT_ON_LINE( I: in INTEGER) Available trace levels are: 1 - Entry/Exit only 2 - Entry/Exit and Decision Point 3 - Every statement Enter option ( default is 2): 3 Source Instrumenter User's Guide Page: 7 This file contains two procedures: COUNT, and CHECK_IF_COMMENT_ON_LINE. The user has selected mixed mode and has decided to trace COUNT at Entry/Exit only, and to trace CHECK_IF_COMMENT_ON_LINE at every statement. 3.2 Variable Tracing Variable tracing is done when the SMART tool option has been selected or when the user selects user supplied options and answers yes to the question of whether type tracing should be done. When variable tracing is being done the user will be prompted to specify which variables to trace. As each unit is instrumented, the user will be prompted for which variables in that unit to trace. The user may input the variables in that unit that he wants traced, or he can enter "*all" and all variables in that unit will be traced (including any variables in nested blocks). Variable names may include selected components or array indices. The user should include scope names only for variables declared in blocks (only include the block name). All other variables should be specified by using their simple name. An example of a unit being instrumented with type tracing is given below: Source Instrumenter Version 1.0 Instrumenting Options are: 1 - Path/Autopath Analyzer Defaults 2 - Performance Analyzer Defaults 3 - Self Metric Defaults 4 - User Supplied Options Enter option (1, 2, 3, 4, ?, or <cr> for default of 1): 3 Current Unit Being Instrumented is: procedure count Enter variables to trace. Enter one variable per line or *ALL to trace all variables in the scope. Terminate the list with a blank line >> Number_of_Comments >> last >> Current Unit Being Instrumented is: procedure CHECK_IF_COMMENT_ON_LINE( I: in INTEGER) Enter variables to trace. Enter one variable per line or *ALL to trace all variables in the scope. Terminate the list with a blank line >> *all >> Source Instrumenter User's Guide Page: 8 In this example, the unit is being instrumented for the SMART tool, so variable tracing is being done. There are two units: the main procedure COUNT, and a nested procedure. In the example, only the variables NUMBER_OF_COMMENTS and LAST are being traced in procedure COUNT, but all variable are being traced in the nested procedure. Values for the following types of variables can be displayed: enumerated types (predefined and user defined), all numeric types, strings, and subtypes and derived types of the above. All variables being traced will have their values written to the log file at each breakpoint where they are visible. Variables contained in a named block can be traced by including the name of the variable qualified with the block name when the instrumenter asks for variables to trace in the enclosing unit. Thus, if variable TEMP is declared in a block labeled BLOCK1, then the variable can be traced by specifing BLOCK1.TEMP when the instrumenter asks which variables should be traced for the enclosing scope. Variables in un-labeled blocks cannot be specified, since there is no way to specify their name; however, they can be traced by specifing that all variables in the enclosing unit should be traced. In order to trace variables declared in a package specification or variables whose type is defined in a package specification, the package specification must be instrumented with type tracing on. When a package specification is instrumented, it will not be altered; however several files will be created in the current working directory that contain the information needed from the package specification. The user will be prompted as to whether the package specification should be included in the instrumented source file. If the user does not want the package specification to be recompiled then he should reply no. When a unit is instrumented that "withs" a package, the tracing information for the package (contained in files created by the souce instrumenter) will be used to construct tracing routines for the variables and types contained in the package specification. If these files do not exist, then the instrumenter assumes that no variables or types from the package are being traced. Source Instrumenter User's Guide Page: 9 4.0 RESTRICTIONS AND LIMITATIONS 1) Values of record types, array types, access types, private types (outside of the package body), and task types will result in the following message being written to the log file: "Values of type <type_mark> cannot be displayed". 2) The following types of variables cannot be traced: variables of generic types, named numbers, and variables whose types are declared in a package specification that was not instrumented with type tracing on. Attempting to trace these variables will result in a compile time error. 3) A compile time error will also occur when invalid selected components or indexed components are specified. 4) Loop variables cannot be traced. Attempts to trace loop variables will result in a message "Variable not found". 5) Variables in un-named blocks can only be traced by specifing that all variables in the enclosing unit be traced (*all). Source Instrumenter Page: A - 1 APPENDIX A SAMPLE LISTING Source Instrumenter Page: A - 2 Source Instrumenter Output on 07/27/85 at 15:20:04 1 Source File: count.ada Bkpt Number Source Text ------ ----------- with TEXT_IO; use TEXT_IO; procedure COUNT is MAX_LINE : INTEGER := 255; CURRENT_LINE : STRING(1 .. MAX_LINE); LAST_CHAR : NATURAL; NUMBER_OF_COMMENTS : INTEGER := 0; NUMBER_OF_CODE : INTEGER := 0; NUMBER_OF_COMMENTED_CODE : INTEGER := 0; INPUT_FILE : FILE_TYPE; FILE_NAME : STRING(1 .. MAX_LINE); LAST : NATURAL; package NUMBER_IO is new INTEGER_IO(INTEGER); use NUMBER_IO; procedure CHECK_IF_COMMENT_ON_LINE(I : in INTEGER) is begin 1 for J in I .. LAST_CHAR loop 2 if CURRENT_LINE(J) = '-' then 3 if J < LAST_CHAR and CURRENT_LINE(J + 1) = '-' then 4 NUMBER_OF_COMMENTED_CODE := NUMBER_OF_COMMENTED_CODE + 1; end if; end if; end loop; 5 end CHECK_IF_COMMENT_ON_LINE; begin 6 PUT("Type name of input file: "); GET_LINE(FILE_NAME, LAST); NEW_LINE; OPEN(INPUT_FILE, IN_FILE, FILE_NAME(1 .. LAST)); 7 while not END_OF_FILE(INPUT_FILE) loop 8 GET_LINE(INPUT_FILE, CURRENT_LINE, LAST_CHAR); 9 for I in 1 .. LAST_CHAR loop 10 if CURRENT_LINE(I) /= ' ' then 11 if CURRENT_LINE(I) = '-' and I < LAST_CHAR then 12 if CURRENT_LINE(I + 1) = '-' then 13 NUMBER_OF_COMMENTS := NUMBER_OF_COMMENTS + 1; else 14 NUMBER_OF_CODE := NUMBER_OF_CODE + 1; CHECK_IF_COMMENT_ON_LINE(I); end if; else 15 NUMBER_OF_CODE := NUMBER_OF_CODE + 1; CHECK_IF_COMMENT_ON_LINE(I); end if; 16 exit; end if; Source Instrumenter Page: A - 3 Source Instrumenter Output on 07/27/85 at 15:20:04 2 Source File: count.ada Bkpt Number Source Text ------ ----------- end loop; end loop; PUT("Number of comments is "); PUT(NUMBER_OF_COMMENTS); NEW_LINE; PUT("Number of code lines is "); PUT(NUMBER_OF_CODE); NEW_LINE; PUT("Total lines (non-blank) is "); PUT(NUMBER_OF_COMMENTS + NUMBER_OF_CODE); NEW_LINE; 17 if NUMBER_OF_CODE /= 0 then 18 PUT("Percent of code lines commented is "); PUT(100*NUMBER_OF_COMMENTED_CODE/NUMBER_OF_CODE); end if; 19 end COUNT; Source Instrumenter Page: B - 1 GLOSSARY Breakpoint A breakpoint is a transfer of control to the Run Time Monitor. Each breakpoint within a compilation unit is assigned a unique breakpoint number by the Source Instrumenter. In general, breakpoints correspond to executeable statements. However, not all executeable statements are assigned breakpoints. Execution Log File A sequential file created by the Run Time Monitor during execution of an Ada program that has been instrumented by the Source Instrumenter. The terms "Execution Log File" and "Log File" are interchangeable. Log File A sequential file created by the Run Time Monitor during execution of an Ada program that has been instrumented by the Source Instrumenter. The terms "Execution Log File" and "Log File" are interchangeable. Overhead The term "overhead" refers to the additional execution time imposed on the program under test by the Run Time Monitor. Program Unit Ada programs may contain four different types of program units: procedures, functions, tasks and generics. However, the Ada language also provides the ability to include executeable code within a package body that is not included within any of the program units contained in that package. This code is executed during package initialization when the package is elaborated. SMART reports dynamic information for this "initialization" code the same as for Ada program units. Therefore, for reporting purposes, initialization code within a package body is also treated as a program unit. Report Time The time that the SMART Report Generator was executed. Source Instrumenter Page: B - 2 Run Time Monitor A runtime kernel that gains control upon entry to and exit from each instrumented Ada program unit and at each user specified breakpoint. The Run Time Monitor logs information about the program's execution to an execution log file for later processing by one or more of the Test_Tools report generator. The runtime execution monitor must be linked with the instrumented Ada program units. Source Instrumenter An Ada program that parses the target Ada program and inserts "hooks" that transfer control to the Run Time Monitor upon entry to and exit from each instrumented Ada program unit and at each user specified breakpoint. Each Ada program unit for which dynamic information is to be recorded must be instrumented by the Source Instrumenter prior to compilation. Test ID A unique test identification specified by the user when the program under test is executed. The Test ID is stored in the log file by the Run Time Monitor and included in the Test Configuration Report by the SMART report generator. Test Time The date and time that the program under test began execution.