home *** CD-ROM | disk | FTP | other *** search
Text File | 1988-05-03 | 38.6 KB | 1,178 lines |
-
-
-
-
-
- GENERIC APSE EVALUATION QUESTIONS
-
- by Paul Dobbs
- General Dynamics
- Fort Worth, TX
-
- December, 1983
-
-
- A. APSE CONFIGURATION:
-
- 1. What tools exist in your system which are directly user-accessible?
-
- 2. What tools and packages are not directly user-accessible?
-
- 3. Briefly describe each tool.
-
- 4. What tool or combination of tools (if any) support the following
- functions?
-
- - Command input
-
- - On-line assistance
-
- - Editing
-
- - Text formatting
-
- - Configuration control
-
- - Compilation
-
- - Assembly
-
- - Source code formatting
-
- - Statistical profiling
-
- - Cross reference generation
-
- - Library management
-
- - Linking
-
- - Downloading to target systems
-
- - Requirements processing
-
- - Data dictionary management
-
- - Rapid prototyping
-
- - Design language processing
-
- - Target environment simulation
-
- - Debugging of programs
-
- - Text management
-
- - Cost estimation
-
- - Scheduling
-
- - Tracking
-
- - Auditing
-
- - Complexity measurement
-
- 5. What functions not listed above are provided, and what tools provide
- those functions?
-
- 6. Are any predefined packages other than those defined in the Ada
- standard provided for inclusion in programs developed using the
- system?
-
- B. KAPSE:
-
- 1. Do the interfaces to the KAPSE conform totally to the CAIS?
-
- 2. If the answer to question 1 is no, do they conform to a subset of
- the CAIS?
-
- 3. If so, what CAIS packages are fully implemented?
-
- 4. What CAIS packages are only partially implemented or modified?
-
- 5. In what way are the packages listed in response to question 4
- incomplete or modified?
-
- 6. What additional non-CAIS interfaces to the KAPSE exist?
-
- 7. Is the KAPSE run on the bare machine, or does it function through
- some underlying operating system?
-
- 8. If the KAPSE is built on an underlying operating system, is there
- provision for bypassing the KAPSE and accessing operating system
- functions directly?
-
- C. COMMAND LANGUAGE:
-
- 1. What language is the command language interpreter written in?
-
- 2. What are the interfaces between the command language interpreter and
- the KAPSE?
-
- 3. Are all interfaces from the command language interpreter and the
- tools through the KAPSE?
-
- 4. If not, what interfaces are not?
-
- 5. If there is an operating system underlying the KAPSE, does the
- command language interpreter support a mechanism to allow direct
- access to the functions of the underlying operating system?
-
- 6. Is the command language a menu system?
-
- 7. If so, are there methods by which experienced users can skip menu
- steps or otherwise expedite their path through the menu?
-
- 8. If a menu system is used, can an experienced user escape to a normal
- command language?
-
- 9. Is the command language host dependent?
-
- 10. Is the command language target dependent?
-
- 11. Is the syntax of the command language consistent with Ada syntax?
-
- 12. Is there a provision for a user defined startup command file to be
- executed at log on?
-
- 13. Does the command language interpreter provide facilities for user
- defined profiles (default settings) which are retained between
- sessions?
-
- 14. Does the command language interpreter provide a facility by which
- the user can query the system for information on his environment?
-
- 15. Does the command language interpreter provide facilities which
- support team programming efforts?
-
- 16. Does the command language interpreter allow a user to create or
- define a "working directory"?
-
- 17. Can a user change working directories?
-
- 18. If the answer to question 14 is yes, what restrictions are placed on
- such movement?
-
- 19. Can the user query system information such as date and time?
-
- 20. Is user access to all tools in the environment controlled by the
- command language or can the user bypass such controls by using the
- underlying operating system (if any)?
-
- 21. Is the tool invocation command format consistent across all tools in
- the environment?
-
- 22. When invoking a tool, what restrictions are placed on parameters?
-
- 23. Do default parameters exist for any of the tools?
-
- 24. Can a user change or modify such default parameters in his own
- environment?
-
- 25. Are default parameters displayed at tool invocation?
-
- 26. Are any tools restricted from invocation by levels of privilege?
-
- 27. If so, what are the levels of privilege, and who or what determines
- the level of privilege for a user?
-
- 28. Does the command language interpreter provide execution results to
- the user after a tool has completed execution?
-
- 29. Are such results consistent throughout the environment?
-
- 30. Are facilities provided by the command language to test the results
- of the execution of a tool?
-
- 31. What (if any) facility is provided to request that a tool be
- executed in background mode?
-
- 32. If a tool is executed in background, is the user provided with
- execution results upon completion of processing?
-
- 33. May a user query the status of a tool executing in background?
-
- 34. If background execution is allowed, what restrictions are placed on
- the number of executing background tasks?
-
- 35. Can a user terminate the execution of an arbitrary tool which is
- executing in background?
-
- 36. Is a batch execution ability provided?
-
- 37. If so, can the user halt the execution of a previously submitted
- batch process?
-
- 38. Can the user remove a previously submitted batch process from the
- queue before it begins execution?
-
- 39. Are the commands for accomplishing the operations in questions 34
- and 35 the same?
-
- 40. If not, how do they differ?
-
- 41. How does the user display the status of a batch process?
-
- 42. Can the user examine the batch queue in its entirity?
-
- 43. Can the user submit a command file which contains multiple command
- language interpreter commands for batch execution?
-
- 44. Can the user submit a command file which contains multiple command
- language interpreter commands for background execution?
-
- 45. Can the user submit a command file which contains multiple command
- language interpreter commands for foreground execution?
-
- 46. Can the user temporarily delay the execution of a previously
- submitted batch process?
-
- 47. Can the user specify that a batch process only be executed after the
- execution of another batch process has terminated?
-
- 48. Can the user specify that a batch process only execute if another
- batch process terminates successfully?
-
- 49. Can the user abort the processing of a tool being run in the
- foreground?
-
- 50. Can the user suspend the processing of a tool in foreground?
-
- 51. Can the user resume the processing of a suspended foreground tool?
-
- 52. Can the user accomplish the functions in questions 47 and 48 for a
- tool executing in background?
-
- 53. Can the user accomplish the functions in questions 47 and 48 for a
- batch execution?
-
- 54. Are the commands in a command file which is executed in the
- foreground echoed to the user's console?
-
- 55. Does the user have control over the echoing of such commands?
-
- 56. What branching and control mechanisms are allowed in command files?
-
- 57. Can the user control the issuing of error messages from command
- files?
-
- 58. Can a command file take action based on errors in its own execution?
-
- 59. Are macro or parameter facilities provided for use with user defined
- command files?
-
- 60. If so, what restrictions are placed on their use?
-
- 61. Does the command language interpreter provide facilities for a user
- to input comments?
-
- 62. Does the comment syntax conform to Ada comment syntax?
-
- 63. Does the command language interpreter recognize string literals?
-
- 64. Does the command language interpreter perform expression evaluation?
-
- 65. Does the command language interpreter perform arithmetic operations?
-
- 66. If so, does the operator precedence conform to Ada operator
- precedence?
-
- 67. If not, what is the operator precedence?
-
- 68. Does the command language interpreter perform boolean operations?
-
- 69. If so, does the boolean operator precedence conform to Ada operator
- precedence?
-
- 70. If not, what is the boolean operator precedence?
-
- 71. Does the command language interpreter perform string operations?
-
- 72. Are all Ada string operations performed?
-
- 73. Does the string operation precedence conform to Ada operator
- precedence?
-
- 74. If macro substitution capability is provided for command files, are
- there any predefined substitutions?
-
- 75. If so, list them.
-
- 76. Can a user redefine these predefined substitutions?
-
- 77. Is a facility provided for user symbol definition?
-
- 78. Is a distinction made between local and global symbols and macro
- substitution definitions?
-
- 79. Can the user record all terminal I/O in a user specified file?
-
- 80. Can the user redirect the input or output of a tools to/from a user
- defined file?
-
- 81. Can the user redirect the output of diagnostic and error messages to
- a file?
-
- 82. Does the command language interpreter provide the capability to
- construct Unix-like pipes?
-
- 83. Does the command language interpreter provide a branching mechanism
- such as an IF statement?
-
- 84. Does the command language interpreter provide a multi-way branching
- mechanism such as a CASE statement?
-
- 85. Does the command language interpreter provide a LOOP WHILE
- mechanism?
-
- 86. Does the command language interpreter provide an indexed looping
- mechanism?
-
- 87. Is the nesting of the above mechanisms allowed?
-
- 88. Are any restrictions placed on the nesting of such constructs?
-
- 89. Are conditional or unconditional exiting or cycling mechanisms
- provided for loops?
-
- 90. Does the command language interpreter provide for the display of
- commands after macro/parameter substitutions have been made but
- before execution?
-
- 91. Does the command language interpreter provide for the display of
- commands after expression evaluation has taken place, but before
- execution?
-
- 92. How does the user end an interactive session?
-
- 93. Can the user automatically execute command procedures as part of or
- after logoff?
-
- 94. What additional capabilities are provided to the user by the command
- language interpreter?
-
- D. ON-LINE ASSISTANCE:
-
- 1. Is the help facility hierarchical in nature?
-
- 2. If the facility is hierarchical, can the user move up and down the
- hierarchy at will?
-
- 3. Can the user execute sideways movement in the hierarchy?
-
- 4. Does the help facility provide both tutorials and explanatory text
- for all commands?
-
- 5. Does the help facility provide the required format for all command
- language commands?
-
- 6. How does the help facility react to requests for information that it
- does not have?
-
- 7. Is the help facility easily modified to add information for new
- tools or to delete information for tools no longer offered?
-
- 8. Are all interfaces of the help facility routed through the KAPSE?
-
- 9. Does the help facility have a capability to expand upon and explain
- all the error and diagnostic messages which are produced by the
- command language interpreter and by all the tools?
-
- E. EDITING:
-
- 1. What tools exist for the entry and revision of text in the system?
-
- 2. What languages are these tools written in?
-
- 3. Are the interfaces from these tools through the KAPSE?
-
- 4. List and explain any interfaces which are not through the KAPSE.
-
- 5. What on-line assistance is provided for the user of these tools?
-
- 6. Can this assistance be invoked without leaving these tools?
-
- 7. Does the system contain an Ada syntax directed editor?
-
- 8. If so, what special hardware does this editor require to operate?
-
- 9. Can the user escape from syntax oriented editing to screen or text
- oriented editing?
-
- 10. Does this editor output text or some other form of the program?
-
- 11. Can program text originally entered on a text editor be edited using
- this editor?
-
- 12. If so, does any special processing have to take place beforehand?
-
- 13. Does the system contain a full screen editor?
-
- 14. If so, what special hardware does this editor require to operate?
-
- 15. Does this editor display text exactly as it would be printed out?
-
- 16. Does the system contain a line oriented editor?
-
- 17. Does this editor require any special hardware to operate?
-
- 18. What occurs when the user attempts to access a non-existent file?
-
- 19. If a file is created when the user attempts to access a non-existent
- file, does the user have the opportunity to abort this action?
-
- 20. What occurs when the user attempts to access an inappropriate type
- of file?
-
- 21. What types of files are considered inappropriate for editing?
-
- 22. What happens when the user attempts to access a file for which the
- user does not have access rights?
-
- 23. Is there a facility by which the user can browse through a file
- which cannot be changed?
-
- 24. Is there a facility by which the user can recover from an abnormally
- terminated editing session?
-
- 25. If so, describe this facility, giving special attention to its
- granularity (how fine is the detail which is saved, is every
- keystroke saved, is the entire file resaved every so often, does
- this slow the user during editing, etc.)?
-
- 26. What other facilities are provided with the system for text entry
- and revision?
-
- F. TEXT FORMATTING:
-
- 1. What tools exist in the environment for the formatting of text?
-
- 2. What languages are these tools written in?
-
- 3. Are all the interfaces from these tools through the KAPSE?
-
- 4. List and describe any interfaces which are not through the KAPSE.
-
- 5. What on-line assistance is given to the user of the text formatting
- tools?
-
- 6. If the tools are interactive in nature, can this assistance be
- obtained without terminating operation of the tools?
-
- 7. What special hardware is required for the output of the tools?
-
- 8. Do the tools provide both left and right justification?
-
- 9. Do the tools provide line centering?
-
- 10. Do the tools provide page headers and footers?
-
- 11. Do the tools provide automatic page numbering?
-
- 12. Do the tools generate indexes and tables of contents?
-
- 13. Do the tools provide variable line spacing?
-
- 14. Do the tools provide variable margins?
-
- 15. What other text formatting functions do the tools provide?
-
- G. CONFIGURATION CONTROL:
-
- 1. Is configuration control an integral part of the environment, or
- must the user invoke special tools to achieve configuration control?
-
- 2. If special tools are used, list them.
-
- 3. What languages are these tools written in?
-
- 4. What are the interfaces between these tools and the KAPSE?
-
- 5. Are all interfaces to configuration control through the KAPSE?
-
- 6. If not, list and explain those which are not.
-
- 7. What on-line assistance is provided to users with regard to
- configuration control?
-
- 8. Are new revisions of objects automatically created each time the
- objects are modified?
-
- 9. If so, are previous revisions still available?
-
- 10. If previous revisions of objects are still available, are they
- stored in toto, or as a list of changes and revisions from some
- base?
-
- 11. If versions are stored as a list of changes and revisions from a
- base, is the base the original object, the current version of the
- object, or some other version?
-
- 12. Can earlier versions of an object be changed by the user?
-
- 13. Can earlier versions of an object be changed by any of the tools?
-
- 14. If earlier versions of objects can be changed, does this cause a
- branching of the revision history?
-
- 15. Can a version be frozen so that it is safe from change?
-
- 16. Can a frozen version be unfrozen?
-
- 17. What privilege level is required to accomplish these actions?
-
- 18. Can the entire data base for a project or for the system be
- archived?
-
- 19. What facilities are available to archive a selected portion of the
- data base?
-
- 20. Is the user allowed to record comments about the reason for a change
- to an object under the configuration control system?
-
- 21. Are the time and date of changes automatically recorded for all
- changes to objects?
-
- 22. Is the identity of the user making a change recorded?
-
- 23. Can a history of an object and its changes be output?
-
- 24. Can the configuration control system automatically insert version
- and date information in text form into an object under its control?
-
- 25. What additional configuration control capabilities are provided?
-
- H. COMPILER:
-
- 1. What language is the compiler written in?
-
- 2. Are all interfaces from the compiler through the KAPSE?
-
- 3. List and describe any interfaces which are not through the KAPSE.
-
- 4. What target machines does the compiler compile to?
-
- 5. Does the compiler compile to the host computer?
-
- 6. What on-line assistance is available to the user of the compiler?
-
- 7. Has the compiler passed the ACVC validation suite?
-
- 8. If not, has it been submitted for validation?
-
- 9. What percentage of the ACVC tests does the compiler pass?
-
- 10. How large is the compiler?
-
- 11. Is the compiler ever completely in memory?
-
- 12. If not, how large a portion of the compiler is in memory at any
- time?
-
- 13. What is the processing rate of the compiler?
-
- 14. How was this rate obtained?
-
- 15. For each target, does the compiler directly output object code?
-
- 16. For each target, does the compiler output assembly language?
-
- 17. For each assembly language output by the compiler, is a standard
- version of the assembly language used, and if so, which one?
-
- 18. Does the compiler support distributed machine targets? If so,
- explain for what targets this is true.
-
- 19. Is DIANA used as an intermediate language?
-
- 20. Is the DIANA generated by the various parts of the compiler
- available for use by other tools?
-
- 21. Can the compiler front end be run separately as a front end for
- other tools?
-
- 22. What global optimization capabilities are provided by the compiler?
- List specific optimizations.
-
- 23. What local optimization capabilities are provided by the compiler?
- List specific optimizations.
-
- 24. To what extent does the user control these optimizations?
-
- 25. Can the user specify optimization for minimal memory usage?
-
- 26. Can the user specify optimization for maximum speed of execution?
-
- 27. Can the user specify the use or non-use of specific optimizations?
-
- 28. What control does the user have over listings? Can the user control
- each of the following (on and off):
-
- - listing of the source text
-
- - reformatting of the source text
-
- - listing of text brought in with an "include" pragma
-
- - listing of the private parts of packages
-
- - listing of the attributes of all symbols in the source text
-
- - listing of a cross reference of symbols by locations used
-
- - listing of statistics concerning usage of symbols and Ada
- language constructs
-
- - listing of machine code or assembly language generated
-
- - listing of the intermediate language generated at various
- stages of compilation
-
- - listing of diagnostic and informational messages
-
- - listing of compilation statistics
-
- 29. Can calls into the runtime support packages be clearly distinguished
- in the machine code listing or assembly code listing?
-
- 30. Does the compiler generate a history file which records the person
- who performed the compilation and the date/time of the compilation?
-
- 31. Does the compiler record the versions of all compilation units used
- in the compilation?
-
- 32. Can the user halt the compilation after some stage, examine the
- current state of the compiler output, and restart the compiler?
-
- 33. Will the compiler generate code to automatically trace the execution
- of a statement, group of statements, or module?
-
- 34. Will the compiler generate code to track changes in value of any
- variable and print such changes?
-
- 35. Will the compiler generate code to provide statistics on the
- execution time/frequency of usage of any segment of code?
-
- 36. Are there various levels of diagnostic messages?
-
- 37. List the levels of diagnostic messages, give their meanings, and the
- result, if any of their occurrence in a program unit under
- compilation.
-
- 38. Can the user control the level of error which will abort
- compilation?
-
- 39. Can the user control the level of error which will abort the
- generation of object code?
-
- 40. Will the compiler generate code which will trace the occurrence of
- exceptions and the level at which they are handled?
-
- 41. Will the compiler generate code which will provide the same trace
- for exceptions which are not handled?
-
- 42. Do these facilities exist for programmer defined exceptions?
-
- 43. List and explain any pragmas which the compiler provides which are
- not specified in the language reference manual.
-
- 44. How many passes over the code does the compiler make?
-
- 45. Name and explain these passes.
-
- 46. What other capabilities are provided by the compiler?
-
- I. ASSEMBLER:
-
- 1. What tools are available in the environment to do assembly?
-
- 2. What languages are these tools written in?
-
- 3. What machines are these assemblers targeted to?
-
- 4. Are all the interfaces from these assemblers through the KAPSE?
-
- 5. List and explain any interfaces which are not through the KAPSE.
-
- 6. What on-line assistance is available to the user of these
- assemblers?
-
- 7. For each target, does the assembler accept a standard assembly
- language? If so, what standard?
-
- 8. Can modules assembled by these assemblers be interfaced to code
- produced by the Ada compiler?
-
- 9. Do these assemblers have a macro capability?
-
- 10. Do these assemblers have a conditional assembly capability?
-
- 11. What other capabilities do these assemblers provide?
-
- J. SOURCE CODE FORMATTING:
-
- 1. What tools exist in the system to provide formatting of Ada source
- code?
-
- 2. What languages are these tools written in?
-
- 3. Are all interfaces for these tools through the KAPSE?
-
- 4. List and describe any interfaces which are not through the KAPSE.
-
- 5. What on-line assistance is available to users of these tools?
-
- 6. Does the formatting standard imposed by these tools comply with that
- suggested by the language reference manual?
-
- 7. Does the user have the ability to customize this standard in any
- way?
-
- 8. How do these tools react to structural errors in the input program?
-
- 9. What occurs when the user attempts to use these tools to reformat a
- file of an inappropriate type?
-
- 10. What occurs when the user attempts to access a file for which the
- user does not have the access privilege?
-
- 11. What additional capabilities are provided by these tools?
-
- K. Ada LIBRARY MANAGEMENT:
-
- 1. Is library management an integral part of the environment, or does
- the user have to invoke specific tools to accomplish library
- management?
-
- 2. If the user must invoke specific tools, list these tools.
-
- 3. What language are these tools written in?
-
- 4. Are all interfaces used by these tools routed through the KAPSE?
-
- 5. List and explain any interfaces not through the KAPSE.
-
- 6. Are all Ada objects automatically part of an Ada library?
-
- 7. Can the user specify the library to be used as a current library?
-
- 8. Can the name of the current library be displayed?
-
- 9. What on-line assistance is provided to the user with regard to the
- library system?
-
- 10. Is the configuration control system integrated with the library
- system?
-
- 11. Are all library objects automatically under configuration control?
-
- 12. Can the user rename a library object?
-
- 13. Can the user create a library object?
-
- 14. Can the user delete a library object?
-
- 15. Can the user freeze an entire library?
-
- 16. Are there any privilege level restrictions on the manipulation of
- library objects?
-
- 17. How are objects grouped and related within a library?
-
- 18. Can the user directly access and manipulate objects in libraries?
-
- 19. How do objects in libraries differ from other data base objects?
-
- 20. Does the library provide a mechanism for determining "with"
- dependencies among library objects?
-
- 21. What additional facilities in library management are provided?
-
- L. LINKER:
-
- 1. What tools exist on the system to allow the user to link together
- modules compiled which have been separately compiled?
-
- 2. What languages are these tools written in?
-
- 3. Are all interfaces from these tools through the KAPSE?
-
- 4. List and describe any interfaces which are not through the KAPSE.
-
- 5. What on-line assistance is available to the user of these tools?
-
- 6. Are the linkers for the various targets of the system separate, or
- is a single (possibly parameterized) universal linker used?
-
- 7. Do these linkers support overlaying of code?
-
- 8. Do these linkers support linking of modules produced from different
- languages?
-
- 9. Do these linkers support distributed targets?
-
- 10. What other capabilities are provided by these tools?
-
- M. DOWNLOADING:
-
- 1. What tools are available in the environment to accomplish
- downloading of linked code to separate target systems?
-
- 2. What languages are these tools written in?
-
- 3. Are all interfaces from these tools through the KAPSE?
-
- 4. List and describe any interfaces which are not through the KAPSE.
-
- 5. What on-line assistance is available to the user of these tools?
-
- 6. What special hardware or interfaces are required by these tools?
-
- 7. What software has to exist in the target systems to allow the use of
- these tools?
-
- 8. What additional capabilities are provided by these tools?
-
- N. REQUIREMENTS PROCESSING:
-
- 1. What tools are available in the environment to process requirements?
-
- 2. What languages are these tools written in?
-
- 3. Are all interfaces from these tools through the KAPSE?
-
- 4. List and describe any interfaces which are not through the KAPSE.
-
- 5. What on-line assistance is available to the user of these tools?
-
- 6. Are any of these tools graphics based?
-
- 7. If so, is graphics used in the input or the output of these tools?
-
- 8. Is any special hardware required to support these tools?
-
- 9. Describe and explain these tools.
-
- O. DATA DICTIONARY MANAGEMENT:
-
- 1. What tools are provided for the management of data dictionaries?
-
- 2. What languages are these tools written in?
-
- 3. What are the interfaces between these tools and the KAPSE?
-
- 4. Are all interfaces between these tools through the KAPSE?
-
- 5. If the answer to the previous question was no, what interfaces are
- not through the KAPSE?
-
- 6. What on-line assistance is given to the user of the data dictionary
- system?
-
- 7. How is a data dictionary created?
-
- 8. Are data dictionaries automatically under configuration control?
-
- 9. How is a data item description added to the data dictionary?
-
- 10. Must all data items be fully defined, or can a data item be listed
- as TBD?
-
- 11. Can data items be defined hierarchically?
-
- 12. Can data items be defined in a logical (that is, implementation
- independent) manner?
-
- 13. If data items are defined in an implementation dependent manner,
- does this conform to Ada syntax?
-
- 14. Can data dictionaries be used throughout the life cycle?
-
- 15. Can a data dictionary be renamed by the user?
-
- 16. Are there any privilege level restrictions placed on manipulations
- of data dictionaries?
-
- 17. Can data dictionaries be reused by similar projects?
-
- 18. Does the data dictionary system allow automated lookup of data item
- descriptions by name?
-
- 19. Does the data dictionary system allow automated lookup of data items
- by keywords from their descriptions or associated comments?
-
- 20. What other facilities does the data dictionary system provide to the
- user?
-
- P. RAPID PROTOTYPING:
-
- 1. What tools exist in the system to do rapid prototyping?
-
- 2. What languages are these tools written in?
-
- 3. Are all interfaces to these tools through the KAPSE?
-
- 4. List and describe any interfaces which are not through the KAPSE.
-
- 5. What on-line assistance is available to the user of these tools?
-
- 6. Describe and explain the functioning of these tools.
-
- Q. DESIGN LANGUAGE PROCESSING:
-
- 1. What tools are available in the environment to process design
- languages?
-
- 2. What languages are these tools written in?
-
- 3. Are all interfaces from these tools through the KAPSE?
-
- 4. List and describe any interfaces which are not through the KAPSE.
-
- 5. What on-line help is available to the user of these tools?
-
- 6. To what extent is the design language recognized by these tools Ada?
-
- 7. Do these tools produce cross references of variables which appear in
- their input?
-
- 8. Do these tools produce cross references of program units which
- appear in their input?
-
- 9. Can these tools produce other cross references as specified by the
- user?
-
- 10. Do these tools produce module calling trees which show the calls
- made by all modules in their input?
-
- 11. Do these tools reformat the input according to the same standard as
- used by the system's source formatting tools?
-
- 12. What action do these tools take when they discover a structural
- error in their input?
-
- 13. What action do these tools take when the user attempts to access a
- file of the improper type?
-
- 14. What other capabilities are provided by these tools?
-
- R. TARGET ENVIRONMENT SIMULATION:
-
- 1. What tools exist in the environment to simulate target environments?
-
- 2. What languages are these tools written in?
-
- 3. Are all interfaces from these tools through the KAPSE?
-
- 4. List and describe any interfaces which are not through the KAPSE.
-
- 5. What on-line assistance is available to users of these tools?
-
- 6. Do these tools provide facilities for halting simulations and
- examining the status of the simulation?
-
- 7. Can these tools capture a trace of the simulation in a file which
- can be printed out later?
-
- 8. Can these tools be used with the source level debugger of the
- environment (if one exists)?
-
- 9. What other capabilities are provided by these tools?
-
- S. DEBUGGER:
-
- 1. What tools are available in the environment for debugging programs
- developed on the environment?
-
- 2. What languages are these tools written in?
-
- 3. Are all interfaces from these tools through the KAPSE?
-
- 4. List and describe any interfaces which are not through the KAPSE.
-
- 5. What on-line assistance is available to users of these tools?
-
- 6. Are these debuggers at the machine level or at the source code
- level?
-
- 7. If the debuggers are at the source code level, can a user insert
- breakpoints at specific statements?
-
- 8. Can the user ask for a program interruption each time a specific
- variable is changed?
-
- 9. After a program interruption has been issued, can the user query the
- value of specific variables by name?
-
- 10. Does the debugger provide a mechanism by which the user can query
- the values of large pointer-connected structures without repeatedly
- typing long path names?
-
- 11. Can the user ask that the value of a specific variable be printed
- each time that it is changed?
-
- 12. Do these tools work with programs run directly on the host machine,
- run in simulated target environments which run on the host machine,
- or run on a remote host?
-
- 13. If the tools work with programs run on a remote host, what special
- hardware is required for their operation?
-
- 14. What other capabilities are provided by these tools?
-
- T. TEST MANAGEMENT:
-
- 1. What tools are available to the user to accomplish management of
- testing?
-
- 2. What languages are these tools written in?
-
- 3. Are the interfaces between these tools, between the tools and the
- other tools in the system, and between the tools and the user all
- through the KAPSE?
-
- 4. List and describe any interfaces which are not through the KAPSE.
-
- 5. Does the test management system aid in the generation of tests?
-
- 6. Does the test management system aid in the generation of stubs for
- testing?
-
- 7. Does the test management system compare the results of testing with
- the expected results and deliver a report on success/failure?
-
- 8. Does the test management system keep a record of tests
- attempted/passed/failed?
-
- 9. Can the test mangement system report on the percentage of tests
- completed?
-
- 10. Does the test mangement system provide a driver for unit testing?
-
- 11. What other functions are provided by the test management system?
-
- 12. What on-line assistance is available to the user of the test
- management system?
-
- U. COST ESTIMATION:
-
- 1. What tools are provided in the system to support cost estimation?
-
- 2. What languages are these tools written in?
-
- 3. Are the tool to tool and tool to user interfaces through the KAPSE?
-
- 4. List and describe any interfaces which are not through the KAPSE.
-
- 5. What on-line help is available to the user in dealing with cost
- estimation?
-
- 6. What type of model is used in cost estimation?
-
- 7. What is the primary input on which the cost is based?
-
- 8. What other factors can be input to modify the cost estimate?
-
- 9. Can the user print a justification of the cost estimate which is
- derived automatically from the input data?
-
- 10. Are there any privilege level restrictions in the use of the cost
- estimation system?
-
- 11. Is the cost estimation system linked to the schedule generating
- system?
-
- 12. What other functions are supported by the cost estimation sytem?
-
- V. SCHEDULE MANAGEMENT:
-
- 1. What tools are available to aid in schedule generation?
-
- 2. What languages are these tools written in?
-
- 3. Are all interfaces from these tools through the KAPSE?
-
- 4. List and describe any interfaces which are not through the KAPSE.
-
- 5. Does the schedule system allow for the generation of a critical path
- analysis, showing which tasks are part of the longest path through
- the schedule?
-
- 6. Does the schedule system allow for the generation of alternate
- schedules by changing parameters and assumptions?
-
- 7. Is the generated schedule displayed in a graphic format?
-
- 8. Does the display or printing of the schedule require any special
- hardware?
-
- 9. What other functions does the scheduling system provide?
-
- W. TRACKING:
-
- 1. What tools are provided to assist with management tracking of the
- software development process?
-
- 2. What languages are these tools written in?
-
- 3. Are the interfaces from these tools through the KAPSE?
-
- 4. List and describe the interfaces which are not through the KAPSE.
-
- 5. Does the tracking system allow the tracking of project expenditures?
-
- 6. Does the tracking system allow the tracking of manhours?
-
- 7. Does the tracking system allow the tracking of the use of computer
- resources?
-
- 8. Does the tracking sytem allow the comparison of project milestones
- achieved versus scheduled milestones?
-
- 9. Does the tracking system support graphical output for histograms,
- line charts, activity charts, milestone charts and/or other such
- reports?
-
- 10. What other features does the tracking system provide?
-
- X. AUDITING:
-
- 1. What tools are provided in the system for doing auditing?
-
- 2. What languages are these tools written in?
-
- 3. Are the interfaces from the auditing function through the KAPSE?
-
- 4. List and explain any interfaces which are not through the KAPSE.
-
- 5. What on-line assistance facilities are available to the user of
- auditing?
-
- 6. What standards and rules are checked for by the auditor?
-
- 7. Can these standards be easily changed by the user?
-
- 8. Are there any privilege level restrictions involved in the use of
- the auditor?
-
- Y. COMPLEXITY MEASUREMENT:
-
- 1. What tools are provided in the system for doing complexity
- measurement?
-
- 2. What languages are these tools written in?
-
- 3. Are the interfaces from the complexity measurement function through
- the KAPSE?
-
- 4. List and explain any interfaces which are not through the KAPSE.
-
- 5. What on-line assistance facilities are available to the user of
- complexity measurement?
-
- 6. What metrics are used for complexity measurement? (for example
- McCabe's Cyclomatic Complexity or Halstead's Software Science)
-
- 7. What other capabilities are provided by these tools?
-