EVALUATION CRITERIA FOR ADA COMPILERS 11 September 1984 Author: Elizabeth Kean (Rome Air Development Center) in support of the E&V Team In June 1983, the AJPO proposed the formation of the E&V (Evaluation and Validation) Task and a tri-service APSE E&V Team, with the Air Force (Air Force Wright Aeronautical Laboratories (AFWAL)) designated as the lead service. The purpose of the E&V Team is to develop the techniques and tools which will provide a capability to perform assessment of APSEs and to determine the conformance of APSEs to the CAIS (Common APSE Interface Set). In order to accomplish this goal, a criteria questionnaire was developed and distributed at the SigAda meeting (30 July - 1 August 1984) in Hyannis, MA. Based upon comments received regarding this questionnaire, the following list of evaluation criteria for Ada compilers was composed. The criteria/tests are designed to complement the Ada Compiler Validation Capability (ACVC) test suite. The criteria are collected according to the set of attributes defined in the Requirements for Evaluation and Validation of Ada Programming Support Environments, dated 17 October 1984. Compiler areas yet to be covered indepth include the run-time system, program library tools, linking and loading facilities, etc. AVAILABILITY 1. Have there been any major software systems developed using this compiler? 2. For each of the compiler's targets, approximately how many users are there? To what degree have the targets been exercised? 3. Can the compiler be invoked in both an interactive and a batch mode? 4. Can the compiler be invoked while using other APSE tools? (While in the editor, for example.) 5. Are there any additional tools supplied with the compiler (e.g., symbolic debugger, target simulator, downloader, linker, etc.)? 6. Are there significant compiler features that could be considered above and beyond that specified in the Ada language specification? CAPACITY 1. What is the maximum number of errors detectable on a single line? 2. What is the maximum number of errors detectable in a single compilation unit? 3. What is the maximum number of Ada statements allowed for a single compilation unit? How are statements to be counted for this purpose? 4. What is the maximum number of compilation units allowed for a single file? 5. What is the maximum number of symbols allowed per compilation unit? 6. What is the maximum length allowed for a source line? KNOWN TESTS: (AIE)fen0801a, (AIE)fen0801b 7. What is the maximum size of the Intermediate Language that can be generated? 8. What are the maximum number of elements allowed in an enumeration type? 9. What is the maximum level of nesting in packages, loops, cases, record type declarations, IF-statements, blocks, etc? KNOWN TESTS: Packages: (AIE)fen0807a, (AIE)fen0807b, (AIE)fen0807c Loops: (AIE)fen0805a, (AIE)fen0805b, (AIE)fen0805c IFs: (AIE)fen0806a, (AIE)fen0806b, (AIE)fen0806c Blocks: (AIE)fen0804a, (AIE)fen0804b, (AIE)fen0804c Subprograms: (AIE)fen0808a, (AIE)fen0808b, (AIE)fen0808c, (AIE)fen0809a - 2 - 10. What are the maximum number of operands in expressions? 11. What is the maximum number of WITHed units allowed? KNOWN TESTS: 196-199 WITHs: (AIE)fen0803a, (AIE)fen0803b, (AIE)fen0803c, (AIE)fen0803d, (AIE)fen0803e 12. What is the maximum number of DIANA structures (i.e., symbol table entries) imported for a single compilation? 13. What is the maximum number of tasks that can be in existence? 14. What is the maximum number of tasks that can be spawned for a single program? 15. What is the maximum length of an identifier? 16. What is the maximum number of discriminants in a constraint? 17. What is the maximum number of fields in a record aggregate? 18. What is the maximum number of formals in a generic unit? 19. What is the maximum number of nested contexts? 20. What is the maximum number of indices in an array aggregate? 21. What is the maximum number of parameters in a call? 22. What is the maximum depth of expansion of INLINE subprograms? 23. What is the maximum precision of the fixed point, floating point and integer arithmetic? KNOWN TESTS: (AdaFair84)cauchf1.ada, (AdaFair84)cauchfy.ada, (AdaFair84)univ_ar.ada, (AdaFair84)cauchun.ada, (AVO)MATHTEST 24. What is the maximum/minimum size of the run-time system with/without tasking? 25. What is the size of the each of the phases of the compiler? 26. What is the minimum memory required to execute the compiler? 27. Is LOW_LEVEL_IO supported? 28. Do SEQUENTIAL_IO and DIRECT_IO "Work as expected?" Under what conditions, if any is USE_ERROR raised? 29. Do the compiler and runtimes implement non-blocking I/O and non-blocking operating system service calls? 30. What is the range of PRIORITY? What is the runtime task scheduling algorithm? 31. Are any limitations imposed on the use of pragmas SYSTEM_NAME, - 3 - STORAGE_UNIT, and MEMORY_SIZE? 32. What is the maximum number of overloadings per identifier? 33. Can generic bodies be compiled separately from their specifications? 34. Can any instances of erroneous execution be detected? 35. Can deadlock be detected? 36. What overhead is involved in context switching between tasks? KNOWN TESTS: (SRI)chain2, (SRI)chain5, (SRI)chain10, (SRI)chain20 37. How does the Ada compiler perform when compiling various applications programs? KNOWN TESTS: (AdaFair84)bsearch.ada, (AdaFair84)random.ada, (AdaFair84)set.ada (AVO.WORLDOF)directory_utility, (AVO)KPTOOLS, (AVO)FORMATTER, (AVO)STUBBER 38. How large is the compiler? 39. Is the compiler ever completely in memory? 40. What is the value of each of the predefined attributes (e.g., INTEGER'FIRST, INTEGER'LAST, FLOAT'DIGITS, etc.) 41. What is the maximum number of users that can invoke the compiler simultaneously? Has this number been verified? CONFIGURATION MANAGEMENT (CM) 1. What is the general CM plan? 2. Who is responsible for CM (for the supplier)? 3. Are all the source and object modules for a complete version available in one area (e.g., on a tape or a separately controlled disk area)? Is this area accessible to only one person or anyone on the project? 4. How are compiler fixes or enhancements incorporated into a new version? 5. How are new versions controlled and released? Which of these versions will be recertified or revalidated? 6. If the compiler produces code for more than one target, how are common and machine-dependent modules controlled? For example, are there conventions for naming common function modules for each target? 7. How are approved compiler changes incorporated into new versions? (The change approval procedures). - 4 - 8. Does there exist a list of all modules (including the run time system) that are needed for a complete compiler version? (A version description document). COST 1. What are the costs of acquiring the compiler? These costs should be given in terms of: a. Does the cost include the installation and required maintenance support? b. Is source code or only object code supplied? c. Costs of additional (non-supplier developed tools)? d. Monthly (or periodic) additional maintenance fees? e. Does the cost include receipt of new versions of the compiler and needed tools? f. Various cost options depending on licensing (proprietary) arrangements? 2. What is the estimated cost for a compiler rehost? 3. What is the estimated cost for a compiler retarget? 4. What do the costs for rehosting and retargetting include (test, integration, installation, etc.)? DOCUMENTATION 1. Is a requirements document available? Verify the content and quality. 2. Are complete design specifications available? Verify the content and quality as follows: a. An overview of the compiler design showing the major structure and design. b. Details of the compiler phases and passes. c. Separate sections which outline the design of the host and each target (to include the run time system(s)). d. Is the design detail sufficient such that an experienced compiler software engineer could maintain the compiler? e. Are compiler design changes updated in the documents? 3. Is sufficient user documentation available for the host and each target? Is it on-line? - 5 - 4. Does the users (or reference) manual contain an Appendix F which describes all implementation dependent characteristics? 5. Is the compiler release and compilation date identified on the hard copy listing? 6. Is documentation available for any special tools that were used for the compiler development? 7. Is documentation available for any separate tools that are needed for compiler operation? 8. What are the procedures used to update the documentation as a result of compiler changes? Who is responsible to verify that this is done? 9. Are there any alternative structures suggested for the production of low cost low quality back ends and high quality back ends? EFFICIENCY (For the following set of questions, speed is the number of source statements per CPU minute.) 1. What is the speed of object code generation (from the beginning of compilation)? KNOWN TESTS: (AIE)ben0701a 2. What is the speed of the Front-End if it can be run separately? 3. What is the speed of the Back-End if it can be run separately? 4. What is the speed of listing generation? 5. What is the speed of compilation with OPTIMIZATION=time? OPTIMIZATION= space? OPTIMIZATION=none? 6. Do the pragmas supported by the compiler, if any, impact the compilation speed? 7. What target machine independent optimizations are performed? For example, does the compiler perform constant propagation, constant folding, common subexpression elimination, expression simplification, strength reduction, range and constraint checks, removing unreachable code, cross-jumping, task context switch minimization, code sharing between generic instantiations, etc.? KNOWN TESTS: (AIE)ben0301a, (AIE)ben0301b, (AIE)ben0302a, (AIE)ben0302b, (AIE)ben0302c, (AIE)ben0302t, (AIE)ben0302e, (AIE)ben0303a, (AIE)ben0303b, (AIE)ben0303c, (AIE)ben0303d, (AIE)ben0303e, (AIE)ben0303f, (AIE)ben0303g, (AIE)ben0303h, (AIE)ben0303i, (AIE)ben0303j, (AIE)ben0303k, (AIE)ben0303l, (AIE)ben0303m, (AIE)ben0304a, (AIE)ben0304b, (AIE)ben0304c, (AIE)ben0304d, (AIE)ben0304e, (AIE)ben0304f, (AIE)ben0304g, (AIE)ben0401a, - 6 - (AIE)ben0401b, (AIE)ben0402a, (AIE)ben0501a, (AIE)ben0501b, (AIE)ben0501c, (AIE)ben0501d, (AIE)ben0502a, (AIE)ben0502b, (AIE)ben0502c, (AIE)ben0502d, (AIE)ben0502e, (AIE)ben0502f, (AIE)ben0502g, (AIE)ben0502h, (AIE)ben0502i, (AIE)ben0503a, (AIE)ben0504a, (AIE)ben0504b, (AIE)ben0504c, (AIE)ben0504d, (AIE)ben0504e 8. What target dependent optimizations are employed? 9. What is the probable effect of optimization on the raising and handling of exceptions (ess LRM 11.6)? 10. To what extent does the user control the optimizations? 11. Can the user specify the use/non-use of specific optimizations? 12. How does using select alternatives affect the performance of the executable code? KNOWN TESTS: (SRI) guard2, (SRI) guard20, (SRI) guard20e, (SRI) guard20et, (SRI) guards20t, (SRI) guards2e 13. Do idle tasks impact the performance of the executable code? KNOWN TESTS: (SRI)idle1, (SRI)idle5, (SRI)idle10, (SRI)idle20 14. Is it better to have many small tasks with single entry choices or a few large tasks with many select choices? KNOWN TESTS: (SRI)moretasks, (SRI)moretasks1, (SRI)moreselct, (SRI)moreselctr 15. Does the ordering of entry clauses in a SELECT impact execution speed? KNOWN TESTS: (SRI)order31, (SRI)order31r, (SRI)order32, (SRI)order100 16. If multiprocessing is supported by the implementation, are Ada tasks mapped to a single underlying process, or is each task mapped to a separate process? 17. If the target is a distributed system, how is "immediately" defined for conditional entry calls? 18. Does the size of a parameter affect performance? KNOWN TESTS: (SRI)passarrys, (SRI)passarryb, (SRI)passinout 19. Does the number of select choices affect performance? KNOWN TESTS: (SRI)select2, (SRI)select2e, (SRI)select20, (SRI)select20e 20. Can the Ada scheduler starve a task? KNOWN TESTS: (SRI)schedtest 21. What is the size of the resulting object code for OPTIMIZE=none? OPTIMIZE=space? OPTIMIZE=time? - 7 - 22. What is the CPU time required for execution of code compiled with OPTIMIZE=none? OPTIMIZE=time? OPTIMIZE=space? KNOWN TESTS: (AVO.WORLDOF)eratos.r, (AVO.WORLDOF)float_test.r, (AVO.WORLDOF)generator.r, (AVO.WORLDOF)io_test.r, (AVO.WORLDOF)item_body.r, (AVO.WORLDOF)item_spec.r, (AVO.WORLDOF)1f_disk_80.tex, (AVO.WORLDOF)1f_pgmap.text, (AVO.WORLDOF)list_body.r, (AVO.WORLDOF)sieve_task.r, (AVO.WORLDOF)stinger.r, (AVO.WORLDOF)timing.r, (AVO.WORLDOF)tower.r, (AVO.WORLDOF)tran_spec.r, (AVO.WORLDOF.juggling1)e_ball_1.r, (AVO.WORLDOF.juggling1)1_hand_1.r, (AVO.WORLDOF.juggling1)o_ball_1.r, (AVO.WORLDOF.juggling1)r_hand_1.r, (AVO.WORLDOF.juggling2)counter.r, (AVO.WORLDOF.juggling2)even_ball.r, (AVO.WORLDOF.juggling2)get_jug.r, (AVO.WORLDOF.juggling2)left_hand.r, (AVO.WORLDOF.juggling2)monitor.r (AVO.WORLDOF.juggling2)odd_ball_1.r, (AVO.WORLDOF.juggling2)print_pos.r, (AVO.WORLDOF.juggling2)right_hand.r 23. What is the difference in size between object code generated by an Ada compiler as compared with other languages (e.g., Pascal, COBOL, C, FORTRAN, Assembler, etc.)? 24. What is the difference in execution time between object code generated by an Ada compiler as compared with other languages? 25. What is the execution time (in CPU seconds) for file operations (with TEXT_IO, DIRECT_IO, etc.)? KNOWN TESTS: (AdaFair84)char_dir.ada, (AdaFair84)char_pnum.ada, (AdaFair84)int_dir.ada, (AdaFair84)char_text.ada, (AdaFair84)int_text.ada 26. What is the execution time (in CPU seconds) for arithmetic and logical operations? KNOWN TESTS: (AdaFair84)boolvec.ada, (AdaFair84)floatvec.ada, (AdaFair84)intvec.ada, (AVO)AHL, (AVO)AUSSIE, (AVO)BASEMATH, (AVO)ADD, (AVO)MULT, (AVO)PUZZLE, (AVO)SEARCH, (AVO)SIEVE, (AVO)SYNTHETIC(Whetstone), (AVO.MARK)ADD, (AVO.MARK)MULT, (AVO.MARK)PUZZLE, (AVO.MARK)SEARCH, (AVO.MARK)SIEVE, (AVO.MARK)SYNTHETIC(Whetstone) 27. What is the execution time (in CPU seconds) for simple procedures with scalar parameters? KNOWN TESTS: (AdaFair84)proccal.ada 28. What is the execution time (in CPU seconds) for a simple rendezvous? KNOWN TESTS: (AdaFair84)rendez.ada 29. What is the effect on code size and execution time of the pragma SUPPRESS? 30. What is the effect on code size and execution time of using - 8 - UNCHECKED_DEALLOCATION vs. pragma CONTROLLED vs. neither? 31. What is the effect on code size of a DEBUG option, if is exists? 32. What is the procedure calling overhead for procedures and functions? KNOWN TESTS: (AdaFair84)ackerman.ada, (AVO)ACKER, (AVO.WORLDOF)ackerman.r 33. Are there Ada features that produce more efficient object code with respect to time and space? KNOWN TESTS: (AdaFair84)qsortpar.ada, (AdaFair84)qsortseq.ada EXTENDABILITY 1. What were the original design goals of the compiler? a. Was the design intended for a particular class of users? b. Are any specific applications envisaged? 2. Is the compiler is written in Ada, were the use of certain language constructs avoided (e.g., tasking, generics, real arithmetic)? If so, which ones and why? 3. If the compiler is written in Ada, has it successfully recompiled itself? 4. Were any special tools such as a compiler-compiler, translator writing systems, etc., used during the development. If so, are they available to possibly construct additional tools? a. Do these tools generate a source program of the compiler or do they translate directly into object code? b. If these tools do not generate an Ada (or other language) program, how can the tools be retargeted? c. What languages are the tools written in? GRANULARITY 1. What are the major compiler phases? What phases are in memory as the compilation progresses? 2. To what degree are the components of the compiler separately executable tools? Is their use documented? 3. What parts of the compiler are seen as useful in building other tools? HARDWARE - 9 - 1. What are the host/target pairs for the compiler? 1. For each target, does the compiler directly output (relocatable) object code? 3. For each target, does the compiler output assembly language? Is it a standard version? 4. What is the character set of the host? of the target? 5. Are hardware machine dependencies clearly identified in both the code and documentation? 6. Is the compiler designed to use virtual memory? 7. Is the compiler designed to take advantage of a multiprocessing implementation? (Was the compiler designed to use multitasking?) 8. Are hardware dependencies concealed by module interfaces? 9. Does the compiler support distributed machine targets? If so, which ones? INTERFACES 1. Is the major design interface to a KAPSE or the host operating system? 2. If the interface is to a KAPSE, what KAPSE facilities does the compiler use? 3. Does the compiler operate in a particular APSE? If so, what APSE (or MAPSE) tools does it require, if any? 4. If not part of an APSE, what characteristics of the host operating system does the compiler rely on? Are all such system dependencies concealed behind module interfaces? 5. Which interfaces are regarded as significant for rehosting or retargeting? 6. What other tools (e.g., symbolic debugger) does the compiler interface with? a. To what extent are the interfaces documented? b. Can alternative tools be written conforming to these interfaces? INTEROPERABILITY 1. What compiler generated information is available to other tools? Symbol table? Cross-reference table? Intermediate forms? Listing outputs? - 10 - 2. Does the compiler share, or make use of, other APSE (or operating system) tables or information? MAINTAINABILITY 1. What language(s) is the compiler written in? 2. Are instructions available to enable a non-compiler person to install the compiler on an identical host system? 3. Are the procedures for complete compiler generation (from source to executable) documented? 4. What arrangements are available for maintenance? Such arrangements can range from postal service to an on-call maintenance staff. 5. What is the quality of maintenance support? a. Designated persons for maintenance contract? b. Availability of maintenance documentation? c. Telephone query service, visits by supplier staff, courses, etc? 6. What are the arrangements for charging for maintenance and/or support of the compiler? PROPRIETARY 1. Can a user install the compiler, or must the supplier do the installation? 2. Are there any proprietary restrictions on compiler release (e.g., no source supplied, data rights, etc.)? 3. Are there any restrictions on special (non-supplier developed) tools needed for compiler operation? Also, for any optional tools that may be useful? 4. Does the supplier allow others to perform a rehost or retarget? 5. Under what circumstances may the source be made available for a rehost or retarget? 6. What are the licensing arrangements for the compiler (e.g., at how many sites can the compiler be used)? 7. What agreement does the user have to sign before the compiler may be supplied to others? 8. Can a license to distribute the compiler to others be bought or leased? What parts of the compiler (run time system, packages, - 11 - separate tools, etc.) can be distributed? Can source be included? 9. Can a license to use the compiler be bought outright or leased? 10. What are the arrangements (if any) for the release of information about the compiler's internal structure? 11. Are there any restrictions on the use and/or distribution of software produced by the compiler? It should be noted that the software produced often contains a run time system delivered by the compiler supplier. REHOSTABILITY 1. Has the compiler been rehosted? 2. What module (or modules) of the front end (machine independent) need to be modified for the rehost? 3. Is there a manual which describes the steps necessary to rehost the compiler? 4. Are system dependencies adequately isolated and documented? 5. Is there a kit of tools and/or components available to help with the rehosting task? 6. Is the compiler sufficiently modular to allow implementation of critical parts (such as major data structures) to be easily altered for the rehost task? 7. Is an estimate of time given for the rehost? RETARGETABILITY 1. What modules of the back end (machine dependent) need to be modified for a retarget? 2. What modules of the front end need to be modified for a retarget and what are their interfaces? a. What techniques are used: Package standard, parameter file, a special package that is linked in, etc.)? Hopefully, this is not wired in to the compiler. 3. Are there any automated tools to aid in the retarget process? 4. Is an estimate of time given for the retarget task? 5. Is there a manual describing the procedures for retargeting? Possibly with examples. 6. For the intermediate language retargeting interface, is the intermediate language tree structured, linear, etc? - 12 - 7. Is there more than one level of intermediate language at which retargetting is carried out? 8. For the retarget process, what assumptions are made in the design of, and requirements of, the run time system (e.g., tasking monitor, storage allocation scheme, etc.)? 9. For a retarget, do the presence of other tools in the compilation system or APSE affect the back end? ROBUSTNESS 1. What safeguards are implemented for protection and recovery against unforeseen system, user and its own failures? Data protection? Internal exception handlers? Trace back facility? 2. As more than one simultaneous user invokes the compiler, by how much does the mininum size of memory needed to run the compiler increase? 3. Are any resources other than primary and secondary storage needed to invoke the compiler? 4. Are compiler phases overlaid to reduce memory occupancy? If so, are any requirements placed on the system? TEST AVAILABILITY 1. What tests are available from the supplier to verify compiler operations? a. Are the tests documented in a test plan? b. Are instructions for use available? USABILITY 1. If the compiler interface is to the host operating system, under which operating system version (or release) does the compiler operate? Also, which target(s) operating system(s)? 2. If the compiler interface is to an APSE, under what version (or release) of the APSE does the compiler operate? 3. What are the values outside the range of safe numbers for real types? 4. Are there any restrictions on the use of the generic procedure UNCHECKED_DEALLOCATION? 5. Are there any restrictions on the use of the generic procedure UNCHECKED_CONVERSION? 6. Does the compiler generate a history file which records source - 13 - file name, compilation unit name, program library name, owner(s) of each and the date/time of the compilation? 7. Does the compiler record the versions of all compilation units used in the compilation? 8. How many passes over the code (source and IL) does the compiler make? 9 What compiler options are available? For example, can the user specify the following options: no semantics, no code, comments, lookahead, optimize (off, space, time, etc), choice of runtime kernels, compiler maintenance, compiler new version of package STANDARD, debug, main program identification, prettyprint source, provide user with traceback information for unhandled exceptions, etc? KNOWN TESTS: (AIE)fen0501a, (AIE)fen0502a, (AIE)fen0503a, (AIE)fen0503b, (AIE)fen0503c, (AIE)fen0503d, (AIE)fen0503e, (AIE)fen0503f, (AIE)fen0504a, (AIE)fen0504b, (AIE)fen0504c, (AIE)fen0504d, (AIE)fen0504e, (AIE)fen0504f, (AIE)fen0505a, (AIE)fen0505b, (AIE)fen0506a, (AIE)fen0506b, (AIE)fen0506c, (AIE)fen0506d, (AIE)mid0501a, (AIE)mid0502b, (AIE)mid0502d, (AIE)mid0503a, (AIE)ben0201a, (AIE)mid0501b, (AIE)mid0501c, (AIE)mid0502c, (AIE)ben0102a (AIE)mid0502a, (AIE)ben0101a, (AIE)ben0202a 10. What control does the user have over listings? For example, can the user control: listing source text, reformatting the source, listing text of an instantiated generic unit, listing private parts of packages, listing attributes of all symbols in source text, listing cross reference information, listing statistics, listing machine/assembly code, listing the intermediate language generated, listing diagnostics, listing use of machine-dependent features, pragmas, etc.? 11. Does abnormal termination leave a consistent program library? If not, how can a user restore consistency? Are checkpointing and/or transaction updating supported? 12. Is the intermediate language generated available for other tools? 13. Can the source be reconstructed from the intermediate language (e.g., DIANA, or Abstract Syntax Tree)? 14. Are all LRM pragmas properly recognized or handled? What implementation dependent pragmas are supported, if any? KNOWN TESTS: Pragma OPTIMIZE: (AIE)mid0503a, (AIE)mid0503b Pragma STATIC: (AIE)mid0504a, (AIE)mid0504b Pragma INLINE: (AIE)mid0505a, (AIE)ben0505a, (AIE)ben0505b 15. Are the parameter passing methods and subprogram calls handled efficiently? KNOWN TESTS: PASSED BY VALUE (<= 64 BITS): (AIE)mid0701a, (AIE)mid0701b PASSED BY VALUE (>= 64 BITS): (AIE)mid0702a, (AIE)mid0702b SELECTED PASSING METHODS: (AIE)mid0703a, (AIE)mid0703b PASSED BY REFERENCE (DYNAMIC): (AIE)mid0704a, (AIE)mid0704b - 14 - 16. Are there any restrictions on unchecked conversions? on unchecked deallocation? KNOWN TESTS: (AdaFair84)lowlev.ada 17. Will the compiler generate code to automatically trace the execution of a statement, group of statements, or module? 18. Will the compiler generate code to track changes in value of any variable and print changes? 19. Will the compiler generate code to provide statistics on the execution time/frequency of usage of any segment of code? 20. Will the compiler generate code which will trace the occurrence of exceptions and the levels at which they are handled? 21. If representation specifications are implemented, are there any restrictions? If so, what are they? KNOWN TESTS: (AdaFair84)derived.ada 22. What is the accuracy of the error message positioning? 23. What is the time consumption for error detection and recovery? 24. What is the clarity of error messages in terms of the language? KNOWN TESTS: (AdaFair84)friend.ada 25. Will the compiler produce an executable object program even if errors are present? If so, under what conditions and what is the effect of executing a source statement that contains an error? 26. Are steps taken to avoid "cascading" of compilation errors? 27. What warning messages does the compiler print? For example, warning for ignored pragmas, warning for an unusually expensive construct, etc.? Is the user able to switch off certain warning messages? 28. Can the user control the level of error which will abort the compilation? 29. Can the user halt the compilation after some stage, examine the current state of the compiler output and restart the compiler? 30. Is the compiler re-entrant? 31. Does using the debugger, if one exists, require compiling the software with a special DEBUG option? 32. Can two or more compilations access and/or update the same library simultaneously? If so, how is consistency maintained? 33. Can any instances of erroneous execution be automatically - 15 - detected? How are such instances handled? - 16 - REFERENCES 1. Generic APSE Evaluation Questions, General Dynamics 2. Ada Integrated Environment (AIE) Test Procedures for the Compiler Front-End, Intermetrics, Inc. (Available through RADC upon completion of the AIE contract) 3. AIE Test Procedures for the Compiler Middle-Part, Intermetrics, Inc. (Available through RADC upon completion of the AIE contract) 4. AIE Test Procedures for the Compiler Back-End, Intermetrics, Inc. (Available through RADC upon completion of the AIE contract) 5. AIE Test Procedures for the Compiler Subsystem, Intermetrics, Inc. (Available through RADC upon completion of the AIE contract) 6. SRI Tests (Available on USC-ECLB) 7. AdaFair 84 Tests (Available on USC-ECLB) 8. World of Ada Tests (Available through AJPO) 9. Ada-Europe Guidelines for Ada Compiler specification and selection by J C D Nissen and B A Wichmann (et al) (ACM Ada Letters, July, August 1983) 10. Mike Burlakoff, University of Missouri 11. Requirements for Evaluation and Validation of Ada Programming Support Environments, Version 1.0, XX XXX XX. **It is the intent of the E&V Team to eventually make as many tests as possible available through EV-INFORMATION. - 17 - APPENDIX A ADA FAIR '84 TESTS ackerman.ada recursion, procedure calling overhead boolvec.ada time for "AND" operation on boolean vector bsearch.ada generic binary search cauchf1.ada floating point accuracy cauchfy.ada fixed point accuracy univ_ar.ada universal real and integer arithmetic cauchun.ada test universal arithmetic package char_dir.ada time for file operations using DIRECT_IO with characters char_pnum.ada time for file operations using TEXT_IO and ENUMERATION_IO with characters char_text.ada time for file operations using TEXT-IO with characters int_dir.ada time for file operations using DIRECT_IO with integers int_text.ada time for file operations using TEXT_IO with integers conprod.ada tasking performance using buffering task derived.ada inter_conversion of derived types with different representations floatvec.ada time for adding elements of a large floating point vector intvec.ada time to add elements to a large integer vector friend.ada friendliness of compiler - warnings,exceptions lowlev.ada test length of clauses and unchecked conversion proccal.ada time for simple procedure calls with scalar parameters qsortpar.ada compare parallel and sequential sort algorithms qsortseq.ada compare parallel and sequential sort algorithms random.ada random number generation, generate linear congruential sequences rendez.ada time for simple rendezvous set.ada implementation of sets shared.ada tasking to provide shred access to global variables - 18 - AVO TESTS ACKER ackerman function AHL test arithmetic and random number AUSSIE machine arithmetic BASEMATH simulation of base number arithmetic for comparing COMP check assumptions about behavior of computer COMPA declarations - separate statements and lines COMPB declarations - one statement and many lines COMPC assignments - several per line COMPD assignments - one per line COMPE alternating comments and assignments - one per line COMPF comments followed by assignments - one per line COMPG tests linearity, alternating comments and assignments COMPN null procedure COMPT with and use COMPZ test program - 1 assignment 1 declaration MARK set of benchmarks ADD simple addition MULT simple multiplication PUZZLE move pieces SEARCH find character string SIEVE byte prime number benchmark SYNTHETIC synthetic benchmark MATHLIB elementary math functions MATHTEST test elementary math functions KPTOOLS insulate "software tools" from system dependencies FORMATTER pretty printer system - put code in military standard STUBBER creates package bodies from text WORLDOF set of thirty-nine benchmarks ackerman.r enumeration types, non-primitive recursion, speed requirements, space requirements ambiguity.r overloading, disambiguation, package linkage, separate compilation dir_body.r locate, insert, delete, create - item/dictionary dir_spec.r directory functions - create, delete, change, add, display, scan dir_util.r create and manipulate directory using directory functions eratos.r iterative statements, basic arrays, speed of execution float_test.r floating points with transcendental functions generator.r separate procedure specifications and body specifications io_test.r I/O capabilities item_body.r get, display item item_spec.r get, display, compare items 1f_disk_80.tex disk functions 1f_pgmap.text page functions list_body.r generics, private types, pragmas, exceptions list_test.r generics, overloading, private types, pragmas, exceptions manager.r generics, overloading, private types, pragmas, exceptions overload.r ambiguous overloading of "+" resolution.r overloading - 19 - roman.r overloading, disambiguation, with Roman_Numerals package roman_body.r overloading, disambiguation, with Roman_Numerals package sieve_body.r body of sieve task sieve_task.r solve sieve of Erotothenes problems using tasking stinger.r basic string operations timing.r timing functions tower.r records, discriminants, non-primitive recursion, execution speed tran_body.r body of transcendental functions using iteration tran_spec.r number crunching abilities juggling1.r tasking, entry calls, exceptions, subunits, single compilation of multiple units e_ball_1.r tasking, subunits 1_hand_1.r tasking, subunits o_ball_1.r tasking, subunits r_hand_1.r tasking, subunits juggling2.r tasking, selective waits, families of entries, exceptions, subunits, separate compilations counter.r same as juggling2.r even_ball.r same as juggling2.r get_jug.r same as juggling2.r left_hand.r same as juggling2.r monitor.r same as juggling2.r odd_ball_1.r same as juggling2.r print_pos.r same as juggling2.r right_hand.r same as juggling2.r - 20 - SRI TESTS ALL TESTS ARE ADA TASKING TESTERS DETERMINE OVERHEAD IN CONTEXTS SWITCHES BETWEEN TASKS chain2 with chain length of two chain5 with chain length of five chain10 with chain length of ten chain20 with chain length of twenty DETERMINE IF GUARDS ON ENTRY STATEMENTS IMPACT PERFORMANCE guard2 with one guard set to true and one set to false guard20 with one guard set to true and nineteen set to false guard20e with one guard set to true and nineteen set to false guard20et with all guards set to true guards20t with all guards set to true guards2e with one guard set to true and one set to false DETERMINE WHETHER IDLE TASKS IMPACT PERFORMANCE idle1 with one idle task idle5 with five idle tasks idle10 with 10 idle tasks idle20 with twenty idle tasks DETERMINE IF IT IS BETTER TO HAVE LOTS OF LITTLE TASKS WITH SINGLE ENTRY CHOICES OR A FEW BIG TASKS WITH MANY SELECT CHOICES moretasks master task calls twenty slave tasks with single entries moretasks1 each task has a single entry embedded in a select statement moreselct master task calls each entry in a single slave task moreselctr entries are listed in opposite order from calling order DETERMINE IF ORDERING OF ENTRY CLAUSES IN A SELECT MATTERS order31 with thirty-one choices order31r entries are called in reverse order of select statement order32 with thirty-two choices order100 with one hundred choices DETERMINE IF SIZE OF PASSED PARAMETER MAKES A DIFFERENCE passarrys with small "in" array of integers passarryb with larger "in" array of integers passinout with large "in out" array of integers DETERMINE IF THE NUMBER OF SELECT CHOICES MAKES A DIFFERENCE select2 with two choices, desired entry is first select2e with two choices, desired entry is last select20 with twenty choices, desired choice is first select20e with twenty choices, desired choice is last DETERMINE IF ADA SCHEDULER MAY STARVE A TASK schedtest with two entry select statements used independently by three other tasks - 21 - AIE FRONT END TESTS SYNTAX ERRORS (AIE)fen0301a checks frequent comment syntax mistakes (AIE)fen0302a checks miscellaneous common spelling mistakes (AIE)fen0303a checks common Ada/Pascal errors (AIE)fen0304a checks common errors in subprogram specifications (AIE)fen0305a checks common errors in subprogram declarations COMPILER OPTIONS (AIE)fen0501a verifies that default options are as specified (AIE)fen0502a verifies that specified options take on the given values (AIE)fen0503a checks the NOSEM option (AIE)fen0503b checks the NOSEM option (AIE)fen0503c checks the NOSEM option (AIE)fen0503d checks the NOSEM option (AIE)fen0503e checks the NOSEM option (AIE)fen0503f checks the NOSEM option (AIE)fen0504a checks the NOCODE option (AIE)fen0504b checks the NOCODE option (AIE)fen0504c checks the NOCODE option (AIE)fen0504d checks the NOCODE option (AIE)fen0504e checks the NOCODE option (AIE)fen0504f checks the NOCODE option (AIE)fen0505a checks the COMMENT option (AIE)fen0505b checks the COMMENT option (AIE)fen0506a checks the LOOKAHEAD option (AIE)fen0506b checks the LOOKAHEAD option (AIE)fen0506c checks the LOOKAHEAD option (AIE)fen0506d checks the LOOKAHEAD option CHECK CORRECT MEANING OF DOT "." AND TIC "'" (AIE)fen0701a verify dot as a delimiter in numeric range (AIE)fen0701b verify dot as part of a numeric literal (AIE)fen0701c verify tic in attribute selection (AIE)fen0701d verify tic in type qualification (AIE)fen0701e verify tic in character literal CHECK HIDING (AIE)fen0702a by nested blocks (AIE)fen0702b by loop parameters (AIE)fen0702c by nested packages (AIE)fen0702d using the extended scope of a pkg spec through its corresponding body (AIE)fen0702e with nested (non-overloaded subprograms (AIE)fen0702f within a record type declaration (AIE)fen0702g by an object renaming declaration (AIE)fen0702h by nesting a subprogram declaration (AIE)fen0702i by a formal parameter of nested subprogram declaration (AIE)fen0702j by nesting a task definition (AIE)fen0702k of an object by a subprogram declaration with the same identifier (AIE)fen0702l checks hiding of a predefined type from STANDARD CHECKS OVERLOADING ENTITIES (AIE)fen0703a using a procedure and a function (AIE)fen0703b using procedures with different numbers of parameters (AIE)fen0703c using functions with different numbers of parameters - 22 - (AIE)fen0703d using procedures with different base types of parameters (AIE)fen0703e using functions with different base types of parameters (AIE)fen0703f using functions with different return base types (AIE)fen0703g using procedures with differently ordered parameters (AIE)fen0703h overloading an operator CHECKS USE CLAUSES (AIE)fen0704a using two nested packages, one which is USEd (AIE)fen0704b using nested package which has an identifier that should not hide a directly visible identifier (AIE)fen0704c using nested packages with dot selection (AIE)fen0704d using separately compiled pkgs using WITH and USE CHECKS VERY LARGE SOURCE LINES (AIE)fen0801a verifies line length of 255 allowed (AIE)fen0801b verifies line length of >255 flagged CHECKS LARGE NUMBERS OF VMM SUBDOMAINS (AIE)fen0803a opens 198 subdomains using a WITH chain 197 units (AIE)fen0803b opens 199 subdomains using a WITH chain 198 units (AIE)fen0803c opens 200 subdomains using a WITH chain 199 units (AIE)fen0803d opens 199 subdomains using a WITH chain 196 units with 2 nested units (AIE)fen0803e opens 200 subdomains using a WITH chain 197 units with an indirect WITH and a subunit CHECK OVERFLOW OF PARSE STACK OR STATE STACK OF THE PARSER (AIE)fen0804a using nested block statements level 5 (AIE)fen0804b using nested block statements level 10 (AIE)fen0804c using nested block statements level 15 (AIE)fen0805a using nested loop statements level 5 (AIE)fen0805b using nested loop statements level 10 (AIE)fen0805c using nested loop statements level 15 (AIE)fen0806a using nested if statements level 5 (AIE)fen0806b using nested if statements level 10 (AIE)fen0806c using nested if statements level 15 (AIE)fen0807a using nested package statements level 5 (AIE)fen0807b using nested package statements level 10 (AIE)fen0807c using nested package statements level 15 (AIE)fen0808a using nested subprogram statements level 5 (AIE)fen0808b using nested subprogram statements level 10 (AIE)fen0808c using nested subprogram statements level 15 (AIE)fen0809a using WITH, USE, variable decls, arithmetic and logical expressions which are not quickly resolvable - 23 - AIE MIDDLE PART TESTS CHECKS GENERIC INSTANTIATIONS (AIE)mid0301a verify Diana is not modified if no generic instantiation (AIE)mid0302a verify instance body created for instantiation is correct Diana CHECK GENERICS COMPILED AT DIFFERENT TIMES IN VARIOUS ORDERS (AIE)mid0303a generic procedure declaration, body and instantiation (single compilation) (AIE)mid0303b generic procedure declaration, instantiation, and body (single compilation) (AIE)mid0303c generic procedure declaration, body and instantiation (separate compilation) (AIE)mid0303d generic procedure declaration, instantiation and body (separate compilation) (AIE)mid0303e generic procedure declaration, its package body containing the body and an instantiation - main proc using the instantiation (separate compilation) (AIE)mid0303f generic procedure declaration, its package body containing the generic body stub and an instantiation, the generic body subunit (main proc using the instantiation)(separate compilation) (AIE)mid0304a generic package declaration, body and instantiation (single compilation) (AIE)mid0304b generic package declaration, instantiation, and body (single compilation) (AIE)mid0304c generic package declaration, body and instantiation (separate compilation) (AIE)mid0304d generic package declaration, instantiation and body (separate compilation) (AIE)mid0304e generic package declaration, its package body containing the body and an instantiation - main proc using the instantiation (separate compilation) (AIE)mid0304f generic procedure declaration, its package body containing the generic body stub and an instantiation, the generic body subunit (main proc using the instantiation)(separate compilation) (AIE)mid0305a generic procedure declaration, instantiation, and main procedure (single compilation) (AIE)mid0305b generic package declaration, instantiation and main procedure (separate compilation) (AIE)mid0306a generic package declaration, instantiation, and main procedure (single compilation) (AIE)mid0306b generic package declaration, instantiation and main procedure (separate compilation) CHECK SHARING OF GENERIC BODY INSTANTIATION REPRESENTATIONS (AIE)mid0307a check that 'in' actual parameters match (AIE)mid0308a check that 'in out' actual parameters match (AIE)mid0309a check that integer type actual parameters match (AIE)mid0310a check that floating point type actual parameters match (verifies when they have the same amount of storage) (AIE)mid0310b check that floating point type actual parameters match (verifies if they have identical rep specs) (AIE)mid0311a check that fixed point type actual parameters match - 24 - (verifies when they have the same amount of storage) (AIE)mid0311b check that fixed point type actual parameters match (verifies if they have identical rep specs) (AIE)mid0312a check that discrete type actual parameters match (verifies when they have the same amount of storage) (AIE)mid0312b check that discrete type actual parameters match (verifies if they have identical rep specs) (AIE)mid0313a check that access type actual parameters match CHECK CROSS REFERENCE INFORMATION TO si_refs ATTRIBUTE OF DEF_IDs (AIE)mid0401a which are subtypes, types, and variables (AIE)mid0401b verify no information is added when LIST => NOXREF (AIE)mid0401c which are procedures, functions, operators, subprogram parameters (AIE)mid0401d which are labels and packages (AIE)mid0401e verifies references are not included from uses in other units (AIE)mid0401f when LIST => XREF CHECK CROSS REFERENCE INFORMATION TO si_calls ATTRIBUTE OF DEF_IDs (AIE)mid0402a when subprograms invoked from within subprograms (AIE)mid0402b check no information is added when LIST => NOXREF (AIE)mid0402c when subprograms and operations invoked within subprograms, operations and packages (AIE)mid0402d when invocations from a package specification and body CHECK ACROSS REFERENCE INFORMATION TO si_external_refs ATTRIBUTE OF A COMPILATION UNIT NODE (AIE)mid0403a using a procedure compilation unit (AIE)mid0403b verify not si_external refs added if LIST => NOXREF (AIE)mid0403c using package compilation units (AIE)mid0403d using subunit compilation unit (AIE)mid0403e using a function body compilation unit (AIE)mid0403f using a procedure specification compilation unit CHECK CROSS REFERENCE INFORMATION TO si_global_refs ATTRIBUTE OF A BLOCK OR BODY NODE (AIE)mid0404a using a procedure body (AIE)mid0404b verify no information idded if LIST => NOXREF (AIE)mid0404c using a function body (AIE)mid0404d using a block statement (AIE)mid0404e using a package specification and body VERIFY VALUE GIVEN TO THE si_labeled ATTRIBUTE OF STATEMENTS (AIE)mid0405a using assignment, goto and null statements (AIE)mid0405b using if, loop and exit statements (AIE)mid0405c using block and case statements (AIE)mid0405d using procedure calls and return statements VERIFY VALUE GIVEN TO THE si_context ATTRIBUTE OF NAME_EXP NODES (AIE)mid0406a those that should receive ADDRESS_CONTEXT value (AIE)mid0406b those that should receive FLOW_CONTEXT value (AIE)mid0406c those that should receive PARAMETER_CONTEXT value (AIE)mid0406d those that should receive VALUE_CONTEXT value VERIFY VALUE GIVEN TO THE si_opt_level ATTRIBUTE OF A BODY OR BLOCK (AIE)mid0501a using procedure body without a pragma (OPTIMIZE=>NONE) (AIE)mid0501b using procedure body without a pragma (OPTIMIZE=>TIME) (AIE)mid0501c using procedure body without a pragma (OPTIMIZE=>SPACE) (AIE)mid0502a using procedure body with pragma OPTIMIZE(SPACE) (OPTIMIZE=>TIME) - 25 - (AIE)mid0502b using block statement with nested pragmas OPTIMIZE(SPACE) and OPTIMIZE(TIME) (OPTIMIZE => NONE) (AIE)mid0502c using package spec and body with pragma OPTIMIZE(TIME) (OPTIMIZE=>SPACE) (AIE)mid0502d using nested block statements with pragma OPTIMIZE(SPACE) (OPTIMIZE=>NONE) CHECK IMPROPER USES OF PRAGMA OPTIMIZE (AIE)mid0503a using second pragma OPTIMIZE in a declarative part (OPTIMIZE=>NONE) (AIE)mid0503b pragma OPTIMIZE cannot be placed in package specification VERIFY THE AIE DEFINED PRAGMA STATIC (AIE)mid0504a verify call frame for each of subprogram arguments is allocated static storage (AIE)mid0504b verify error when there is dynamically sized local VERIFY PRAGMA INLINE (AIE)mid0505a verifies subprogram bodies are expanded inline VERIFY DECISIONS MADE WHEN CHOOSING A LAYOUT REPRESENTATION FOR STORAGE (AIE)mid0601a predefined type INTEGER given a layout of one word (AIE)mid0601b predefined type SMALL_INTEGER given a layout of 1/2 word (AIE)mid0601c verify user defined integer types given smallest layout (AIE)mid0602a predefined type FLOAT is represented as single word (AIE)mid0602b predefined type LONG_FLOAT is represented as single word (AIE)mid0602c user defined floating point types are given smallest layout (AIE)mid0602d error issued when accuracy requested is too precise for implementation (AIE)mid0603a fixed point types represented as single word (AIE)mid0603b or issued when accuracy requested is too precise for implementation (AIE)mid0604a object represented with 3 bits is given 3 bits in packed record (AIE)mid0604b object represented with 3 bits is given a byte in an unpacked array (AIE)mid0604c object represented with 3 bits is given 1/2 word as a local ojbect to a subprogram (AIE)mid0605a statically sized components are stored in the beginning of record and dynamically sized components at end with pointer from beginning (AIE)mid0606a representations are as small as possible in packed array (AIE)mid0606b representations are expanded to byte to ease addressing in unpacked array (AIE)mid0607a storage for variant parts of records is overlaid (AIE)mid0607b storage for disjoint blocks within subpgm bodies is overlayed (AIE)mid0608a verify allocation on secondary stack for dynamic sized arrays not in records (AIE)mid0609a checks length clauses for numeric type (rep specs) (AIE)mid0609b checks length clauses for access type (rep specs) (AIE)mid0609c checks length cluases for task type (rep specs) (AIE)mid0610a checks record rep clause that specifies the packing - 26 - in 2 words VERIFY STATICALLY SIZED OBJECTS SIZED <= 64 BITS ARE USED AS FORMAL PARAMS - ACTUALS PASSED BY VALUE (AIE)mid0701a checks parameters of procedures (AIE)mid0701b checks parameters and return values of functions VERIFY STATICALLY SIZED OBJECTS SIZED >= 64 BITS ARE USED AS FORMAL PARAMS - ACTUALS PASSED BY VALUE (AIE)mid0702a checks parameters of procedures (AIE)mid0702b checks parameters and return values of functions VERIFY CORRECT PARAMETER PASSING METHODS ARE SELECTED (AIE)mid0703a checks parameters of procedures (AIE)mid0703b checks parameters and return values of functions VERIFY DYNAMICALLY SIZED OBJECTS USED AS FORMAL PARAMS PASSED BY REFERENCE (AIE)mid0704a checks parameters of procedures (AIE)mid0704b checks parameters and return values of functions - 27 - AIE BACK-END TESTS CHECK COMPILER OPTIONS ben0101a verify memory size with OPTIMIZE => SPACE ben0102a verify cpu time with OPTIMIZE => TIME ben0201a check FLOW with OPTIMIZE => NONE ben0202a check strength reduction and code motion when OPTIMIZE => SPACE CHECK FLOW AND WALKS OF BILL TREE ben0203a with LOOP and no subprogram calls beno2o3b with GOTO and no loops or subprogram calls beno203c strength reduction unit processing ben0203d with subprogram call and INLINE pragma ben0203e with subprogram call, subprogram defined in program ben0203f with subprogram call, subprogram declared in package ben0203g with subprogram call, subprogram in library unit ben0203h with subprogram call, recursive subprogram in program ben0203i with subprogram call, recursive subprogram in package ben0203j with subprogram call, recursive subprogram in library ben0203k with GOTO, no loops or subprograms, detection of common subexpressions ben0204a with units eligible for strength reduction when OPTIMIZE => TIME ben0205a with straightline program CHECK CONSTANT PRAPAGATION WITH EXPRESSION SIMPLIFICATION ben0301a when constant is a named number ben0301b when constant is a true constant CHECK FOLDING IMPLEMENTATION BY FLOW ben0302a where expression has binary operation and integer operands ben0302b where expression has unary operation and integer operands ben0302c where expression has deeply nested unary operations ben0302d where boolean expression has unary or binary operations and boolean operands ben0302e where constants are adjacent and where they are dispersed CHECK TREATMENT OF EXPRESSIONS AS COMMON SUBEXPRESSIONS ben0303a with binary expressions and no change of any variable ben0303b with binary expressions and a change of some variable ben0303c with components of an array ben0303d with binary expressions, variable changed in IF and CASE ben0303e with subexpresions permuted or muddled by parentheses ben0303f with array offset ben0303g with array offset and value assigned object ben0303h with arguments to subtype conversion functions ben0303i with arguements to type conversion functions ben0303j with labelled statement to GOTO ben0303k with labelled statement with no GOTO ben0303l with nested levels of declaration of subexpressions ben0303m with record components CHECK EXPRESSION SIMPLIFICATION ben0304a with integer , no side effect as a sub_operand ben0304b with integer, with side effect as an operand - 28 - ben0304c with floating point, with side effect as an sub_operand ben0304d with floating point, no side effect as a sub_operand ben0304e with boolean, no side effect as a sub_operand ben0304f with boolean, with side effect as a sub_operand ben0304g with cancelling of subexpressions CHECK STRENGTH REDUCTION ben0401a with FOR loop ben0401b with FOR loop, elements not in usual arrangement ben0402a with invariant code movement CHECK RANGE INFORMATION AND UNNECESSARY CONSTRAINT CHECKS ben0501a at END IF statement ben0501b at END CASE statement ben0501c with BACK END check range information ben0501d with BACK END check access checks CHECK REMOVAL OF UNREACHABLE CODE ben0502a by FINAL with 'if-else-then' with constant propagation ben0502b by FINAL with 'if-else-then' with range information ben0502c by FLOW with LOOPS, condition always false or range null ben0502d by FINAL with CASE, with constant propagation ben0502e with GOTO ben0502f between RAISE, with unhandled exception ben0502g between RAISE, with handled exception ben0502h by FLOW in CASE, where alternative chosen by null range ben0502i by FLOW in CASE, don't need OTHERS choice ben0503a nested GOTOs changed to one GOTO CHECK "CROSS-JUMPING" WITH COMPILER OPTION, OPTIMIZE => SPACE ben0504a at end of THEN, ELSE ben0504b at end of THEN, ELSIF, ELSE ben0504c at end of CASE ben0504d at end of CASE where several sets of duplicate code ben0504e at end of THEN, ELSE where code semantically the same CHECK PRAGMA INLINE SUBPROGRAM ben0505a instead of compilation unit ben0505b in declaration of space optimized subprograms CHECK BACK END USE OF LOCAL SPILL ben0601a with integer expression that requires 16-plus registers ben0601b with floating . expression that requires 4-plus registers ben0601c with complex expression that requires 16-plus registers ben0602a with simple expression CHECK COMPILER BACK END LIMITATIONS ben0701a verify speed of compilation ben0702a verify size of compiler ben0703a verify maximum size of BILL node - 29 -