Work With Test Runs

The Work With Test Runs display can be accessed either by selecting the option for ResultsGwhich shows the same information but limited to Runs initiated by the current user id.

Warn The number of warning messages that have been generated for this Test Run, see below for further information.

Run One of the following Test Run status codes will be displayed.

TB-Error A TestBench error has occurred during the initiation of the test – refer to the job log.

Halted A script has been replayed and there is no matching Sub Run, usually due to multiple executions of the script; An error was encountered during playback such as ‘Cursor in protected area of display’ which stopped the script; F3 was pressed on the initial screen to cancel playback; One of the script error counts were exceeded.

Stopped Other general script errors; an exception message occurred and as a result the user cancelled the program under test; any of the Sub Runs are at ‘Crashed’ status.

Abnormal Any of the Sub Runs are at ‘Stopped’ or ‘Cancelled’ status.

Cancelled Option ‘4’ has been keyed against a Test Run to cancel it. This does not necessarily mean that the job running in the system has been cancelled, only that the processing within TestBench was cancelled. See the notes about using option 4 below.

Queued The Test Run is waiting on a job queue.

Running The Test Run is in progress.

Complete The Test Run has finished successfully and none of the above is true.

Pass The Pass column is populated with the status value of the Test Run which is set on the Pass/Fail screen (option 3). See below for more details.

Clr? This indicates whether low level details of the test run have been cleared with option 7. If cleared, some information about the test run is deleted. See the notes below for option 7 – Clear/Delete.

Options

1 – Warnings TestBench will automatically generate warning messages for:-

• Messages in joblog greater than designated severity and not covered by joblog exceptions
• Returned Parameters different to expected values
• Actual screen images different to those expected if a script was played using TestDrive or ‘Native’ Record and Playback
• One or more Data Rule Failures
• Differences in results comparison
• Physical file not protected due to multiple library references
• Logical file could not be re-created in test library
• Dynamic data protection is not valid when protecting files with constraints
• TestBench error occurred during job initiation, execution or clean up. Refer to the job log to determine the cause
• Test Case Process errors
• User Exit program errors
• Unable to save data at the end of a Test Run
• ENDCMTCTL being successfully executed at the end of a Test Run if STRCMTCTL was issued as part of the Test Run

1 – Warnings Key 1 against a Test Run for a summary of the warnings generated during the test run. Warnings can also be accessed from the Sub Run details display.

3 – Pass/Fail Key 3 to retrieve the list of Test Items selected at the start of the run or added during the run, and where the run was on Version 4.1 or prior of TestBench the scores entered so far. Where the Test Run has been performed on TestBench 4.2 or higher then the Test Items will be displayed along with the option to view and update the pass/fail status of both the individual elements of the run and the Test Run as a whole. See the two sections below on Test Run Score Sheet and Test Run Pass/Fail for more information.

4 – Cancel This option will cancel an active batch job or a queued batch job. Using 4 will end that job, and update the status of the run inside TestBench. If used on an active interactive job it will have no effect and the job will continue. However, if the interactive job has been ended by a command such as ENDJOB *IMMED, TestBench will believe the run is still active until this option is used to update the status to cancelled. If an interactive job is cancelled by using System Request option 2, TestBench will handle the request and carry out its normal post run tidy up activities, before flagging the run as ended abnormally.

5 – Details To display details of the Test Run and its sub runs. See the section below for more information.

6 – Report_IT Produces the Test Run report for the selected Test Run. You can choose what level of information is to be printed. See the separate section on Report_IT.

7 – Clear/Delete A window is displayed where the user can choose to either delete or clear the Test Runs that have been selected with option ‘7’. The ‘Delete’ option will delete permanently all references to the Test Run from inside TestBench. The ‘Clear’ option will delete some of the information associated with a Test Run, leaving a reduced record that the run took place. This option will permanently remove job log messages, file and data area effects, parameter values, and data rules associated with the Test Run. It will also attempt to delete the temporary run-time test library if it still exists.

An alternative to this option is to Purge Data Runs using the option from the Main Menu, see System chapter for more information.

8 – Job This displays the OS/400 job information, either for an active batch job (a submitted test run), or an active interactive job which started the Test Run. Once the job has terminated, this option is no longer allowed.

9 – Notes Use option 9 to access the Notes Editor and to enter or amend text notes about the Test Run. These notes can be accessed during an interactive Test Run, and are printed on the Test Report document by Report_IT. For more information on using the Notes Editor, refer to the separate Chapter in this document.

13 – Timeline Option 13 will display a chronological view of selected data logged in the results. This intersperses captured information such as joblog entries, 5250 screen pictures, database updates etc into a single view in date and time sequence.

14 – Compare Run a results comparison between this Test Run and another Run within this Test Case. See the Results Comparison section for more information.

15 – Summary View the results of a previously executed comparison between this Test Run and another Run within this Test Case. See the Results Comparison section for more information.
Function Keys

F8 – Expand/Drop Expand the single line display to two lines showing Test Run description and Script name if applicable.

F10 – Baselines Work with baseline Test Runs that are used when performing results comparisons. This option is only available when viewing results for a specific Test Case and is hidden from view when this screen has been accessed using option ‘21’ from the Main Menu. See the specific section for more information.

Test Run Score Sheet
For Test Runs carried out on version 4.1 of TestBench and earlier the display for the score sheet will be as below. However for runs carried out on TestBench 4.2 or higher see the following section.

Score 0 = Fail/not tested (0%)
5 = Successful test (100%)
Use values in between to indicate different degrees of errors/success if required.

Comment Brief comments if required to clarify each scored item. Use F14 for additional text notes.

New Test Items can be added to the Score Sheet after the Test Run has completed.

Function Keys

F14 – Run Notes Additional text notes can be recorded with the Notes Editor against the Test Run by pressing F14. These can appear on the Test Report document. For more information on the Notes Editor, refer to the separate Chapter in this document

Test Run Pass/Fail
For Test Runs carried out on version 4.2 of TestBench or higher the following screen will be displayed on taking option 3 from the Work With Test Runs display.

Status Enter the status for the Test Run from the available valid selection (0=None, 1=Fail and 2=Pass). Note that the selection must be different from the current status.

Notes Enter a note to describe the Test Run. Note that this is a mandatory field.

Select Option Enter a valid option against a Test Item from those displayed. Entering a ‘6’ or ‘7’ will populate the row with either a pass or fail. Choosing ‘2’ will allow you to change the status and details of the Test Item – see screenshot below. Choosing option 5 will show a window which details the settings for that Test Item. Note this is only an available option against System defined Test Items.

Function Keys

F6-Add Add a new Test Item.

F10 – History This will display the Pass/Fail historical log for the Test Run. See below for more information.

F14 – Warnings Displays the warnings associated with the Test Run. For more details of ‘Warnings’ see the Results chapter.

F21 – Display Run Displays the Test Run Details screen. See Results chapter for further information.

Test Item Maintenance
The following screen is accessed when either option ‘2’ to change or F6 to add a Test Item are keyed on the Test Run Pass/Fail screen.

Test Item This field is mandatory and describes the testing task.

Comment An optional field which can be used to provide a reason for passing or failing a Test Item.

Status Enter the status for the Test Item from the available valid selection (0=None, 1=Fail and 2=Pass).

Test Run Status History
Access this screen with an F10 from the Test Run Pass/Fail display. It shows an audit trail of changes to the Test Run status, however updates to the individual Test Items are not included.

5-Display Notes Displays the full comments that were keyed into the notes field when this status change was made.

Test Run Details
The following display can be accessed from the Work With Test Runs display by using option 5.

The top half of the screen shows general information about the Test Run, whilst the lower portion lists each test Process represented by a Sub Run. There may be more than one Sub Run for a Process where a Test Case uses incremental job dates. Results for each Sub Run are kept separately inside TestBench. If submitted jobs are being tracked for a Process, they will appear on a separate indented line below the Sub Run from which they were initiated with a unique Submit ID.

Status Many of the Sub Run status codes are the same as those that are logged against the Test Run level and a full explanation of these can be found in the earlier section. The following codes however have a different meaning.

Stopped A job log message exceeded the maximum severity logged against the Test Case and the option to stop the Test Run was also selected. This causes a Test Run status of ‘Abnormal’.

Errors A job log message exceeded the maximum severity logged against the Test Case and the option to continue the test run was also selected. This causes a Test Run status of ‘Abnormal’.

Crashed The main Sub Run program call failed. This causes a Test Run status of ‘Stopped’.

Options

1 – Warnings TestBench will automatically generate warning messages for:-

• Messages in joblog greater than designated severity and not covered by joblog exceptions
• Returned Parameters different to expected values
• Actual screen images different to those expected if a script was played using TestDrive or ‘Native’ Record and Playback
• One or more Data Rule Failures
• Differences in results comparison
• Physical file not protected due to multiple library references
• Logical file could not be re-created in test library
• Dynamic data protection is not valid when protecting files with constraints.
• TestBench error occurred during job initiation, execution or clean up. Refer to the job log to determine the cause
• Test Case Process errors
• User Exit program errors
• Unable to save data at the end of a Test Run
• ENDCMTCTL being successfully executed at the end of a Test Run if STRCMTCTL was issued as part of the Test Run

Key 1 against a Test run for a summary of the warnings generated during the test run. Warnings can also be accessed from the Work With Test Runs display.

2 – Joblog Any messages written to the job log during the Sub Run are captured by TestBench and stored in the database, provided the Test Option controlling this is activated. This means that they remain available until you choose to delete/clear them and they are limited to the messages from that portion of the test run. See the section below on the Display Sub Run Messages screen.

3 – Programs If the tested program has been called with parameters, these can be seen with this option. The parameter names are listed, together with the ending and expected values. Using F9 in the parameter window enables the initial and ending values to be compared. Also, parameter values for any programs for which Log Call Interception has been defined will be displayed here. In both views, differences are highlighted in green blocks.

4 – Rules Use option 4 to display details of Data Rules which apply to this Test Case (whether Global level, Project level or Test Case level) and whether the rules have been passed or failed. Failed rules will also generate a warning message from which the details can be viewed. For more information on Data Rules, see the later section.

5 – Info Option 5 will display a window of basic data about the selected sub run, such as the job number, start and end times, rule applicability. It also has an F4 prompt facility to display the full cmd construction which maybe abbreviated on the current display.

6 – DB Effects Files which have been protected from update by TestBench have each update captured and stored against the test run, provided the Test Option controlling this is activated. This comprises a before and after image of each record. Also, updates for any journals which have been specified in Manual References can be viewed here. Use option 6 to view these updates. Other files which have not been updated are not displayed. See the later section for more information and the relevant screens.

7 – Data Areas Any data areas (including the Local Data Area) which have been protected during the test run can have their before and after positions viewed. Just like file effects, this includes intermediate updates made to the data area, with the exception of the *LDA which only shows the initial and final values. Differences are indicated by an asterisk.

8 – Job This displays the OS/400 job information, either for an active batch job (a submitted test run), or an active interactive job which started the Test Run. Once the job has terminated, this option is no longer allowed.

9 – Screens The list of screens which were played as part of a ‘Native’ Record and Playback script is displayed.

10 – Data Queues Display all messages sent to and received from Data Queues during the Test, provided the Test Option controlling this is activated.

11 – MQ Queues Display certain messages sent to and received from MQ Queues during the Test if this function has been enabled. Refer to the separate TestMQ chapter for more information.

12 – Spool files Display all spooled files that have been captured for the test, if none exist display spooled files for the job.

13 – Timeline Display event information for the selected sub-run in chronological sequence. Refer to later section on Timeline display

F7 – Run Details This displays details about the Test Case definition at the time this run was initiated, and run time information such as Start and End Time. It also shows the Test Run Reference, a user defined field that is keyed at Test Case execution time, for example to link several Test Runs via a single change request number.

F9 – Notes Text notes can be recorded with the Notes Editor against the Test Run by pressing F9. These can appear on the Test Report document. For more information on the Notes Editor, refer to the separate Chapter in this document.

F11 – Lib List View the library list that was used by the Test Run. This is the library list that was set at the start of the test and does not reflect any changes made by the programs under test.

F14 – Pass/Fail Access the Test Run Pass/Fail display. See previous section for more information.

Display Joblog Messages
This display can be accessed from the Test Run Details display and shows the messages which TestBench has extracted from the job log for the Sub Run. TestBench has added initial and closing messages for identification purposes which are shown in green reverse image. The information is retained after the job log is cleared from the system. The content of the display can be altered using F13 by selecting or omitting types of messages, or searching by time zone. Messages are removed if the Test Case is cleared. It is possible to de-activate the storing of messages via a Test Case Option.

The warning severity specified for the Test Case is displayed at the top of the screen and any messages which exceed this severity and caused a warning message to be produced are highlighted. Those messages that were excluded using joblog exceptions will not be highlighted.

If Log Call Program Interception is being used for this Test Run, additional messages are generated when intercepted programs have been called. It is however possible to switch off these messages in System Values in order for a more representative job log to be obtained.

Options

5 – Display Key 5 against a message to retrieve the full standard description for the message.

7 – Parameters Key a 7 against the Sub Run Start or End message to display the parameter values for the Master program. If Log Call Interception is turned on and the messages have not been suppressed in System Values, option 7 against the logged message will display initial and returned parameter values for this call.

Function Keys

F8 – Expand/Drop To expand the single line display into a two line display showing information about the sending and receiving programs, and vice versa.

F13 – Options Use this function to change the sequence of the messages shown, or to select messages using the following options:-

• Minimum severity (defaults to the value defined for the Test Case)
• Message type (e.g. CPF, MCH, RPG)
• Minimum message id number
• Maximum message id number
• Start time. Hours, minutes, seconds
• End time. Hours, minutes, seconds
• Sending program
• Receiving program

Programs (Tracked Programs)
This display can be accessed from the Test Run Details screen with option ‘3’. If the tested program has been called with parameters, these can be seen with this option. Also, parameter values for any programs for which Log Call Interception has been defined will be displayed here. For more information on viewing parameters of called programs, see the section on Program Interception and how to set this up in a Test Case.

Program The name of the program being executed or intercepted.

Call No This increments for each execution within program name.

Type ‘Master’ indicates that this is the program being executed by the Test Case Process. ‘Intercept’ is shown for programs set up for Log Call Interception using F8 in Test Case Maintenance.

Options

5 – Call Details Provides further information about the program execution including start and end date and time, CPU and auxiliary IO.

7 – Parameters For the master program the start/end or end/expected parameter values can be displayed. For interception programs this will display initial and returned parameter values for this call.

Available Data Rules
This screen is accessed by taking option ‘4’ for Rules from the Sub Run details display. It shows each Data Rule applicable to the Test Run, and whether this Rule has been passed or failed.

Data Rules can be defined for Physical Files and Data Areas, and can be set up at either the Global, Project or Test Case level. Unless suppressed for the run, Project level rules are checked at the end of execution of each Test Case in that Project and Global rules are checked for every Test in every project. Data rules only apply to files for which file (DB) effects have been generated, or for Data Areas which have been included in run-time library Data Protection (*PGM, *PGMREF, *DYNAMIC or manually included).

Refer to the separate section on Data Rules in the Testing chapter for more information on how to use Data Rules.

Options

5 – Display Failures Display Record Failures for files (see screen below). For Data Areas the Rule and data values are shown.

6 – Display Rule Shows the Data Rule description and values.

7 – Re-run Rules Use this option to re-check the data captured in the Test Run against the current version of the Data Rule. This means that if a Data Rule is incorrect, previous test runs do not need to be repeated.

Function Keys

F13 – Repeat Duplicate the first option on the screen to the end of the display.

F15 – Re-run Rules Re-check the data captured in the Test Run against the current version of all Data Rules. This will include any new rules which have been added.

F18 – Filter Display a window which enables the view to be tailored to specific rules only.

Level Choose between all rules (default), Global, Project or Test Case rules.

Utilisation Choose between all rules (default), those which have been applied to this Test Run or those which have not been applied. A rule is only applied to a Test Run if the file over which it is built has been updated during the test.

NOTE: Results created prior to version 3.2 of TestBench for Project level Data Rules will list the Rule Level as Global due to the introduction of Global level rules in this version.

Display Record Failures
This screen is accessed by option 5 from the Available Data Rules screen. After a Test run, TestBench captures an image of each update on a protected file or data area, which is then stored inside TestBench. These are compared against the Data Rules at the end of the run.

Options

5 – Display Data Key 5 against a record to view a before and after image of the update, with changed fields highlighted.

6 – Display rule Key 6 to view the Data Rule and its values, side by side with the actual values from the file.

7 – Track RRN Use this option to analyse all the updates to an individual file record (Relative Record Number), bringing these together on the screen.

Function Keys

F13 – Repeat Duplicate the first option on the screen to the end of the display.

View Data Rule Details
This display can be accessed from viewing Warnings or viewing Rules in Results. The Rule values and the actual data values are shown side by side. The data that does not comply with the required Rule value is highlighted. If Alternative File Description fields were used as part of the Data Rule definition, these will also be shown here.

In the case of test operations such as ‘EXISTS’ where there is no real data value to return, the result display will show a 0 if the test has passed, and a value of 1 if the test has failed.

View Field Information
This screen is accessed from a number of places in TestBench to display results of updates, adds, and delete operations to files which were analysed during a test run. Differences between the two situations are shown as highlighted field values. Leading zeros are suppressed from numeric fields for clarity. If an Alternative File Description exists for the file in either the library from which it originated or *LIBL, the results for these additional fields will also be displayed.

Function Keys

F4 – Extend If the field you wish to view is longer than the available space on the screen, move the cursor to that field and press F4 to extend the view. This will also compare before and after images, identifying changes at the byte level.

F6 – Print Produce a formatted report of the detailed file effect.

F7 – Description Use F6 to view the File description to check field names, lengths and types.

F8 – Changed Fields For database updates display only those fields which are different on the Before and After image. F8 toggles between changes only or all fields.

F9 – Hex/Char Use F9 to alternate between viewing the data values in character or hexadecimal format. The view always starts in character format.

Display Statistical Information – File and Data Area Effects
This display is accessed from the Test Run Details display by keying either option 6 or option 7 against a Sub Run. It shows all files or data areas that have been protected during a Test Run and updated, and updates for any journals specified in Manual References, with options to view the details of each update. It is possible to de-activate the storing of database updates via a Test Case Option. The layout of the following screen differs slightly depending on whether files or data areas are being viewed.

Options

1 – Explode To expand the analysis to the program level, with the ability to select updates generated only by each program.

5 – Display Details To drill down to view a list of updated records for the selected object.

6 – Rules Available Data Rules for the object (see previous section).

7 – Re-run Rules Check the data captured in the Test Run against the current version of all Data Rules for the object, including any which have been added to the Test Case or Project.

Function Keys

F8 – Expand/Fold Toggle between the standard view and an expanded view which includes commitment control totals.

F9 – Program Use this option to display a summary of the physical write and update operations to the protected objects, by program.

F13 – Repeat Duplicate the first option on the screen to the end of the display.

F14 – All Rather than using option 5 to display a list of updated records for each file or data area, F14 shows the same information but includes all updates to all objectss in chronological order.

F15 – Re-run Rules Re-check the data captured in the Test Run against the current version of all Data Rules. This will include any new rules which have been added.

Display File Information
This display is reached via either option ‘5’ or F14 on the previous Display Statistical Information screen. It lists for a selected file or data area, or all files or data areas affected in a Test Run, the images of writes, updates and deletes. It also shows for each operation the Rule status of Pass, Fail or N/A (no rule for this object).

This display may also be accessed from the timeline view of results when a number of record updates have been summarised in the timeline display. In this case, the individual updates will have an asterisk ‘*’ next to the object name to indicate each individual file effect which is summarised in the timeline entry which was selected.

Rules The summary status of any rules which have been applied to this record are indicated in the Rule column. The values and meanings are:

No chk The data has not been checked against any rules. There are no rules which apply to record, either because there are no rules, the ‘when’ selection conditions do not apply, the operation is a delete, or the intermediate updates are not being checked and this is not a final update to this record.

Fail One or more of the rules applicable have failed when compared to the data vales in the record after the operation.

Pass The data values comply with the conditions in all applicable rules.

Options

5 – View Key 5 against a record to view a before and after image of the update, with changed fields highlighted if this is a file or indicated with an asterisk for data areas. This screen is described earlier under the heading View Field Information.

6 – Rules To view the Data Rules applicable to this file record and their pass/fail status, with the option to view Rule values and data side by side.

7 – Track RRN Use this option to analyse all the updates to an individual file record (Relative Record Number), bringing these together on the screen.
Function Keys

F6/F7/F8–Search Depending upon the screen view, function keys F6, F7 and F8 will enable a File, RRN or Program search. All three function keys will only be available if the screen is showing all file effects and the view has not been limited. The value for the search is entered in the Find field at the top of the display.
Enter a file, RRN or program in the Find prompt and press the relevant function key to display the next occurrence of the object or record at the top of the screen. Alternatively, place the cursor on the line which contains the file, RRN or program you wish to search for and press the relevant function key.

F13 – Repeat Duplicate the first option on the screen to the end of the display.

F14 – Pick Fields This enables the display to be replaced with the data values from selected fields in the file being viewed. This means that all the updates on a field can be viewed concurrently, rather than using option 5 to view each individually. This option is not available when viewing updates for all files.

F15 – Data/Core View Toggle between the core view of the screen above and the data view, which shows the value contents of each of the fields, comma separated across the screen. F19/F20 become available to scroll the screen left or right so that the full data content of the record can be viewed.

F17 – Top Display the first page of updates.

F18 – Bottom Display the last page of updates.

F19 – Left This function key is only enabled after F15 has been pressed for the Data view. Scroll the screen to the left to view the values of the fields higher up in the record layout.

F20 – Right This function key is only enabled after F15 has been pressed for the Data view. Scroll the screen to the right to view the values of the fields lower down in the record layout.

F21 – Search Display the Data Values Search screen which enables a specific value for one or more fields on the file to be located. Once the search criteria have been entered, the above screen is re-displayed with the cursor located on the first matching record. If no matches exist an error message is displayed. See later section for more information.

F22 – Search Again This key is only valid after F21 has been used to enter the search criteria and locate the first matching record. F22 repeats the search and places the cursor on the next record in the list which matches the criteria. You can continue to press F22 until no further matches are found, at which point an error message is displayed.

Data View
The following display is accessed by pressing F15 on the above screen when on the Core View. It shows the data values for all of the fields for every record, separated by commas, or for all data areas. Use F19/F20 to scroll left and right.

Pick Fields to Display
This display is obtained by pressing F14 – Pick Fields from the standard File Information Display when viewing updates for a single file only. It can be used when you wish to view the content values of one of more fields across all records for the file.

1 – Select Every field on the file is listed. Key an option ‘1’ next to each field for which you would like to view the data values.

In the example above, the Amount and Narrative fields have been selected from the file, and when enter is pressed the data in these fields is shown in columns on the following display for all the affected records.

The same Function key options apply on this display as the previous example. See above.

Data Values Search
Keying F21 on the Display File Information screen enables a specific value for one or more fields on a given file to be located. If F21 was pressed while viewing database updates for all files you will be required to key in the name of the file to be searched (use F4 to select from a list of files updated during this test), however if viewing a single file only this information is defaulted.

Position To Key in the name of the field for which you wish to specify a value and press enter to position the cursor to that field. This is to assist with locating fields on files with very long record lengths.

Value Key in the formatted value for which you wish to search. For example, fields with decimal places must be keyed with the decimal point and alpha fields are case sensitive. The special value of *blank enables empty alpha fields to be located.

Once the required values have been keyed in, press enter to display the Data View of the Display File Information screen with the cursor positioned to the first record which matches the search criteria. Press F22 to repeat the search and locate the next matching record. If no matches are found an error message is displayed.

Screen Results
This screen can be accessed by entering Option 9 against the Sub Run on the Test Run Details display.

The screens which were actually displayed as part of the script are listed. If variable data has been used, transaction headers are also shown. The number of differences and also the response time is shown for each screen. This is the elapsed time between the action key being pressed on the previous screen and this screen being displayed.

Options

The following options are available for 5250 scripts replayed by Native R&P only. TestDrive results must be viewing using Qualify – Asset Explorer.

5 – Display View the actual screen image which was played as part of the script.

Function Keys

F8 – Expected View the screen images which were expected to appear. Therefore, if a script was halted before the end it is still possible to see a complete set of expected results. For scripts using variable data, this option will display the variable data field names as opposed to the data values.

Both the Expected and Actual versions of the screen image can be displayed, with the latter being the default. Any fields which are different will be displayed in reverse image.

Any fields being filled with variable data will show the actual value which was substituted from the variable data file, unless displaying Expected screens only when the variable data field name is shown.

Line 24 on the screen has been used to display the information needed to use the View Screen Results program. If any information is present on line 24 a ‘+’ sign is displayed in the reverse image bar, and it can be accessed by pressing Page Down. As well as showing the Function Keys available, line 24 also shows whether the Expected or Actual Results are being displayed. Also, the sequence number of this screen in the script and the total number of screens is shown.

The cursor is shown either in the initial position (expected and actual), or in the final position when viewing the input (keystrokes) display.

Keys

F3 – Exit Return to view screen list.

F8 – Actual/Expected Both the Actual and the Expected screen images can be displayed, and F8 is used to toggle between them. The Actual screen image is the default.

F9 – Keystrokes When looking at the Expected screen image, it is also possible to press F9 to view the keystrokes that were recorded as part of the script. The key used to exit from the screen (e.g. Enter, F3) can be seen in square brackets when looking at Keystrokes.

F12 – Back Display the previous screen in the script.

Enter – Next Display the next screen in the script.

PageDown Show lines 2 – 24 of the screen.

PageUp Show lines 1 – 23 of the screen.

Data Queue Messages
All messages sent to and received from data queues during the test can be intercepted by TestBench and captured in results. Access the following display with option 10 from Sub Run Detail. Data queue messages can also be protected, see the Testing chapter for more information.

A ‘Type’ of ‘Send’ on the above display refers to a message which was sent by a program to the listed data queue, and a ‘Type’ of ‘Recv’ details a message received by a program from the listed data queue.

Options

5 – Message An extended view of the complete message data.

6 – Data Queue View information about the data queue sending or receiving this message, including data queue name and library, description, maximum length and sequence.

7 – Key Data View the data that forms the key for this data queue message, if it is a keyed data queue.
Function Keys

F11 – View 2 Display data queue name and library, program name and the date and time of the entry in place of the data queue message.

Timeline
The Timeline display is designed to bring together key aspects of a set of test run, or sub run, results in chronological order. The information is the same data which is available under the separate headings of Job Log, DB Effects, Programs, Screens, Data Queues, MQ Queues and which can be accessed by the various options available in Test Run Details. It means that the overall sequence of processing and actions performed by the programs under test can be easily viewed to aid understanding and diagnostics.

The date and time from the iSeries event are used to sequence the entries in the Timeline. The time is extracted from the information stored by the operating system for each event and it should be understood that there are some differences in the level of detail stored by the system according to the type of event and the version/release of the operating system. In some cases this means that the logged time may be in whole seconds, in other cases it may be milli or micro seconds. Because of this, it may be possible that the Timeline display may have not been able to replicate the actual event sequence giving the time data available. The data is displayed accurately based on the information available, but there is a theoretical chance that there may be slight inaccuracies in the sequence because of the different levels of information available. The F11 key available on the display, shows the time stored when the event as logged and will indicate the level of accuracy available to determine the relative sequence of the events.

Data included in the Timeline display is:-

• The call of a program or command in a Test Case process.
• A job submission when the track submitted jobs feature is active.
• A program call to a tracked program (see program interception).
• Joblog entries.
• DB file effects (summarised by program and file/data area for consecutive blocks of operations).
• Screens (Native R&P playback).
• Data Queue message data.
• Websphere MQ message data (requires the TestMQ component).

The information may be accessed for a sub run or the test run overall.

Options

5 – Display Provide additional information about each type of entry. In the case of DB updates, the Display file information display for the designated program will be shown, and an asterisk (*) is present adjacent to each line in the results which is referred to from the Timeline display.

Function Keys

F11 – View 2 Display the Timeline screen with the available timestamp data for each event. This is the value which is used to determine the sequence of the displayed data. Depending on the type of entry, the time may be displayed in whole seconds, milli-seconds or micro-seconds.

Spool Files
An option against the Test Case controls whether or not reports generated as part of the test run will be captured and stored in the results database. If any reports have been captured, option 12 will provide the choice of viewing captured reports for the Sub Run or all spooled files for the job. If no reports have been captured, job spooled files are automatically shown.

4 – Delete Remove the captured report from the results database.

5 – Display View the captured report. A screen similar to the following one is displayed.

Test Run Summary
The following screen is displayed when an option ‘22’ is keyed on the Work with Test Cases display. The information displayed includes the average age in days of the Test Runs, percentage of the Test Runs sitting at ‘Complete’ status and average number of warnings.

Function Keys

F7 – Details Display the Work with Test Runs screen so that each Test Run can be interrogated in more detail.

F8 – Warnings Display the following screen which shows the total warnings for each set of Test Runs by type.

Test Run API
The TestBench API TBG969R can be used to retrieve information about a specific Test Run. Please refer to Appendix D for details about the parameter structure.

A user program can be written to execute the above API and perform processing dependant on the statistics returned. This user program can then be added as a Test Case User Exit of type ‘AN’ and automatically executed after a test has completed. User Exit substitution variables enable the Project, Test Case and Test Run number to be passed as parameters. An example of such a program is also shown in Appendix D.

This technique can therefore be used to integrate TestBench with a change management tool, enabling a program to be promoted if the Test Run is successful but triggering further action if warnings were produced.