Test Audit app

The Test Audit apps is provided with a set of most-at-risk objects obtained from the most recent impact analysis and retrieves usage data for them from the specified QA and Production systems. Most-at-risk tokens are used to retrieve matching test executions from the specified Tosca, qTest or ALM Test Repository, and the app reports the run status of each test execution.

In addition to the most-at-risk objects, the Test Audit app identifies all objects that were impacted by a change; these are impactful changes. The app reports on the test execution status for each impactful change.

Prerequisites

The Test Audit app uses a pipeline to identify the QA and Production systems used for the analysis, and the Test Repository to be searched for test assets that match the most-at-risk objects. Before you begin, create a pipeline and set its RFC Destinations and Test Repositories as follows:

  • Set the Comparison System to the RFC Destination for your QA system.

  • Set the Usage System to the RFC Destination for your Production system.

  • Set the Most-at-risk Hits Execution Test Repositories to one or more Tosca, qTest or ALM Test Repositories. The app will search these to find test executions that match the most-at-risk objects.

A user with LiveCompare Editor permissions should download performance history data for the Usage System, and daily usage audit and SC_MON data for the Comparison system. In each case, specify a collection schedule, select Save and then select Update Cache.

A user with LiveCompare Editor permissions should run the Create Test Repository workflow from the Prerequisites template folder with the Pipeline that the Test Audit app will use.

DevOps categories

Testing.

Run the app

To run the Test Audit app, select the app from the Apps screen and create an app variant. Complete the Variant screen as follows:

  • Select the Pipeline that the Test Audit app will use.

  • Set the Start Date control to the start date for which the app will retrieve SC_MON data from the Pipeline’s Comparison System.

  • Set the End Date control to the end date for which the app will retrieve SC_MON data from the Pipeline’s Comparison System.

Click Run. When the variant has completed, you can access its results from the App Cockpit screen.

App results

The Test Audit app generates a Dashboard report which includes the following charts:

  • The Most-at-Risk Execution pie chart summarizes the test execution status for the most-at-risk objects.

  • The Impactful Changes Execution pie chart summarizes the test execution status for all objects impacted by a changing object. Run indicates that at least one test was run for the impactful change; Not Run indicates that no tests were run for the impactful change; Exceptions are impactful changes that could not be checked using the available SC_MON data retrieved from the Pipeline’s Comparison System.

  • The Test Execution bar chart summarizes the test executions in Pipelines Most-at-risk execution Test Repositories according to their execution status.

  • The Testing Date Range panel displays the date range for which the app tried to receive SC_MON data.

  • The QAS System panel displays the name of the Pipeline’s Comparison System.

  • The PRD System panel displays the name of the Pipeline’s Usage System.

  • The Test Repositories panel lists the Most-at-Risk Hits Execution Test Repositories in the specified Pipeline.

The Test Audit app also generates the following reports:

Treemap

The Treemap report displays the tokens that were matched in the specified Most-at-risk Hits Execution Test Repository, and the status of their associated tests, either ‘Not Run, ‘Failed’ or ‘Passed’. Note that the app gives tokens with a usage count of 0 a usage count of 1 so they appear in the treemap. The treemap groups tokens by Application Area.

Test Audit Details

The Test Audit Details Excel report includes the following spreadsheets:

Dashboard

This spreadsheet includes the Dashboard charts listed here.

Test Audit Details

This spreadsheet displays usage data for the Most-at-risk objects, obtained from the Comparison and Usage Systems. The app populates the BUSINESS_CRITICAL column using the Business Critical Objects External Data Source.

  • The APP_AREA column stores the application area for a transaction or program.

  • The TOKEN column stores the name of a most-at-risk transaction, program, or other executable object

  • The DESCRIPTION column stores the description for the object.

  • The QA_STATUS column is set to ‘Run’ if a user has run the object on the Pipeline’s Comparison System.

  • The QA_COUNT column stores the usage count for the object, obtained from the daily usage audit data retrieved for the Pipeline’s Comparison System.

  • The PRD_COUNT column stores the count for the object, obtained from the performance history data retrieved for the Pipeline’s Usage System.

  • The PRD_USERS column stores the number of users who have run the object, obtained from the performance history data retrieved for the Pipeline’s Usage System.

  • The PRD_TOP_USER column stores user who has run the object the most.

  • The PRD_TOP_USAGE column stores usage count of the object for the top user.

  • The CUSTOM column is set to Yes if the object is custom.

  • The BUSINESS_CRITICAL column is set to Yes if the object is business critical.

  • The DEVCLASS column stores the development class to which the object belongs.

Test Execution Details

This spreadsheet displays details for tests stored in the Pipeline’s Most-at-risk Hits Execution Test Repositories.

  • The TEST_REPOSITORY column stores the Test Repository name.

  • The TEST_REPOSITORY_TYPE column stores the Test Repository type.

  • The NAME column stores the name of a most-at-risk object that has an associated test.

  • The TEST_STATUS column stores the test’s status.

  • The LAST_RUN_DATETIME column stores the test’s last start time.

  • The TEST_NAME column stores the test’s name.

  • The TEST_PATH column stores the test’s path.

  • The TEST_ID column stores the test’s identifier.

Impactful Changes – Run

This spreadsheet lists all the impacted objects that have had at least one test run for them in any of the Pipeline’s Most-at-risk Hits Execution Test Repositories.

Impactful Changes – Not Run

This spreadsheet lists all the impacted objects that have not had a test run for them in any of the Pipeline’s Most-at-risk Hits Execution Test Repositories.

Impactful Changes – Exceptions

This spreadsheet lists all the impacted objects that could not be checked using the available SC_MON data retrieved from the Pipeline’s Comparison System.

Analysis Input Data

This Excel report contains a copy of the input parameters used to produce the app’s Dashboard report. The value of each input parameter is stored in a separate worksheet, which is named after the parameter whose value it contains.

Related topics

Standard apps