Impact Analysis Results app

The Impact Analysis Results app allows you to review impactful objects and discover which executables they affect. You can then decide which executables belong in the test scope, select which tests to run, and decide what to do with executables that don't have any tests.

The app uses impact analysis data that the following apps and workflows record automatically:

The app stores the test scope and test selection decisions you’ve made for each analysis run. You can view the decisions directly in the app, or review them in a summary report.

The Impact Analysis Results app maintains a single set of scope and test selection decisions for all users.

Set test scope and decide which tests to run

The Impact Analysis Results app is available to users with LiveCompare Editor or Consumer permissions.

To set test scope and decide which tests to run, follow these steps:

  1. Start the app. To do so, select the Impact Analysis tool button. icon.

  2. The first screen displays all Pipelines you’ve used. To select an analysis to review, perform the following actions: 

    • Select an impact analysis Pipeline in the Pipelines section.

    • Select an analysis run In the Analyses - <Pipeline> section.

  3. The second screen summarizes the most-at-risk, impacted objects, not-impacted objects, and test reduction percentage for your selected analysis. Select one of the following review phases:

  4. Select Report to display the Test Planning Report for the selected analysis.

The screen’s header shows the following:

  • The analysis Pipeline.

  • The analysis date.

  • The number of executables with a scope decision in Phase 1 as a fraction of the total number of executables.

  • The number of tests with a run decision in Phase 2 as a fraction of the total number of tests.

Phase 1 – Test Scope Review

This phase lets you review used executables by their Application Area. You can review individual objects to understand how they affect each executable and tag the executables as in scope, out of scope, or undecided.

The Phase 1 – Test Scope Review has the following columns:

Column Description
Application Area SAP Application Area.
MAR The number of most-at-risk executables in the Application Area.
Impacted The number of impacted executables in this Application Area.
Scope Split A summary of the In Scope, Out of Scope and Undecided decisions made for the executables in this Application Area.
In The number of executables set as In Scope. When you open an analysis for the first time, the app sets all most-at-risk executables as In Scope.
Out The number of executables set as Out of Scope. When you open an analysis for the first time, the app sets all impacted and not impacted executables as Out of Scope.
Undecided The number of executables that don’t have an In Scope or Out of Scope decision.

Review each executable’s scope

Select a row in the Phase 1 –Test Scope Review screen to list the executables in the selected Application Area. You can filter the list to display combinations of the following:

  • Most-at-risk, Impacted, and Not impacted executables.

  • High, Medium, and Low Risk executables.

  • All, Standard Only, or Custom Only executables.

  • All executables, or Business Critical executables only.

The Phase 1 <Application Area> header has the following options:

  • ← APP_AREAs: Return to the Phase 1 – Test Scope Review screen.

  • Bulk: All MAR In Scope: Set all most-at-risk executables in the Application Area to In Scope.

  • Bulk: All Out of Scope: Set all executables in the Application Area to Out of Scope.

The Phase 1 <Application Area> screen lists the used executables in the selected Application Area. It has the following columns:

Column Description
Type The executable’s type.
Object Name The executable’s name.
Description The executable’s description.
Status The executable’s impact status: Most-at-risk, Impacted, or Not impacted.
Risk The executable’s risk: High, Medium, or Low.
Usage The executable’s usage count.
Business Critical This column has the value Y for business critical objects.
Custom This column has the value Y for custom objects.
Scope Decision Indicates whether the object is tagged as In Scope, Out of Scope, or ? (Undecided).
Notes A comment to support the scope decision.

We calculate values in the Risk column as follows:

First, we calculate a risk score for each application area in the range 0 to 1 based on the following:

Total objects = Most-at-risk + Impacted.

Fraction = Impacted / Total objects

Score = Fraction * Fraction

We then convert the score into three risk values as follows:

Score Risk value
Greater than or equal to 0.67 High
Between 0.66 and 0.33 Medium
Less than 0.33 Low

The score value measures this: For everything flagged in in this application area, which fraction are lower-priority impacted executables rather than higher-priority most-at-risk executables. We square the fraction value so that application areas with mostly impacted objects score more highly than application areas where there’s a more even mix of impacted and most-at-risk objects.

Application areas with an overwhelming number of impacted objects are high risk because any most-at-risk objects they contain may be easily overlooked.

Set executable scope

Set an executable’s scope in the Scope Decision column. You can also add a comment into the Notes column. The app saves your changes immediately.

Examine an executable’s impact details

To review a specific executable, select it from the Object Name column. This opens the executable's Details panel.

The panel's header shows the executable’s impact status, risk level, usage count, and whether the executable is Custom or Standard. You can make a scope decision for the executable, and add a note to justify your decision.

For Most-at-risk executables, the Details panel also has a section named Impactful Changes — why this executable is Most-at-risk. This section lists all impactful objects, which are all objects that impact the selected executable. You'll get the following details:

  • The search depth at which LiveCompare found the impactful object, its type and name, and its comparison status in the Pipeline’s analysis and comparison systems. Select the name to display the object’s comparison details.

  • The change that contained the impactful object. This can be a transport number, an SAP ChaRM request, or Object for individual objects specified in the impact analysis.

The Executables dropdown shows a count of the executables that depend on the impactful object. Expand the dropdown to list the objects and view their scope settings:

  • The In this area folder shows executables that are in the same Application Area as the currently selected executable.

  • The In other areas folder shows executables that are in a different Application Area.

For Most-at-risk executables, the Details panel also has a section named Nearest Change. This section shows the impactful object that’s closest to the selected executable in LiveCompare's dependency graph. You'll get the following details:

  • The search depth at which LiveCompare found the impactful object, its type and name, and its comparison status in the Pipeline’s Analysis and Comparison systems. Select the name link to display the object’s comparison details.

  • The change that contained the impactful object. Either a transport number, an SAP ChaRM request, or Object for individual objects specified in the impact analysis.

The Executables dropdown shows a count of the executables that depend on the impactful object. Expand the dropdown to list the objects and view their scope settings.

  • The In this area folder shows executables that are in the same Application Area as the currently selected executable.

  • The In other areas folder shows executables that are in a different Application Area.

The via marker names the intermediate object that connects the impactful object to the selected executable.

Phase 2 – Test Selection & Gaps

This phase lets you review the tests LiveCompare recommends for in-scope executables. You can review which tests to run and manage test gaps, which are executables with no test coverage. You can also plan to create tests for them, choose not to test them, or remove them from test scope.

The Phase 2 – Test Selection & Gaps screen groups the tests by their executable's Application Area. It has the following columns:

Column Description
Application Area The tested executable’s Application Area.
Optimal

The number of optimal or recommended tests for executables in the Application Area. Optimal tests are In Scope by default.
Covering The number of tests that exercise or cover executables in the Application Area. Covering tests are Out Of Scope by default.
Gaps The number of in scope executables in the Application Area that aren’t covered by any tests.
Split

A summary of your Run and Skip decisions for the tests.

  • For test hits, green indicates Run and red indicates Skip.

  • For test gaps, green indicates that the test has a Test, No Test or De-scope decision.

When you open an analysis for the first time, the app sets optimal tests to Run and covering tests to Skip.
Run The number of tests set to Run for executables in the Application Area.
Skip The number of tests set to Skip for executables in the Application Area.
Undecided The number of test gaps that don’t have a Test, No Test, or De-scope decision.

Review each executable’s test selections and gaps

Select a row in the Phase 2 – Test Selection & Gaps screen to display the Phase 2 <Application Area> screen. The upper part of the screen lists the used executables in the selected Application Area. It has the following columns:

Column Description
Type The type of the executable.
Object Name The executable’s name.
Description The executable’s description.
Status The executable’s impact status. Either Most-at-risk, Impacted, or Not impacted.
Risk The executable’s risk. Either High, Medium, or Low.
Usage The executable’s usage count.
Scope The executable’s scope decision, either In Scope or Out of Scope.
Optimal The number of optimal tests for the executable.
Covering The number of covering tests for the executable.

You can filter the list to display combinations of the following:

  • Most-at-risk, Impacted, and Not impacted executables.

  • High, Medium, and Low Risk executables.

The Phase 2 <Application Area> screen’s header has the following options:

  • ← APP_AREAs: Return to the Phase 2 - Test Selection & Gaps screen.

  • Export CSV: Export the screen’s contents to a .csv file.

The lower part of the screen lists all test hits and gaps for the first executable in the list. The app displays this executable in the Filtered field.

  • Select a row to display the test hits and test gaps for the executable in that row.

  • Reselect the row to list the test hits and test gaps for all the executables in the Application Area.

The Tests to Run tab lists the available tests for the selected executables. It has the following columns:

Column Description
Test Name The test name. Tests are grouped into test folders. You can filter the list by test name.
Tested Object The tested executable.
Status The test’s status: Optimal or Covering.
Rank The test’s rank: High, Medium, or Low, based on how recently it was last run, its passes and fails, the number of runs per day, and the number of test steps. Optimal tests have a High rank.
Repository

The Test Repository where the test is stored. A label indicates whether the test is Automated, Mixed or Manual, based on the Test Repository’s type:

  • Tosca and Certify Test Repositories store Automated tests.

  • qTest and Azure DevOps Test Repositories store Mixed tests.

  • ALM, Excel, Octane, SAP Cloud ALM, SAP Solution Manager, and Xray Test Repositories store Manual tests.
Decision The Run or Skip decision for the test. By default, the app sets optimal tests to Run and covering tests to Skip.

The Test Gaps tab displays the selected executables that have no covering tests. It has the following columns:

Column Description
Type The executable’s type.
Object Name The executable’s name.
Action

The action to take for test gap: Test, No Test, or De-scope.
Notes A comment to support the action decision.

Specify which tests to run

To specify which tests to run, follow these steps:

  1. Select the Tests to Run tab.

  2. Select either Run or Skip in each test’s Decision column. You can select Run all, Skip all, or Clear for individual test folders.

The app saves your changes immediately.

Specify an action for each test gap

To specify an action for each test gap, follow these steps:

  1. Select the Test Gaps tab.

  2. Select an action for each test gap:

    • Test: You plan to create a test for the executable.

    • No test: You accept that the executable doesn't have a test.

    • De-scope: Remove the executable from the test scope.

  3. Enter a comment to support your decision in the Justification… field.

The app saves your changes immediately.

Related topics

Dashboard screen