Configure the level of detail in your execution results

By default, Tosca Commander logs all execution results on a TestStepValue level. It also reports all possible test states: Passed, Failed, or Paused (available in manual tests only).

If you are looking for different information, you can change this level of detail in your execution results:

On which level Tosca Commander should log.
Which test states Tosca Commander should consider.

For instance, you can specify you only want results on a TestCase level and only for failed or paused tests.

Specify the level of detail

When a TestStep fails or is paused, Tosca always logs detailed information on the TestStepValue level.

For passed TestSteps, however, you can specify the level of detail in the following ways:

In the settings. In this case, Tosca Commander applies the behavior to all ExecutionLists.
With a test configuration parameter (TCP). In this case, Tosca Commander applies the behavior only to the object(s) for which you have set the parameter.

Note that TCPs override the settings for a particular object. For instance, you can modify the settings to specify a certain behavior for all ExecutionLists in the workspace and then set a parameter that specifies a different behavior for a particular ExecutionList.

Configure the settings

To change the level of detail for all ExecutionLists created with Engines 3.0, modify the setting Report Successful Execution of under Project->Settings->TBox->Logging.

Once you have modified the setting, Tosca Commander implements it in all future executions. Existing result logs stay as they are.

Create a test configuration parameter

You can set the TCP for ExecutionLists, individual ExecutionEntries, or TestMandates.

Note that this parameter only works in actual test executions and not in ScratchBook trial runs.

To change the detail level for individual objects, create a test configuration parameter named TBox_Logging_ReportSuccessfulExecutionof and give it one of the following values:

TestCases: logs execution details until the TestCase level if the TestCase passes. For details on special considerations for failed or paused TestCases, check out "Report Successful Execution of".
TestSteps: logs execution details on the TestStep level if the TestStep either passes, fails, or is paused.
TestStepValuesVerification: works only with TestStepValues whose ActionMode is Verify. Tosca logs execution details on the TestStepValue level if the TestStep either passes, fails, or is paused. Note that if a TestStep fails, Tosca logs its results, even when the ActionMode differs from Verify.
TestStepValuesAll: logs execution details on the TestStepValue level if the TestStep either passes, fails, or is paused. This is the default value.

Once you have added the TCP, Tosca Commander implements it in all future executions. Existing result logs stay as they are.

Examples

Example 1: when the TestCase fails

This example shows the difference between two possible values for the setting Report Successful Execution of. It uses a TestCase with four TestSteps:

The first three pass.
The fourth one fails.

You specify the default value TestStepValue - All. In this case, Tosca logs execution details on the TestStepValue level for all the TestSteps of the TestCase.

Execution results for a sample failed TestCase with setting Report Successful Execution of set to TestStepValue - All or TestCases

Note that for failed TestCases, setting the value to TestCases would give you the same logging. To see the difference in a passed TestCase, check Example 2: when the TestCase passes.

You specify the value TestSteps. In this case, Tosca logs execution details on the TestStep level for the passed TestSteps and the complete details until the TestStepValue level for the failed TestStep.

Execution results for a sample failed TestCase with setting Report Successful Execution of set to TestSteps