Web accessibility testing

Web accessibility testing, using Deque axe-core library (opens in new tab), verifies whether your application is easy to use for all users. Automated accessibility tests that are part of your development process let you find and fix issues quickly, easily, and early. This speeds up your release cycles and reduces costs for manual testing or tricky after-the-fact fixes.

Understand the limitations

The XBrowser Engine 3.0 only supports web accessibility analyses for HTML-based web applications under the following conditions:

  • Work with TestSteps created from Modules with property value Html for Engine, excluding Special Execution Tasks.

  • Use all browsers in your tests, except for InternetExplorer and InternetExplorerInEdge.

  • Exclude closed ShadowDOMs from your tests.

  • Run your tests from the Execution tab.

Web accessibility testing is based on the WCAG 2.2 Level A and AA rulesets.

Types of accessibility tests

You can create these types of accessibility tests:

Accessibility test

Description

Specified Webpage

Tests the accessibility of a specific page that you define in the Search Criteria TestStepValues from the Standard Module Check Webpage Accessibility.

After Inputs

Runs an accessibility analysis after each TestStepValue with ActionMode Input, alongside your existing regression tests. Consequently, a longer ExecutionList may also increase your test execution time.

You may use this option to analyze the new UI state after your inputs.

Before Inputs

Runs an accessibility analysis before each TestStepValue with ActionMode Input, alongside your existing regression tests. Consequently, a longer ExecutionList may also increase your test execution time.

Note that Specified Webpage, After Inputs, and Before Inputs are also values of the RunAccessibilityAnalysis TCP.

Create accessibility tests

To create an accessibility test, follow these steps:

  1. Create a TestCase.

  2. Add the Browser test configuration parameter (TCP) and choose the browser you want to test in as the value.

  3. Decide on the type of accessibility test you want to run.

    • If you want to test along with your regression tests, add the RunAccessibilityAnalysis TCP, set its value to either After Inputs or Before Inputs, and then adjust the setting Report Successful Execution of to TestStepValues - All.

    • If you want to check the accessibility of a specific page, add the Standard Module Check Webpage Accessibility to your TestCase, specify the Search Criteria TestStepValues for the target browser window, and then adjust the setting Report Successful Execution of to anything except TestCases.

  4. To improve performance, you can add the AccessibilityFastMode TCP. To learn how to use it and what its conditions are, see "Improve performance".

  5. Run your ExecutionList.

  6. After the test run is finished, check your accessibility results and reports.

In this example, the XBrowser Engine 3.0 searches for the open browser window that has a Title starting with Tricentis, https://support-hub.tricentis.com/ as the URL, and a Window Index of 2.

Once it finds the correct browser window, the XBrowser Engine 3.0 runs an accessibility analysis on it.

Example - Web accessibility test based on the ModuleAttributes of the Module Check Webpage Acessibility

Partially disable accessibility testing

If you want to run your accessibility tests along with your regression tests, but only want to include specific TestCases or child elements, you can use the hierarchical organization to partially disable accessibility testing on different levels:

  • To exclude entire TestCases, create the RunAccessibilityAnalysis TCP on the TestCase level and set its value to Disabled.

  • To only exclude certain child elements, create the RunAccessibilityAnalysis TCP on the parent level and set its value to After Inputs or Before Inputs. Then set the TCPs to Disabled on specific child elements.

Switch between testing types

To switch from Specified Webpage to either After Inputs or Before Inputs:

  • Add the RunAccessibilityAnalysis TCP and configure its value to After Inputs or Before Inputs.

  • Adjust the setting Report Successful Execution of to TestStepValues - All.

To switch from After Inputs or Before Inputs to Specific Webpage:

  • Add the Standard Module Check Webpage Accessibility to your TestCase.

  • Fill out the Search Criteria TestStepValues to specify the page to test.

  • Change the value of the RunAccessibilityAnalysis TCP to Specific Webpage.

Improve performance

To improve the performance of your accessibility analyses, you can add the AccessibilityFastMode TCP to your test configuration:

Value

Description

Ignore Passed Checks

Improves the performance of your accessibility analyses by excluding the passed checks. As a result, the accessibility report displays only violations without comparison. This is especially useful for large webpages.

Disable Redundant Checks

Improves the performance of your accessibility analyses by avoiding the redundant analysis of unchanged elements. After the initial analysis, Tosca focuses only on the HTML elements that have changed between analyses. Note that this feature detects changes in the following elements:

  • Subtrees

  • Child lists

  • Attributes

  • Character data

Optimize Analysis

Improves the performance of your accessibility analyses by using the functionalities from Ignore Passed Checks and Disable Redundant Checks. This means Tosca applies the combined benefits of earlier analyses to review only new or modified HTML elements, avoiding redundant and previously passed checks.

Off

Disables the performance boost.

View accessibility results and reports

Once you've run your tests, it's time to work with the results:

View the accessibility scoring

To find out the amount of accessibility compliance issues detected, as well as their severity level, check out the Accessibility column in the Execution tab.

Severity levels help you decide the priority of your issues:

Severity level

Description

Priority

Frequency

Critical

Prevents access to fundamental features or content, completely blocking interactions.

Top. Fix as soon as possible.

Rare

Serious

Partially prevents access to fundamental features or content, resulting in serious barriers to overall usability and frustration for the users.

Major. Prioritize fixing.

Common

Moderate

Presents some barriers, but doesn't prevent access to fundamental features or content. It may have less impact on the user experience. However, only a fix can ensure accessibility compliance.

Moderate. Prioritize fixing in this release, if there are no higher-priority issues.

Fairly common

Minor

Represents a nuisance or a minor bug. It may have lower impact on the user experience. However, only a fix can ensure accessibility compliance.

Low. Prioritize fixing if it takes a few minutes.

Very rare

In this example, you have two ExecutionEntries in your ExecutionList:

  • You execute the ExecutionEntry Sample app 1 and run an accessibility analysis for all the TestSteps whose ActionMode is Input. This means you've set the TCP value to either After Inputs or Before Inputs.

    The execution of the TestCase is successful and returns 9 moderate issues.

  • You execute the ExecutionEntry Sample app 2 and run an accessibility check on your webpage. This means you've added the Module Check Webpage Accessibility and set the TCP value to Specified Webpage.

    The execution of the TestCase is successful and returns 1 serious issue and 8 moderate ones.

In total, the ExecutionList reports no critical or minor accessibility issues, just 1 serious, and 17 moderate ones.

Example - Web accessibility test results

Export the accessibility report

To help you better understand the accessibility scoring, possible solutions to the detected issues, and quickly make informed decisions during development, generate an accessibility report. You can export it to either PDF or JSON format.

Reports make your data easier to read and interpret, breaking down the information into multiple sections:

  • A summary dashboard to give you an overview of your overall accessibility conformance.

  • An analytics table to help you identify and address issues. The table displays full details of each issue, including a summary of the problem, the affected control object, the severity of impact rating, and multiple proposed solutions to help you restore or improve user accessibility.

To create a report, follow these steps:

  1. Right-click an ExecutionList and select Export accessibility report from the context menu.

  2. Choose the rulesets you want to include:

    • WCAG A

    • WCAG AA

    • Custom ruleset, if you want to select specific rules within the WCAG rulesets.

  3. Choose the format of your report:

    • PDF

    • JSON

  4. Select Export.