Web accessibility testing

Web accessibility testing, using Deque axe-core library (opens in new tab), verifies whether your application is easy to use for all users. Automated accessibility tests that are part of your development process let you find and fix issues quickly, easily, and early. This speeds up your release cycles and reduces costs for manual testing or tricky after-the-fact fixes.

Understand the limitations

The XBrowser Engine 3.0 only supports web accessibility analyses for HTML-based web applications under the following conditions:

  • Work with TestSteps created from Modules with property value Html for Engine, excluding Special Execution Tasks.

  • Use all browsers in your tests, except for InternetExplorer and InternetExplorerInEdge.

  • Exclude closed ShadowDOMs from your tests.

  • Run your tests from the Execution tab.

Also, web accessibility testing is based on these rulesets:

  • WCAG 2.0 Level A, AA, AAA

  • WCAG 2.1 Level A, AA, AAA.

Types of accessibility tests

You can create two types of accessibility tests:

  • All Inputs tests all TestStepValues with ActionMode Input, alongside with your existing regression tests. Consequently, a longer ExecutionList may also lengthen your execution time.

  • Specified Webpage tests the accessibility of a specific page that you define in the Search Criteria TestStepValues from the Standard Module Check Webpage Accessibility.

Note that All Inputs and Specified Webpage are also values of the RunAccessibilityAnalysis TCP.

Create accessibility tests

To create an accessibility test, follow these steps:

  1. Create a TestCase.

  2. Add the Browser TCP and choose the browser you want to test in as the value.

  3. Decide on the type of accessibility test you want to run.

    • If you want to test along with your regression tests, add the RunAccessibilityAnalysis TCP, set its value to All Inputs, and then adjust the setting Report Successful Execution of to TestStepValues - All.

    • If you want to check the accessibility of a specific page, add the Standard Module Check Webpage Accessibility to your TestCase, specify the Search Criteria TestStepValues for the target browser window, and then adjust the setting Report Successful Execution of to anything except TestCases.

  4. Run your ExecutionList.

In this example, the XBrowser Engine 3.0 searches for the open browser window that has a Title starting with Tricentis, https://support-hub.tricentis.com/ as the URL, and a Window Index of 2.

Once it finds the correct browser window, the XBrowser Engine 3.0 runs an accessibility analysis on it.

Example - Web accessibility test based on the ModuleAttributes of the Module Check Webpage Acessibility

Partially disable accessibility testing

If you want to run your accessibility tests along with your regression tests, but only want to include specific TestCases or child elements, you can use the hierarchical organization to partially disable accessibility testing on different levels:

  • To exclude entire TestCases, create the RunAccessibilityAnalysis TCP on the TestCase level and set its value to Disabled.

  • To only exclude certain child elements, create the RunAccessibilityAnalysis TCP on the parent level and set its value to All Inputs. Then set the TCPs to Disabled on specific child elements.

Switch between testing types

To switch from Specified Webpage to All Inputs:

  • Add the RunAccessibilityAnalysis TCP and configure its value to All Inputs.

  • Adjust the setting Report Successful Execution of to TestStepValues - All.

To switch from All Inputs to Specific Webpage:

  • Add the Standard Module Check Webpage Accessibility to your TestCase.

  • Fill out the Search Criteria TestStepValues to specify the page to test.

  • Change the value of the RunAccessibilityAnalysis TCP to Specific Webpage.

View accessibility results and reports

Once you've run your tests, it's time to work with the results:

View the accessibility scoring

To find out the amount of accessibility compliance issues detected, as well as their severity level, check out the Accessibility column in the Execution tab.

Severity levels help you decide the priority of your issues:

Severity level

Description

Priority

Frequency

Critical

Prevents access to fundamental features or content, completely blocking interactions.

Top. Fix as soon as possible.

Rare

Serious

Partially prevents access to fundamental features or content, resulting in serious barriers to overall usability and frustration for the users.

Major. Prioritize fixing.

Common

Moderate

Presents some barriers, but doesn't prevent access to fundamental features or content. It may have less impact on the user experience. However, only a fix can ensure accessibility compliance.

Moderate. Prioritize fixing in this release, if there are no higher-priority issues.

Fairly common

Minor

Represents a nuisance or a minor bug. It may have lower impact on the user experience. However, only a fix can ensure accessibility compliance.

Low. Prioritize fixing if it takes a few minutes.

Very rare

In this example, you have two ExecutionEntries in your ExecutionList:

  • You execute the ExecutionEntry Sample app 1 and run an accessibility analysis for all the TestSteps whose ActionMode is Input. This means you've set the TCP value to All Inputs.

    The execution of the TestCase is successful and returns 9 moderate issues.

  • You execute the ExecutionEntry Sample app 2 and run an accessibility check on your webpage. This means you've added the Module Check Webpage Accessibility and set the TCP value to Specified Webpage.

    The execution of the TestCase is successful and returns 1 serious issue and 8 moderate ones.

In total, the ExecutionList reports no critical or minor accessibility issues, just 1 serious, and 17 moderate ones.

Example - Web accessibility test results

Export the accessibility report

To help you better understand the accessibility scoring, possible solutions to the detected issues, and quickly make informed decisions during development, generate an accessibility report. You can export it to either PDF or JSON format.

Reports make your data easier to read and interpret, breaking down the information into multiple sections:

  • A summary dashboard to give you an overview of your overall accessibility conformance.

  • An analytics table to help you identify and address issues. The table displays full details of each issue, including a summary of the problem, the affected control object, the severity of impact rating, and multiple proposed solutions to help you restore or improve user accessibility.

To get a report, right-click the ExecutionList and select Export accessibility report.