Target

The European Accessibility Act (EAA) ¹ requirements for ebooks are listed in Annex I sections III and IV Linea f). EPUB Accessibility - EU Accessibility Act Mapping ² is a W3C group note that shows how EPUB files conforming to EPUB accessibility guidelines ( EPUB Accessibility 1.1 and WCAG 2.1 AA) are responding to the European Accessibility Act requirements. Those two documents help us define our target and basis to establish the accessibility deficiencies.

Pre-paginated ebooks (like PDFs and Fixed Layout EPUBs) do not comply with the EAA requirements for the criterion of flexibility and choice in the presentation of the content ³ , a key functionality for persons facing cognitive difficulties (like dyslexia) or with sight impairments. As remediation for these types of ebooks would mean a change of format, we choose to introduce middle-way target remediation to allow the study of remediation to today's format state and possibilities offered by remediation tools. The target for these documents will be compliance with the Web Content Accessibility Guidelines (WCAG) 2.1 ⁴ . In addition, PDFs will have to reach PDF/UA conformity, a dedicated standard registered as ISO ⁵ .

Methodology

The backlist data analysis allowed us to define a wish list of categories of files to collect in order to represent the backlist composition in a small but consistent sample. We had a target objective of 200 files from 5 countries. This objective was exceeded, with 351 files collected from 7 EU countries (Denmark, Finland, France, Germany, Italy, the Netherlands, Spain), additionally including some samples also from the United Kingdom. We used the Thema codes as our reference for classification. Some provided samples were classified to different Thema categories by the publisher. As it was not possible to separate the Thema categories, we chose to multiplicate these samples (one per Thema code, i.e. a book with Thema codes D, F was analysed two times, one as D and one as F), resulting in a total of 376 units to analyse.

We added to this sample one accessible EPUB3 target file⁶, to be sure that the gap emerging from our analysis was fitting the reality and that files already made accessible would not be considered as files with remediation needs. This target file was produced by LIA as born accessible in 2023.

We first established a list of key points indicators (KPI) we wanted to evaluate and from them, we could determine the data to extract from the samples. We verified which of these data were available from existing reporting tools (EPUBCheck, ACE, and RGTK for EPUB files; VeraPDF and PDFIX for PDF files, see annex Tools used in the automated analysis for a brief description of these tools) and determined the missing ones. Fondazione LIA developed a script to extract the missing data, aggregate all the data, and export a unified report. The details of the tests are available in the Detailed evaluation of the tests made on ebooks document, available for project partners and contributing publishers. 15 iterations of the script were made to refine data extraction and the exported reports. We started from a large number of data collected to stretch to a minimum necessary point.

The report was then used to develop calculation methods to define remediation complexity indicators⁷. Iterations were needed for this step as well, as data visualisation produced helped us identify biases, missings and non-relevant information. The results of the evaluation are presented and commented on within this document.

Scoring

Usually, providers of remediation services classify the ebooks per complexity: a book with more images, tables or pages will get a higher score. This method is relevant if the whole set of ebooks to classify is produced from a known production workflow. Looking at the European level, we know that publishers’ workflows differ in the quality of files they produce, which consequently may be totally different in terms of accessibility features, accessibility information and, therefore, remediation needs.

That is why in this project we established a new classification related to remediation complexity, considering that an ebook may be very complex but already produced in accordance with accepted accessibility standards, thus resulting in a very low remediation complexity score. To be sure that the scoring was truly reflecting the remediation needs, we referred to our target file known to be fully accessible and with no remediation needs. With some iterations on scoring, we made sure that the target got a score of zero.

Capturing remediation complexities in relation to different file formats was one of the main challenges of the process. PDFs and Fixed-layout EPUBs are known to be the most complex to remediate as the technologies and languages used to build them imply more complexities and a higher level of programmatic abstraction. That’s why we decided to represent them apart.

One bias we had to deal with is that PDF format allows for less structure and metadata, resulting in less possibilities for analyses, which resulted in abnormally low scores for files in this format. To address this bias, we had to establish a complementary scoring calculation to apply to these files.

Therefore, each format has specificities related to contents found in the files and accessibility related features missing. To find the correct marker, a threshold of calculated key indicators has been established thru iterations.

Identified limits and bias

As previously commented, files in PDF format do not have the same accessibility possibilities as files in the EPUB format. Therefore, the comparison between the two formats must be done very consciously and should not lead to categorical formulas.

Most of the publishers providing samples are de facto aware of the accessibility subject and therefore the collection we have might be a biased representation of the backlist. A way to verify that would be to do a similar analysis on a large number of files not specifically selected for this type of test. This analysis perspective has been discussed with three members of EDRLab (Beletrina, De Marque and Hachette Livres) and we hope to be able to provide it as a complementary ABE Lab publication in the future.

At the time of writing this report, some remediation needs can not be spotted automatically, but as technological improvements are occurring very fast, we expect that a better gap analysis could be produced in the coming years. Examples of accessibility problems that cannot be automatically detected are incorrect, non-meaningful or insufficient image descriptions and wrong metadata claims, for which we were not able to establish a valid calculation method during this work.

Results

Classification for remediation

The following classification aims to list the remediation workflows to test. A list of six elements is spread across the different categories, here is a summary of it:

1. PDF to PDF/UA (compliant to WCAG 2.1, level AA)

2. PDF to Reflowable EPUB3 (compliant EPUB Accessibility 1.1, WCAG 2.1, level AA)

3. FXL to «accessible» FXL (compliant EPUB Accessibility 1.1, WCAG 2.1, level AA)

4. FXL to Reflowable EPUB3 (compliant EPUB Accessibility 1.1, WCAG 2.1, level AA)

5. EPUB2 to EPUB3 (compliant EPUB Accessibility 1.1, WCAG 2.1, level AA)

6. Reflowable EPUB to Reflowable EPUB3 (compliant EPUB Accessibility 1.1, WCAG 2.1, level AA)

Outcomes

From this gap analysis, we were able to establish a classification of remediation needs and build test files for each of the classifications.

Direct outcomes of this work are

• a remediation complexity assessment methodology applicable to collections of files;

• a view of the remediation complexity per Thema category;

• a view of main accessibility issues detected.

The heavy presence of images and visual resources appears to be the main criteria of demarcation between categories that will reclaim more efforts to remediate (Medicine, Earth sciences and Sports) and others (Fiction, Philosophy, Religion and Law) that will be easier to remediate.

As per the following steps of the ABE Lab project, it allows us to establish a testing classification and methodology as well as building meaningful files to test for remediation tools.

Annexes

Addendum - Per year analysis

As part of the elaborations to define the gap analysis between the accessibility requirements of the EAA and the accessibility issues of the ebooks in the backlist, we also checked the breakdown by year to see if we were able to identify meaningful patterns. Due to the context of our research, this part was not considered so relevant and therefore was not published in the initial version of this report. For clarification and completion of the information related to the study, we have decided to add this information as an Addendum by May 2024.

In the context of the ABE Lab research, the per-year analysis resulted in being fewly relevant because all Thema categories of books are present for most of the years, leading to averages scores per year variation from 20 to 37. This does not indicate that a per-year analysis is not relevant and we suggest that it should be done on defined collections. We expect that a refined per-year analysis crossed with Thema categories done on higher numbers of samples would help to identify errors patterns. This was confirmed by publishers who already started the remediation of their backlists.

This analysis showed that our sample is composed of a third of files produced after 2021, which will probably be the case for most of the ebooks that will be considered to be made accessible to comply with the EAA. It showed higher levels of complexity around the years 2017 to 2020, mainly because the samples collected for those years have a larger number of images.