Search

Validation Platform for AI-based diagnostic evaluation

Digital Diagnosis for Tuberculosis

Every year, approximately 11 million people fall ill with tuberculosis (TB), which primarily affects people whose immune systems are compromised, either by diabetes or HIV or other factors such as alcohol and tobacco usage.

One-quarter of all cases are never diagnosed. This represents a risk not only to the person, but because one person with active TB can transmit the infection to as many as 15 people in a year, it is a risk to their friends and families as well.

An initial screening can be done with a simple chest X-ray. This is relatively inexpensive technology which is widely available in low- and middle-income countries (LMICs). What is not widely available, however, is a skilled radiologist able to read and interpret the X-rays.

Computer-aided Diagnosis

The use of artificial intelligence (AI) in medical imaging offers a promising solution to the shortage of trained radiologists, particularly in low- and middle-income countries (LMICs). Computer-Aided Diagnosis (CAD) tools powered by AI can analyze chest X-rays and identify signs of tuberculosis (TB) and other pulmonary diseases within seconds—significantly accelerating diagnosis and expanding access to care.

These AI-based systems have the potential to support clinical decision-making at all levels of the health system, including in remote or underserved areas. By facilitating rapid triage and diagnosis, they can help overcome traditional barriers such as long wait times and limited radiology expertise.

CAD tools are already being developed and used to detect TB, COVID-19, and other lung conditions. To ensure they are safe and effective in real-world settings, these tools must undergo thorough evaluation, including assessments of diagnostic accuracy, consistency across populations, and operational performance. This is especially important when deploying AI in diverse clinical environments. For adoption in LMICs, AI-enabled CAD tools must also meet stringent World Health Organization (WHO) prequalification standards.

FIND Validation Platform for AI-based diagnostic evaluation

Figure 1: Design and working of the validation platform)

 

Validation Platform

To support this goal, FIND developed and implemented the Validation Platform (VP), a system designed to enable independent and reproducible evaluations of CAD software using a private, curated, and representative library of chest X-ray images.

How does it work?

Vendors of AI-based CXR CAD tools for TB diagnosis install their software directly onto the VP (Vendor mode – Figure 1). Once installed, the software becomes inaccessible to the vendor, preventing any further modifications nor access to the log files. This safeguards the integrity of the evaluation process and ensures the privacy of the data used during testing. (FIND mode – Figure 1)

The software is then assessed within a secure environment, referred to as the “sandbox,” using a proprietary data library. This data library was constructed in collaboration with global data partners and contains chest X-rays paired with corresponding reference standard data, all de-identified to ensure privacy and annotated by qualified radiologists. There are nearly 4,000 X-rays of adults, 500 images of people living with HIV, and the largest library of X-rays of children in the world, with almost 9,000 images. Images from the data library are automatically sent to the vendor’s software, and the results are returned in a table. (FIND mode – Figure 1)

The results are then assessed against radiologist interpretations and reference standard data to ensure a comprehensive and robust evaluation (Reporting Space – Figure 1). This validation platform facilitates evidence generation, which can then be used to support the adoption of AI-based healthcare diagnostic tools in LMICs and assist vendors in meeting WHO pre-qualification.

Evidence generation

FIND, as a certified independent evaluator, conducts these assessments without granting vendors access to the underlying data. As of early 2025, 8 CAD diagnostic software packages were evaluated for the WHO evaluation.

Learn more about our CAD work for TB screening here.

Publications

  1. Worodria W, Castro R, Kik SV, Dalay V, Derendinger B, Festo C, et al. An independent, multi-country head-to-head accuracy comparison of automated chest x-ray algorithms for the triage of pulmonary tuberculosis [Internet]. Public and Global Health; 2024 [cited 2025 Mar 5]. Available from: http://medrxiv.org/lookup/doi/10.1101/2024.06.19.24309061
  2. Linsen S, Kamoun A, Gunda A, Mwenifumbo T, Chavula C, Nchimunya L, et al. A comparison of CXR-CAD software to radiologists in identifying COVID-19 in individuals evaluated for Sars CoV-2 infection in Malawi and Zambia. Bielick CG, editor. PLOS Digit Health. 2025 Jan 23;4(1):e0000535.
  3. Sebastian J, Olaru ID, Giannakis A, Arentz M, Kik SV, Ruhwald M, et al. Detection of other pathologies when utilising computer-assisted digital solutions for TB screening. IJTLD OPEN. 2024 Dec 1;1(12):533–9.
  4. Kik SV, Gelaw SM, Ruhwald M, Song R, Khan FA, Van Hest R, et al. Diagnostic accuracy of chest X-ray interpretation for tuberculosis by three artificial intelligence-based software in a screening use-case: an individual patient meta-analysis of global data [Internet]. Infectious Diseases (except HIV/AIDS); 2022 [cited 2025 Mar 5]. Available from: http://medrxiv.org/lookup/doi/10.1101/2022.01.24.22269730
  5. Crowder R, Thangakunam B, Andama A, Christopher DJ, Dalay V, Dube-Nwamba W, et al. Head-to-head comparison of diagnostic accuracy of TB screening tests: Chest-X-ray, Xpert TB host response, and C-reactive protein [Internet]. Infectious Diseases (except HIV/AIDS); 2024 [cited 2025 Mar 5]. Available from: http://medrxiv.org/lookup/doi/10.1101/2024.06.20.2430840