Report for WHO: A multicentre non-inferiority diagnostic accuracy study of the Ultra assay compared to the Xpert MTB/RIF assay
The development of the Xpert MTB/RIF (Xpert) was a major step forward for improving diagnosis of tuberculosis (TB) and resistance to rifampicin (RIF) globally. However, Xpert sensitivity is imperfect, particularly in smear-negative and HIV-associated TB, and some limitations remain in the determination of RIF-resistance status. The Ultra Xpert MTB/RIF (Ultra) was developed as a next-generation assay to overcome these limitations.
The main study comprised a 10-site, 8-country prospective multicentre diagnostic accuracy study in adults with signs/symptoms of pulmonary TB. Xpert and Ultra were performed from the same specimen and accuracy was determined with four cultures as the reference standard for TB detection (two MGIT tubes + two LJ slants, performed on two specimens obtained on separate days) and phenotypic drug-susceptibility testing as well as sequencing for RIF-resistance detection. In parallel to the main study, several retrospective studies were performed to assess the performance of Ultra in extrapulmonary (EPTB) and paediatric samples as well as in non-high burden developing country (HBDC) settings. Decision analytic modelling (full report provided separately) assessed the trade-offs of the assay based on the performance seen in the main study.
In the main study, 1,520 patients were enrolled. Sensitivity of the Ultra was 5% higher than that of Xpert (95%CI +2.7, +7.8) but specificity was 3.2% lower (95%CI -2.1, -4.7). Sensitivity-increases were highest among smear-negative patients (+17%, 95%CI +10, +25) and among HIV-infected patients (+12%, 95%CI +4.9, +21). Specificity-decreases were higher in patients with a history of TB (-5.4%, 95%CI -9.1, -3.1) than in patients with no history of TB (-2.4%, 95%CI -4.0, -1.3). Reclassifying ‘trace-calls’ (the semi-quantitative category of the Ultra assay that corresponds to the lowest bacillary burden) as ‘TB-negative’, either in all cases or in those with a history, mitigates most of the specificity losses (Specificity –1.0% and -1.9% if trace reclassified for all cases or only cases with TB history, respectively) while maintaining some of the sensitivity gains over Xpert (Sensitivity +7.6% and +15%). Employing Ultra ‘with re-testing of trace-calls’ (i.e. patients with trace-calls re-tested and considered tuberculosis-negative if result negative upon re-testing) yields similar results to reclassifying ‘trace-calls’ as ‘TB-negative’ in those patients with a history (Specificity –2.0%, Sensitivity +15% compared to Xpert). Ultra performed as well as Xpert in detection of RIF-resistant. The number of patients with RIF-resistance enrolled was insufficient to confirm analytical results that suggested a superior performance of Ultra for RIF-resistance detection.
The additional retrospective studies demonstrate that in settings where there is very limited TB transmission, (i) specificity of Ultra is close to perfect (99.3%, 95%CI 96-99); and (ii) the increased sensitivity can possibly aid TB elimination efforts. For EPTB and paediatric TB, studies highlighted the benefit of the increased sensitivity (primarily due to the ‘trace-call’) with a sensitivity of 95% for Ultra versus 45% for Xpert in TB meningitis and 71% for Ultra on respiratory samples in children versus 47% for Xpert.
The modelling demonstrated that Ultra could improve pulmonary TB case detection and outcomes. Depending on the patient population, Ultra could detect 1 additional TB case per 100 to 1000 individuals evaluated, and prevent one additional TB death per 500 to 10,000 individuals evaluated. However, the increased in case detection comes at a cost: 1 false TB diagnosis and unnecessary treatment per 40 to 70 individuals evaluated and 10 to 500 unnecessary treatments per TB death prevented. The acceptable level of unnecessary treatments per prevented death (or per additional or earlier diagnosis) is likely to vary between settings. A similar trade-off exists regardless of whether the trace-call is used.
Ultra has higher sensitivity than Xpert particularly in smear-negative and HIV-infected patients and at least as good accuracy for RIF detection. However, as a result of the increased sensitivity, Ultra also detects non-viable bacilli present particularly in patients with recent history of TB. This results in reduced specificity predominantly in adult patients with pulmonary TB in high burden settings, while in low transmission settings, EPTB and paediatric TB it does not appear to be a problem. Thus, impact of this trade-off between overtreatment and increased diagnosis/decreased TB deaths varies substantially between different settings with variable populations determined by HIV, prior TB history, and prevalence. The willingness to accept this trade-off has to be considered and implementation challenges have to be addressed.