Understanding BLOODPAC’s Analytical Validation Protocols for Late-Stage Solid Tumor ctDNA Assays
The Generic Protocols for the Analytical Validation of Next-Generation Sequencing-Based ctDNA Assays are a set of guidelines developed by BLOODPAC in consultation with the FDA. They consist of 11 protocols designed to demonstrate and document the analytical performance of NGS-based circulating tumor DNA (ctDNA) assays for late-stage solid tumors. The protocols also include 4 methods for basic ctDNA-related procedures such as preparation of patient sample pools. In this post, we’ll cover the background, usage, and evolution of the AV protocols.
Why were BLOODPAC’s AV protocols developed?
The liquid biopsy field is changing rapidly, with new tests being developed every year. As ctDNA assays are increasingly used in routine clinical care, it is vitally important that the community–including regulatory bodies such as the FDA – aligns upon a set of benchmarks for assay validation. Shared validation standards confer many benefits, including:
Increased speed and efficiency of liquid biopsy development and dialogue with the FDA
Increased confidence in validation procedures and resulting data
Increased confidence that tests work as intended in the clinic and provide reliable information to patients and treating physicians
The AV protocols were created to fill this need and serve as a continually-evolving resource for the liquid biopsy community.
Analytical validation is a critical part of the assay development process. While pre-analytical validation covers sample acquisition, processing, and storage, analytical validation ensures that the assay accurately and reliably works as intended. Figure from Masucci et al. (2016).
What validation challenges are unique to ctDNA assays?
ctDNA-based assays pose several analytical validation challenges not encountered in traditional tissue-based tests. For instance, ctDNA is generally present at very low concentrations in patient plasma. The majority of circulating cell-free DNA (cfDNA)–generally over 99%–is not the ctDNA of interest. This cfDNA may also contain non-tumor-derived mutations, which must be distinguished from the ctDNA present. These challenges necessitate validation protocols showing that the ctDNA assay of interest is both highly sensitive and specific.
In addition, the inherent low fraction of ctDNA in patient samples leads to limited and variable validation materials. Consequently, the field has been working to develop contrived materials mimicking human plasma with defined variant allele fractions– spiked-in cell line ctDNA with known mutations–to introduce more consistency. However, these contrived materials must also be validated to show they reliably reproduce the properties of patient samples. All of these validation challenges are addressed in the AV protocols.
Who created the AV protocols?
As a consortium of 60+ ctDNA-focused industry, academic, nonprofit, payer, and regulatory organizations, BLOODPAC was well-positioned to create the first published analytical validation protocols in this space. The subject-matter-expert working group that drafted the protocols started with existing assay validation documents from entities such as the Association for Molecular Pathology (AMP) and the Clinical and Laboratory Standards Institute (CLSI). The group regularly obtained feedback from the FDA’s Center for Devices and Radiologic Health (CDRH) throughout the writing process.
What are the intended uses for the AV protocols?
These protocols are meant to be used to guide the analytical validation of NGS-based ctDNA tests with a locked assay design, but are agnostic to workflow, chemistry, and analysis instrument. In general, the documents were written for analysis of human plasma and for tests used to guide treatment decision-making in patients diagnosed with late-stage solid tumors. These protocols are not intended for use in validation of early detection and screening ctDNA assays, multi-cancer early detection (MCED) assays, blood tumor mutational burden (bTMB) assays, or molecular residual disease (MRD) assays. In fact, BLOODPAC is drafting a supplement to this work specific to bTMB assay validation, as well as an additional stand-alone protocols for tumor informed MRD assay validation. These documents will be available within the next year.
What methods and study types do the AV protocols cover?
Each protocol and method is available in the supplement of the original paper describing this work. Every protocol has a standard format consisting of an introduction, experimental design, a statistical analysis section, and an example data presentation format. Biostatisticians were consulted to define the minimal test requirements, sample size, and to give guidance on the appropriate statistical analysis. The figure below shows each protocol and method.
Protocols and methods included in BLOODPAC’s Generic Protocols for the Analytical Validation of Next-Generation Sequencing-Based ctDNA Assays. The complete documents are available in the supplementary material of the publication.
Are the protocols being updated as technologies change?
As noted earlier, BLOODPAC is finalizing a bTMB supplement to this work and a new protocol for tumor informed MRD assay validation. In addition, the consortium is currently asking for feedback on the AV protocols from the liquid biopsy community, with the goal of creating an updated and revised version. We encourage anyone who has read or used the validation protocols to share their thoughts through this feedback form so we can continue to serve the field.
