Bogaerts Bert, Winand Raf, Fu Qiang, Van Braekel Julien, Ceyssens Pieter-Jan, Mattheus Wesley, Bertrand Sophie, De Keersmaecker Sigrid C J, Roosens Nancy H C, Vanneste Kevin
Transversal Activities in Applied Genomics, Sciensano, Brussels, Belgium.
Bacterial Diseases, Sciensano, Brussels, Belgium.
Front Microbiol. 2019 Mar 6;10:362. doi: 10.3389/fmicb.2019.00362. eCollection 2019.
Despite being a well-established research method, the use of whole-genome sequencing (WGS) for routine molecular typing and pathogen characterization remains a substantial challenge due to the required bioinformatics resources and/or expertise. Moreover, many national reference laboratories and centers, as well as other laboratories working under a quality system, require extensive validation to demonstrate that employed methods are "fit-for-purpose" and provide high-quality results. A harmonized framework with guidelines for the validation of WGS workflows does currently, however, not exist yet, despite several recent case studies highlighting the urgent need thereof. We present a validation strategy focusing specifically on the exhaustive characterization of the bioinformatics analysis of a WGS workflow designed to replace conventionally employed molecular typing methods for microbial isolates in a representative small-scale laboratory, using the pathogen as a proof-of-concept. We adapted several classically employed performance metrics specifically toward three different bioinformatics assays: resistance gene characterization (based on the ARG-ANNOT, ResFinder, CARD, and NDARO databases), several commonly employed typing schemas (including, among others, core genome multilocus sequence typing), and serogroup determination. We analyzed a core validation dataset of 67 well-characterized samples typed by means of classical genotypic and/or phenotypic methods that were sequenced in-house, allowing to evaluate repeatability, reproducibility, accuracy, precision, sensitivity, and specificity of the different bioinformatics assays. We also analyzed an extended validation dataset composed of publicly available WGS data for 64 samples by comparing results of the different bioinformatics assays against results obtained from commonly used bioinformatics tools. We demonstrate high performance, with values for all performance metrics >87%, >97%, and >90% for the resistance gene characterization, sequence typing, and serogroup determination assays, respectively, for both validation datasets. Our WGS workflow has been made publicly available as a "push-button" pipeline for Illumina data at https://galaxy.sciensano.be to showcase its implementation for non-profit and/or academic usage. Our validation strategy can be adapted to other WGS workflows for other pathogens of interest and demonstrates the added value and feasibility of employing WGS with the aim of being integrated into routine use in an applied public health setting.
尽管全基因组测序（WGS）是一种成熟的研究方法，但由于所需的生物信息学资源和/或专业知识，将其用于常规分子分型和病原体特征分析仍然是一项重大挑战。此外，许多国家参考实验室和中心，以及其他在质量体系下工作的实验室，需要进行广泛验证，以证明所采用的方法“适用”并能提供高质量结果。然而，尽管最近有几个案例研究强调了对此的迫切需求，但目前仍不存在一个关于WGS工作流程验证指南的统一框架。我们提出了一种验证策略，特别侧重于对一个WGS工作流程的生物信息学分析进行详尽表征，该工作流程旨在取代具有代表性的小规模实验室中用于微生物分离株的传统分子分型方法，并以病原体作为概念验证。我们针对三种不同的生物信息学分析专门调整了几种经典使用的性能指标：抗性基因表征（基于ARG-ANNOT、ResFinder、CARD和NDARO数据库）、几种常用的分型模式（包括核心基因组多位点序列分型等）以及血清群测定。我们分析了一个核心验证数据集，该数据集包含67个通过经典基因型和/或表型方法进行分型且在内部进行测序的特征明确的样本，从而能够评估不同生物信息学分析的可重复性、再现性、准确性、精密度、敏感性和特异性。我们还通过将不同生物信息学分析的结果与常用生物信息学工具获得的结果进行比较，分析了一个由64个样本的公开可用WGS数据组成的扩展验证数据集。对于两个验证数据集，我们都展示了高性能，抗性基因表征、序列分型和血清群测定分析的所有性能指标值分别>87%、>97%和>90%。我们的WGS工作流程已作为一个针对Illumina数据的“一键式”管道在https://galaxy.sciensano.be上公开提供，以展示其在非营利和/或学术用途中的应用。我们的验证策略可以适用于针对其他感兴趣病原体的其他WGS工作流程，并证明了采用WGS以整合到应用公共卫生环境中的常规使用中的附加价值和可行性。
