Validation of New Signal Detection Methods for Web Query Log Data Compared to Signal Detection Algorithms Used With FAERS

Susan Colilla, Elad Yom Tov, Ling Zhang, Marie-Laure Kurzinger, Stephanie Tcherny-Lessenot, Catherine Penfornis, Shang Jen, Danny S. Gonzalez, Patrick Caubel, Susan Welsh, Juhaeri Juhaeri

Post-marketing drug surveillance is largely based on signals found in spontaneous reports from patients and healthcare providers. Rare adverse drug reactions and adverse events (AEs) that may develop after long-term exposure to a drug or from drug interactions may be missed. The US FDA and others have proposed that web-based data could be mined as a resource to detect latent signals associated with adverse drug reactions. Recently, a web-based search query method called a query log reaction score (QLRS) was developed to detect whether AEs associated with certain drugs could be found from search engine query data. In this study, we compare the performance of two other algorithms, the proportional query ratio (PQR) and the proportional query rate ratio (Q-PRR) against that of two reference signal-detection algorithms (SDAs) commonly used with the FDA AE Reporting System (FAERS) database. In summary, the web query methods have moderate sensitivity (80%) in detecting signals in web query data compared with reference SDAs in FAERS when the web query data are filtered, but the query metrics generate many false-positives and have low specificity compared with reference SDAs in FAERS. Future research is needed to find better refinements of query data and/or the metrics to improve the specificity of these web query log algorithms.