Objective To conduct a data validation study encompassing an accuracy assessment of the data extraction process for the Axon Registry®. Methods Data elements were abstracted from electronic health records (EHRs) by an external auditor (IQVIA) using virtual site visits at participating sites. IQVIA independently calculated Axon Registry quality measure performance rates based on American Academy of Neurology measure specifications and logic using Axon Registry data. Agreement between Axon Registry and IQVIA data elements and measure performance rates was calculated. Discordance was investigated to elucidate underlying systemic or idiosyncratic reasons for disagreement. Results Nine sites (n = 720 patients; n = 80 patients per site) with diversity among EHR vendor, practice settings, size, locations, and data transfer method were included. There was variable concordance between the data elements in the Axon Registry and those abstracted independently by IQVIA; high match rates (≥92%) were observed for discrete elements (e.g., demographics); lower match rates (<44%) were observed for elements with free text (e.g., plan of care). Across all measures, there was a 76% patient-level measure performance agreement between Axon Registry and IQVIA (κ = 0.53, p < 0.001). Conclusion There was a range of concordance between data elements and quality measures in the Axon Registry and those independently abstracted and calculated by an independent vendor. Validation of data and processes is important for the Axon Registry as a clinical quality data registry that utilizes automated data extraction methods from the EHR. Implementation of remediation strategies to improve data accuracy will support the ability of the Axon Registry to perform accurate quality reporting.
Bibliographical noteFunding Information:
C. Baca, MD, MSHS, has nothing to disclose. She receives funding from the American Epilepsy Society/Epilepsy Foundation. She is a voluntary member of the AAN Registry Committee and she serves as a voluntary chair of the AAN Registry Data Validation Workgroup. S. Benish, MD, FAAN, has received teaching honoraria from the AAN, and she serves as the voluntary chair of the AAN Registry Analytics Subcommittee and is a voluntary member of the AAN Registry Committee. She reports no other disclosures that are relevant to this article. A. Videnovic, MD, FAAN, has received research funding from NIH/NINDS and is a consultant for Acorda (Data Safety Monitoring Board [DSMB] chair), Wilson’s Therapeutics (DSMB member), Pfizer, and Novartis. He is a voluntary member of the AAN Registry Analytics Subcommittee and the AAN Registry Committee. K. Lundgren, MBA, reports no disclosures. She is employed by the AAN. B. Magliocco, MS, reports no disclosures. He is employed by the AAN. B. Schierman, MPH, reports no disclosures. She is employed by the AAN. L. Palmer, BS, reports no disclosures. She is a voluntary member of the AAN Registry Committee. L. Jones Jr, MD, FAAN, has received publishing royalties for the Mayo Clinic Neurology Board Review, has received teaching honoraria from the AAN, and serves as the voluntary chair of the AAN Registry Committee. He reports no other disclosures that are relevant to this article. Go to Neurology.org/N for full disclosures.