• Yi-Chia Lee, Ying-Ting Chao, Pei-Ju Lin, Yen-Yun Yang, Yu-Cih Yang, Cheng-Chieh Chu, Yu-Chun Wang, Chin-Hao Chang, Shu-Lin Chuang, Wei-Chun Chen, Hsing-Jen Sun, Hsin-Cheng Tsou, Cheng-Fu Chou, and Wei-Shiung Yang.
• Integrative Medical Database Center, Department of Medical Research, National Taiwan University Hospital, Taipei, Taiwan; Department of Internal Medicine, College of Medicine, National Taiwan University, Taipei, Taiwan; Institute of Epidemiology and Preventive Medicine, College of Public Health, National Taiwan University, Taipei, Taiwan.
• J Formos Med Assoc. 2022 Sep 1; 121 (9): 1728-1738.

BackgroundThe need is growing to create medical big data based on the electronic health records collected from different hospitals. Errors for sure occur and how to correct them should be explored.MethodsElectronic health records of 9,197,817 patients and 53,081,148 visits, totaling about 500 million records for 2006-2016, were transmitted from eight hospitals into an integrated database. We randomly selected 10% of patients, accumulated the primary keys for their tabulated data, and compared the key numbers in the transmitted data with those of the raw data. Errors were identified based on statistical testing and clinical reasoning.ResultsData were recorded in 1573 tables. Among these, 58 (3.7%) had different key numbers, with the maximum of 16.34/1000. Statistical differences (P < 0.05) were found in 34 (58.6%), of which 15 were caused by changes in diagnostic codes, wrong accounts, or modified orders. For the rest, the differences were related to accumulation of hospital visits over time. In the remaining 24 tables (41.4%) without significant differences, three were revised because of incorrect computer programming or wrong accounts. For the rest, the programming was correct and absolute differences were negligible. The applicability was confirmed using the data of 2,730,883 patients and 15,647,468 patient-visits transmitted during 2017-2018, in which 10 (3.5%) tables were corrected.ConclusionSignificant magnitude of inconsistent data does exist during the transmission of big data from diverse sources. Systematic validation is essential. Comparing the number of data tabulated using the primary keys allow us to rapidly identify and correct these scattered errors.Copyright © 2021 Formosan Medical Association. Published by Elsevier B.V. All rights reserved.

9 keywords...

hide…