• Leticia L Lauricella, Priscila B Costa, Michele Salati, Paulo M Pego-Fernandes, and Ricardo M Terra.
• Thoracic Surgery Division, Faculdade de Medicina, Universidade de São Paulo, São Paulo, Brazil; Instituto do Câncer do Estado de São Paulo, Hospital das Clínicas da Faculdade de Medicina da Universidade de São Paulo, São Paulo, Brazil. Electronic address: leticialauricella@gmail.com.
• J. Am. Coll. Surg. 2018 Jun 1; 226 (6): 1128-1136.

BackgroundDatabase quality measurement should be considered a mandatory step to ensure an adequate level of confidence in data used for research and quality improvement. Several metrics have been described in the literature, but no standardized approach has been established. We aimed to describe a methodological approach applied to measure the quality and inter-rater reliability of a regional multicentric thoracic surgical database (Paulista Lung Cancer Registry).Study DesignData from the first 3 years of the Paulista Lung Cancer Registry underwent an audit process with 3 metrics: completeness, consistency, and inter-rater reliability. The first 2 methods were applied to the whole data set, and the last method was calculated using 100 cases randomized for direct auditing. Inter-rater reliability was evaluated using percentage of agreement between the data collector and auditor and through calculation of Cohen's κ and intraclass correlation.ResultsThe overall completeness per section ranged from 0.88 to 1.00, and the overall consistency was 0.96. Inter-rater reliability showed many variables with high disagreement (>10%). For numerical variables, intraclass correlation was a better metric than inter-rater reliability. Cohen's κ showed that most variables had moderate to substantial agreement.ConclusionsThe methodological approach applied to the Paulista Lung Cancer Registry showed that completeness and consistency metrics did not sufficiently reflect the real quality status of a database. The inter-rater reliability associated with κ and intraclass correlation was a better quality metric than completeness and consistency metrics because it could determine the reliability of specific variables used in research or benchmark reports. This report can be a paradigm for future studies of data quality measurement.Copyright © 2018 American College of Surgeons. Published by Elsevier Inc. All rights reserved.

21 keywords...

hide…