• Clinical epidemiology · Jan 2015

    A simplified approach to the pooled analysis of calibration of clinical prediction rules for systematic reviews of validation studies.

    • Borislav D Dimitrov, Nicola Motterlini, and Tom Fahey.
    • Academic Unit of Primary Care and Population Sciences, University of Southampton, Southampton, United Kingdom ; HRB Centre for Primary Care Research, Department of General Medicine, Division of Population Health Sciences, Royal College of Surgeons in Ireland, Dublin, Ireland.
    • Clin Epidemiol. 2015 Jan 1;7:267-80.

    ObjectiveEstimating calibration performance of clinical prediction rules (CPRs) in systematic reviews of validation studies is not possible when predicted values are neither published nor accessible or sufficient or no individual participant or patient data are available. Our aims were to describe a simplified approach for outcomes prediction and calibration assessment and evaluate its functionality and validity.Study Design And MethodsMethodological study of systematic reviews of validation studies of CPRs: a) ABCD(2) rule for prediction of 7 day stroke; and b) CRB-65 rule for prediction of 30 day mortality. Predicted outcomes in a sample validation study were computed by CPR distribution patterns ("derivation model"). As confirmation, a logistic regression model (with derivation study coefficients) was applied to CPR-based dummy variables in the validation study. Meta-analysis of validation studies provided pooled estimates of "predicted:observed" risk ratios (RRs), 95% confidence intervals (CIs), and indexes of heterogeneity (I(2) ) on forest plots (fixed and random effects models), with and without adjustment of intercepts. The above approach was also applied to the CRB-65 rule.ResultsOur simplified method, applied to ABCD(2) rule in three risk strata (low, 0-3; intermediate, 4-5; high, 6-7 points), indicated that predictions are identical to those computed by univariate, CPR-based logistic regression model. Discrimination was good (c-statistics =0.61-0.82), however, calibration in some studies was low. In such cases with miscalibration, the under-prediction (RRs =0.73-0.91, 95% CIs 0.41-1.48) could be further corrected by intercept adjustment to account for incidence differences. An improvement of both heterogeneities and P-values (Hosmer-Lemeshow goodness-of-fit test) was observed. Better calibration and improved pooled RRs (0.90-1.06), with narrower 95% CIs (0.57-1.41) were achieved.ConclusionOur results have an immediate clinical implication in situations when predicted outcomes in CPR validation studies are lacking or deficient by describing how such predictions can be obtained by everyone using the derivation study alone, without any need for highly specialized knowledge or sophisticated statistics.

      Pubmed     Free full text   Copy Citation     Plaintext  

      Add institutional full text...

    Notes

     
    Knowledge, pearl, summary or comment to share?
    300 characters remaining
    help        
    You can also include formatting, links, images and footnotes in your notes
    • Simple formatting can be added to notes, such as *italics*, _underline_ or **bold**.
    • Superscript can be denoted by <sup>text</sup> and subscript <sub>text</sub>.
    • Numbered or bulleted lists can be created using either numbered lines 1. 2. 3., hyphens - or asterisks *.
    • Links can be included with: [my link to pubmed](http://pubmed.com)
    • Images can be included with: ![alt text](https://bestmedicaljournal.com/study_graph.jpg "Image Title Text")
    • For footnotes use [^1](This is a footnote.) inline.
    • Or use an inline reference [^1] to refer to a longer footnote elseweher in the document [^1]: This is a long footnote..

    hide…