• Clin. Orthop. Relat. Res. · Jun 2004

    Interrater reliability in grading abstracts for the orthopaedic trauma association.

    • Mohit Bhandari, David Templeman, and Paul Tornetta.
    • Department of Clinical Epidemiology, McMaster University, Hamilton, Ontario, Canada. bhandari@sympatico.ca
    • Clin. Orthop. Relat. Res. 2004 Jun 1 (423): 217-21.

    AbstractOnly a small proportion of submitted abstracts to the annual meeting of the Orthopaedic Trauma Association can be accepted for podium presentation. Annual program committee members must ensure that the selection of abstracts is free from bias and transparent to investigators. The objectives of this study are to examine the consistency of reviewers in grading abstracts submitted for podium presentations at the 2001 and 2002 Annual Meetings of the Orthopaedic Trauma Association and to evaluate whether the grades of the actual podium presentations at the meeting are consistent with the grades based on abstracts only. Reviewers independently graded all abstracts submitted to the Orthopaedic Trauma Association for presentation in a blinded manner. Abstracts submitted by members of the review panel were independently adjudicated by six reviewers who were not members of the committee. Before final decision-making, all reviewers met to discuss the abstracts submitted for oral presentation. Among the 440 papers reviewed in 2001 and 438 papers reviewed in 2002, the interreviewer reliability for abstract review was 0.23 and 0.27, respectively. Despite disagreements in the quality of the abstracts, reviewers achieved consensus by discussions to determine the final program. Agreement among unblinded reviewers of the 67 and 73 podium presentations during the 2001 and 2002 meetings, respectively, did not improve interreviewer agreement. Of the papers of the 2002 meeting that ultimately ranked in the top 20 after the full presentation of the papers, 15 papers originally had been ranked less than 20 in the initial grading. Only one of the top three papers of the meeting originally was ranked in the top three before the meeting.

      Pubmed     Full text   Copy Citation     Plaintext  

      Add institutional full text...

    Notes

     
    Knowledge, pearl, summary or comment to share?
    300 characters remaining
    help        
    You can also include formatting, links, images and footnotes in your notes
    • Simple formatting can be added to notes, such as *italics*, _underline_ or **bold**.
    • Superscript can be denoted by <sup>text</sup> and subscript <sub>text</sub>.
    • Numbered or bulleted lists can be created using either numbered lines 1. 2. 3., hyphens - or asterisks *.
    • Links can be included with: [my link to pubmed](http://pubmed.com)
    • Images can be included with: ![alt text](https://bestmedicaljournal.com/study_graph.jpg "Image Title Text")
    • For footnotes use [^1](This is a footnote.) inline.
    • Or use an inline reference [^1] to refer to a longer footnote elseweher in the document [^1]: This is a long footnote..

    hide…