• Michael Kerin Morgan, Rufus M Clarke, Michael Weidmann, John Laidlaw, and Andrew Law.
• Neurosurgery Education Development Committee, Royal Australasian College of Surgeons, Melbourne, Victoria, Australia. morgan@med.usyd.edu.au
• J Clin Neurosci. 2007 Apr 1;14(4):349-54.

UnlabelledCertifying the competence of neurosurgeons is a process of critical importance to the people of Australia and New Zealand. This process of certification occurs largely through the summative assessment of trainees involved in higher neurosurgical training. Assessment methods in higher training in neurosurgery vary widely between nations. However, there are no data about the 'utility' (validity, reliability, educational impact) of any national (or bi-national) neurosurgical training system. The utility of this process in Australia and New Zealand is difficult to study directly because of the small number of trainees and examiners involved in the certifying assessments. This study is aimed at providing indirect evidence of utility by studying a greater number of trainees and examiners during a formative assessment conducted at a training seminar in Neurosurgery in April 2005.AimTo evaluate an essay examination for neurosurgical trainees for its validity, reliability and educational impact.MethodsA short answer essay examination was undertaken by 59 trainees and corrected by up to nine examiners per part of question. The marking data were analysed. An evaluation questionnaire was answered by 48 trainees. Eight trainees who successfully passed the Fellowship examination who had also taken the short essay examination underwent a semi-structured interview.ResultsThe essay examination was found to be neither reliable (generalisability coefficient of 0.56 if the essay paper had comprised 6 questions) nor valid. Furthermore, evidence suggests that such an examination may encourage a pursuit of declarative knowledge at the expense of competence in performing neurosurgery.ConclusionThis analysis is not directly applicable to the Fellowship examination itself. However, this study does suggest that the effect of assessment instruments upon neurosurgical trainees' learning strategies should be carefully considered.

13 keywords...

hide…