Development of an Objective, Case-Based Measure of Pathology Trainee Performance: The Trainee Review Field.
Jennifer H Crow, Frank Schneider, Rob Kloehn, Anand S Lagoo, Patrick J Buckley, Rex Bentley. Duke University Medical Center, Durham, NC; University of Pittsburgh Medical Center, PA
Background: The ACGME Outcome Project has mandated development of assessment methods to evaluate trainee competency; ideal assessment tools are useful, valid, and reliable. We describe the development of a simple, case-based tool (the Trainee Review Field) for objective evaluation of Pathology trainees' competence within areas of Anatomic Pathology. The field was designed to be simple and quick to complete, confidential, and mandatory.
Design: For each trainee case, the trainee's diagnostic report is scored on Accuracy, Difficulty, and Completeness using scales of 0-3 for accuracy and 1-3 for difficulty and completeness. The scoring is performed by the attending at the time of case sign-out and is required for verification of the case. Scores are compiled and presented monthly to each trainee and reviewed regularly by the program director. Anonymized data is presented periodically to the Education Committee for review of program performance.
Results: Through the first 24 months of implementation, 30,867 cases were scored. Seventeen trainees have data for both years, and eight trainees have data for only one year. The average scores for accuracy and completeness are lowest for first-year trainees and highest for fourth-year trainees and fellows. The percentage of cases with an incorrect diagnosis is highest for first-year trainees and lowest for upper-level trainees (fourth-year trainees and fellows), and the percentage of cases with a correct diagnosis increases with training level. Of the seventeen trainees with data from both years, fifteen show significant improvement in their yearly average diagnostic accuracy scores.
Conclusions: The Trainee Review field is a useful, reliable, and valid method for objective evaluation of the diagnostic skills of Pathology trainees (residents and fellows). The data generated is a valuable assessment of individual trainees as well as the training program. Continued use of the field will provide more robust data on trainee performance for each rotation and year of training.
Tuesday, March 1, 2011 9:30 AM
Poster Session III # 128, Tuesday Morning