Assessment Tools of Baseline and Expert Levels of Pathologist and Trainee Competence in Diagnostic Breast Cytology.
Sharon B Sams, Maxwell L Smith, Philippe Vielh, Carrie Marshall, Kerry Councilman, Heather N Mack, Stephen S Raab. University of Colorado, Aurora; Institut Gustave Roussy, Villejuif, France
Background: The establishment of baseline and expert levels of competency in diagnostic interpretation skills has an important bearing on the successful completion of training, pathology boards, day-to-day practice, medical-legal affairs, and hospital credentialing. We developed a standardized and validated method of performance assessment in diagnostic breast cytology.
Design: We developed, validated, and pilot tested glass slide and on-line tools for competency assessment in breast cytology. The tools identify specific weaknesses in individual and group performance related to criteria assessment, cognitive thinking, and evaluation of flawed specimens. The tools may be implemented through off-site preliminary assessment or in depth testing using on-site observations of skills and cognitive processes. Depending on the type of assessment, the tools contain between 10 and 50 cases representing cases of differing complexity, quality, and difficulty. We developed the modules by selecting exemplar cases from a pool of over 3,000 cases and validating expertise using blinded concordance of two internationally expert cytopathologists and clinical-histopathologic follow-up. We established the baseline performance of 10 pathologists (3 practicing pathologists and 7 trainees) on expert-based modules and performed root cause analysis to determine error source and areas of improvement for each pathologist. Performance modules for expert, competent, and novice practices were developed.
Results: In using expert-based performance modules, baseline scores (20%-70%) correlated with level of training. Practicing board-certified cytopathologists showed baseline levels of competency (50%), and no pathologist scored as an expert (>90%). Root cause analysis showed that diagnostic errors for practicing pathologists were more common in poor quality specimens and for specific lesion types (e.g., lobular carcinoma). Specific areas of weakness were identified for specific practicing pathologists (e.g., tubular carcinoma or fat necrosis). Pathology resident weaknesses correlated with year of training and generally were related to cognitive failures in synthesis rather than assessment of criteria.
Conclusions: We conclude that our competency assessment program for breast cytology is able to stratify pathologists and trainees by level of performance. We are able to identify errors and their causes for improvement purposes.
Tuesday, March 1, 2011 9:30 AM
Poster Session III # 132, Tuesday Morning