This presentation on the accuracy of physician self-assessment, given by Jocelyn Lockyer with the University of Calgary at the SACME fall meeting on Nov. 6 in Boston was right on target with the trend toward self-directed learning. As she said, "You have to be able to self-assess before you can do good self-directed learning,"
She began by discussing a 1999 Kurger and Dunning study that showed that students in the bottom quartile actually thought they were in the 62nd quartile, basically because they couldn t tell accuracy from error. Her study was done to determine whether there is a discrepancy between peer and self-assessment for three specialties (psychiatry, pediatrics, and internal medicine); and whether the patterns are the same across the clinical domains of patient management, clinical assessment, professional development, and communication.
Using 360-degree and multisource evaluation data from the College of Physicians and Surgeons of Alberta Physician Achievement Review Program and data from 304 specialists and their colleagues, she examined how they rated themselves versus how they were rated by others in areas including: giving priority to urgent requests; selecting appropriate treatments; collaborating with colleagues; respecting patients rights; taking appropriate responsibility for patients; being timely in sharing mutual-patient information with colleagues; and facilitating learning for others in their practice area. They consolidated these factors into four main domains for purposes of the study.
While the mean self-assessments were the same on all the scales, the top and bottom quartiles showed some differences between how the docs assessed themselves and how their peers assessed them. Docs in the lowest quartile by peers rated themselves 30 to 40 points higher, while docs in the highest peer-assessed quartile rated themselves 30 to 40 points lower. This held true across the specialties and the four domains.
What I got out of it which may or may not be what she intended! was that docs tend to think they re better at their weakest areas than they really are, and worse at their strongest areas than they really are. Unless, of course, they are more accurate at assessing their skills than their peers were.
At the end of the presentation, she asked the provocative question: Would it have been better not to have asked the question? I think it was an important one to ask, and the answer to me seems to say that if we want to do more self-directed learning, we need to find organized ways to give them more objective feedback to make their decisions on. Either that, or teach them how to better assess themselves.
To comment on this post, click on "comments" below. To receive a weekly update, e-mail Sue.
