IT'S 8 P.M. ON SUNDAY NIGHT, and my phone rings. It's from a CMEer who we will call Jim. Jim is running an activity on Monday but has no way to collect information on learners' outcomes. “What can I do on such late notice?” Jim asks. And then in more panicked tones, “I've got to have something measurable, something meaningful.”
“I have a suggestion,” I say, “though it isn't as good as a study based on careful reflection of both your instructors' anticipations and your participants' needs.”
Fast-Track the Needs Assessment
I tell Jim to prepare a sheet with some kind of identification on top. In the past we've used “mother's maiden name,” or “last four digits of the social security number.” The sheet has space for each participant to respond to the question, “What will have to happen during this session so that when it is over, you'll feel your time was well-spent?” These responses are a needs assessment since they identify participants' learning needs. Recall that a characteristic of adult learners is that they learn in response to problems they have.
There is good news here. The simple fact that people write down what they'd like to gain from the session means they will call into consciousness things from their own experiences bearing on the topic of the talk. A cognitive scientist would say that the learners had ‘primed’ certain knowledge and experiences so that they can be more easily accessed during the talk.
On-the-Spot Reflection
Once instruction has begun, Jim collects the forms and photocopies them onto new sheets so that the identification and original question and box are at the top of the page, and the following instructions and another box are below it: “Think about what you learned at this session and then indicate in the space provided how what you learned related to the need you described this morning.”
Is this a good idea? You bet, if only because this is an opportunity for learners (in Donald Schön's terminology) to reflect-on-action — to think about the activity they've just finished. In my terminology, this is an opportunity for learners to extract insights from what they've experienced, insights that may help them solve problems they have. Jim's not just collecting evaluative data; he is contributing to the success of the CME activity.
Jim needs to review both the goals learners had and the things they learned relative to their goals. Doing this means Jim will:
-
Determine the degree to which learners realized their goals.
-
Identify things covered in the learning activity that participants did not comment on. He can then provide that information to participants so they'll be obliged to reflect yet again on what they wanted to learn and actually experienced.
-
Consider whether others (the speaker, the CME office) would be satisfied to know that each participant learned as much as they did in relation to their self-reported needs.
-
And finally, Jim can use the summarized information to provide needs assessment information for subsequent sessions.
Reaching for ROI
So Jim's evaluation summary might say, “Twenty of the 25 people present brought needs with them that were addressable through participation in the session. Of the 20, 18 reported learning things that had a bearing on their goals though only 12 provided acceptably complete answers to the questions posed in their goals.” Sleep well, Jim; you've got a busy day tomorrow.
Henry B. Slotnick, PhD, PhD, visiting professor, CME Office, University of Wisconsin-Madison Medical School, conducts research on how physicians learn. Reach him at hbslotnick@wisc.edu.
