Active to the Finish This column describes an evaluation technique that keeps learners active at the session's end. It was first described by the statistician Frederick Mosteller, who called them minute reports (the word minute suggesting they take little time). In reality, the process takes 5 to 10 minutes.
A minute report asks learners to answer three to five mostly open-ended questions. Open-ended questions are preferable for two reasons. The first is that locating/recalling information and using it to construct an answer is a more cognitively engaging task than is selecting an option from among those offered by the instructor, or selecting a point along a Likert scale. The second is that open-ended answers are more meaningful than forced-choice responses. An example will show how this works.
Pretend that you are a learner in a didactic session I've run on helping clinicians to be active learners, covering the topics addressed in my earlier columns. I now ask you to complete a minute report with four questions:
1. What is/are the most important thing(s) you've learned from this session? To answer this question, you must review the material covered and determine its worth - which you do by seeing how it relates to what you already know and your experiences as both a teacher and a learner. This contemplation leads you to identify what it is you're going to write as your answer, and so you cast your selection in words. Since you're conscientious, you review what you've written to make sure it says what you want it to say.
When I read what you've written, I ask myself two questions: Are your most important items consistent with my list of objectives for the session? Can I understand why your selection is important to you? After looking at all of the minute reports in this way, I construct a picture of what you and your colleagues learned that you found most important, and use that picture to determine how well my instructional objectives were met.
2. In what way(s) do you see yourself using the things you've learned in this session? Now you must think in detail about the application of what you've learned. You must consider whether you can use programmed lectures (MM July/August, page 116) or whether you can convince the speakers who work for you to use them. You are making connections between what you've learned and your professional life.
3. What topics covered in the session remain "muddy" for you? In answering, you provide information I can use for improving the session.
4. What topics considered in this session would you like to know more about? Your answers to this question provide needs-assessment information that I can use in planning subsequent sessions for people like you. (And if I create sessions to address these needs, and then use minute reports and the techniques described in "Active from the Start," MM September/October - I have data applicable to Essential Area 2, Educational Planning and Evaluation, under the Accreditation Council for CME's new system.)
Equal Opportunity Evaluations Minute reports have two problems. First, reading large numbers of responses takes time, and second, there are many questions one might ask. Happily, both problems can be addressed using random sampling. Let's say I feel it isn't necessary to ask any given question of more than 25 people, and let's say there were 100 participants in the didactic session you attended. I'll construct four minute reports, each with different questions, and have each filled out by a random quarter of the learners. In this way, I can provide everyone with opportunities to actively consider the material presented, ask a range of questions, avoid overburdening learners, control the amount I have to read, and do all this in just 5 to 10 minutes.