“There’s a fundamental problem with speaker evaluation sheets,” says Jeffrey De Cagna, chief strategist/founder of Principled Innovation LLC, and co-author of We Have Always Done It That Way: 101 Things About Associations We Must Change. “They assume you can judge every speaker on the same set of metrics. That’s a fallacy.” For example, he says, take a speaker presenting on a Web-based topic who uses the Web to demonstrate ideas, rather than giving a paper handout. Should that speaker be marked down for not providing the handout? If a speaker gets all fives because she does some fairly basic material really well, and another takes risks and pushes back on attendees’ assumptions and doesn’t score as well, does that mean that the latter speaker has not caused learning to happen? “It’s like the difference between giving people a bowl of ice cream and giving them a bowl of spinach that they know they should eat, but don’t like,” De Cagna says. “Too many evaluations just ask if they liked a session, not whether they learned from it.”

The problem, he says, is that many associations, due to the large number of speakers at their annual meetings, only ask questions about the lowest common denominators, such as speaker knowledge of a topic and whether or not they provided handouts. “You’re not going to learn anything about whether the speaker made a difference, based on those types of questions,” he says. And those one-to-five Likert scales have to go, he adds. “Unless it was really great or really awful, most people just pick the number three. You have to look at the comments to find anything meaningful.” De Cagna proposes that associations, instead, should aim to provide more meaningful comparisons among different types of speakers by providing different types of evaluations. “It would be relatively easy to come up with three or four categories on which they’d be evaluated, and to ask speakers to identify the category their type of presentation belongs in. Then you’d get an evaluation report that would be more meaningful.”

Look for the February issue of Association Meetings for more ideas on improving meetings by rethinking some of the traditional ways of doing things. What traditions are holding association meetings back? What would you do to change them? If you have an idea you’d like to share, please call editor Sue Pelletier at (978) 448-0377 or e-mail spelletier@charter.net.