Don't ask; observe

Ever notice how your gut reaction to something changes once you're asked to explain why you felt that way? You're not alone—your attendees probably are doing the same thing, which means those focus groups and evaluations may not be as reliable a predictor of what makes your meeting work as you thought. From Creating Passionate Users:

in some cases, asking a user to explain his choices changes his choices! In his book Blink, Malcolm Gladwell (author of The Tipping Point) gives an example where students were asked to rank order 44 different kinds of strawberry jams. When compared with the rankings of experts, the students did fairly well -- "even those of us who aren't jam experts known good jam when we taste it." But--and here's where it gets weird--when the students were asked in advance to provide not just the rankings but a written explanation of their choices, the student rankings lost virtually all correlation with that of the experts. As Gladwell puts it, "By making people think about jam, [the researchers] turned them into jam idiots."

The post's author, Kathy Sierra, then says:

So how can we hope to learn anything about what our users want and need if the very act of answering a question could change their answer? We have to get better at making inferences from what we observe without intervention. We have to get to the spirit of what we observe, rather than focusing on the specific details. We have to recognize that what they do says much more than what they say, especially when they're not saying anything at all.

(Hat tip to Association Inc.)