There's a lively discussion going on over Bob Bly's post about a recent Business Week article that highlights how the Gap's reliance on surveys, focus groups, and other types of market research resulted in falling sales. (Kevin Holland also has some good thoughts on this here.)
The downfall to relying on this stuff is that the survey questions don't always get at what you really want to know, and focus groups want to make you happy (which is why they should always be facilitated by an outside party, IMHO). Then there's always that old saying, "There are lies, d***ed lies, and statistics." I wouldn't throw them out altogether, but I wouldn't overly rely on them, either.
Before our Pharmaceutical Meeting Planners Forum this spring, our folks spent huge amounts of time talking with potential attendees about what they wanted to learn. From this, we distilled all kinds of great session ideas. And for the most part, it worked out fine. But in the birds of a feather session for the pharma planners, we ended up chucking the agenda of topics they had said they wanted to talk about. Once they got in a room together, the group realized that what they really wanted to do was bond over common issues, not learn more about specific topics. Now we're poring over the evaluations, but taking them with a grain of salt, too.
If you want to blow their socks off at your next meeting, the evals and advance research will only take you so far. That's because people don't know what they want. I might be satisfied with a great session on creating innovative themes at an industry association event, but what is there out there that would really shake me out of my rut that I don't even know exists? People want to be surprised and delighted, not just satisfied. To do that, you have to go beyond what they tell you they want.
This is something I've been struggling with as we work on developing a new design for our Web site. We know what all the traffic data says draws people to the site, but there has to be something else, something that we can do to help our readers in ways they don't even know they want. This is incredibly hard work, which is probably why people tend not to do it. But I keep asking myself "what if..." and when and if I stumble across something good, my "gut and instinct" will let me know. Nothing's pinged me so far, but I'll keep trying...