David McCann wrote an interesting post on MISoapbox about MPI's recent focus on return on investment, and whether or not that focus is setting planners up to fail (he thinks it is—read his post for arguments as to why he thinks this might be). I wasn't able to attend this year's MPI WEC in Dallas, but he says people he spoke with there are not all on the ROI bus on this one.
In a comment on his post, meeting industry maven Joan Eisenstodt points out that ROI isn't all about dollars and cents, but also about what participants take away from the meeting. While I understand the increasing business need to prove meetings are worth the time and money being spent on them (and Sarbannes-Oxley reporting requirements just turned the heat up higher on this one), so much of the real value to the bottom line is intangible, and you're not going to get at it by just doing the math on dollars spent versus dollars increased afterward.
For example, say I attended a journalism conference and came away with a better way of writing a story (I know, it's not possible that I could write any better than I do, but let's go hypothetical here ; o). Will ad sales increase as a result? Maybe, but I highly doubt it. Will I save the company money by using this new technique? Maybe, but probably not. In fact, I may end up costing the company more money by researching using a more expensive-but-better resource, or whatever. Will it make our magazine more valuable to readers? I would think so, but there's no way to put a pricetag on that, is there? That's the problem with trying to measure values that are more abstract than concrete.
Plus, and I'm sorry to say this, folks, it's pretty well-established in the literature that adults generally don't put something into practice after attending a one-time session unless it's a serious no-brainer improvement. We need to be "touched" with the new knowledge at least seven times before it sinks in and we start using it—assuming we buy into it being a better way of doing things than what we had been doing. And there could be barriers to implementing the change that no education can address (say our budget won't allow me to use that new resource I learned about, so I can't do it, even if it would result in better articles). So expecting bottomline results from a one-time event is fairly unrealistic. It needs to be measured over time, in increments, with reinforcements.
And how many people measure results beyond the basic meeting evaluation form, anyway? Only once, ever, has someone followed up with me after a conference to see if I was using what I'd learned (and to my eternal embarrassment, while I remembered loving the session, I couldn't even remember what it was about, much less anything I learned, much less anything I learned that I was using). Continuing medical education providers are the only ones I know who are really doing much in measuring the outcomes of their activities in terms of changed behavior, but even they are stymied by the high cost of this type of research, and the difficulties in getting good, objective data to work with.
Suffice to say that I think the ROI push is putting the cart before the horse. First we need to know what attendees really need to know to improve their performance, then we need to design the meeting so it will be most effective at addressing that need (if it's a straight knowlege data dump, a lecture could work. But if you're trying to change attitudes or beliefs about something, you need a whole different design). Then you have to measure how well that need was met on an educational level. Then you go back and measure, preferably after a period of time has passed, what people are doing differently as a result. And if you want them to be doing things differently, you'll be following up with reminders, tip sheets, webinars, podcasts, papers, etc., to reinforce the learning and make it sticky enough that they'll make some changes.
Then you can try to put some dollar value to those changes. Without all those other pieces, I have a hard time believing that any ROI study is going to come up with results anyone wants to share with the big brass. After all, who wants to measure ROI only to find it in the red?