DURING MEETING PROFESSIONALS INTERNATIONAL'S ROI Summit last July, Tracey Smith, manager, events programming, at Thomson RIA, a Dallas-based software company, had to duck out of a session to take part in a conference call with her vice presidents of sales, marketing, customer operations, and product development. They wanted to talk about an upcoming user conference that she was planning, and about how to measure the return on investment for their attendees.
While sitting on the phone with company brass, Smith came to the realization: “Here I am with a seat at the table. Now, what do I do with it?”
Even those who have had a seat at the table for some time are finding it difficult to qualify and quantify meetings' success, but today's new atmosphere of accountability demands it. Smith and senior planning executives like her are being asked to measure the business impact of their meetings — from the sales generated to the skills learned to the productivity improvements made — in dollars and cents. MPI has thrown its support behind an ROI measurement methodology developed by Jack Phillips, PhD, founder of the ROI Institute, Birmingham, Ala., which shows some promise — but also raises questions because of its complexity.
At the July ROI Summit, held during MPI's World Education Conference, Phillips rolled out his method for attaching metrics to the difficult-to-quantify measures of a meeting's success. His model takes what was learned at a meeting and subsequently applied or implemented on the job — such as customer retention, sales, productivity improvements, and quality improvements — and translates the effect into dollars and cents. The net benefits are then divided by the program costs to arrive at the ROI. (For more, see box on pages 12-13).
ROI is nothing new to meeting planners: It's been on their radar screens for years. But while there are other measurement models out there, none takes it to the level that Phillips does, says Monica Myhill, president, Meeting Returns, Littleton, Colo., who is certified in the program and has a partnership with Phillips to conduct ROI impact studies.
Nor are there many meeting planners calculating ROI in terms of the business impact of their meetings on their companies. “They're tracking cost savings, negotiated savings, maybe cost avoidance — and doing that very well,” says Myhill. “Not many people out there are calculating the ROI on the meeting as a whole. It's bigger than just logistics or budgeting. It's more about content.”
Phillips has spent the past 30 years developing ways to measure the economic impact of ROI, most recently in the training industry through a partnership with the American Society of Training and Development. In the past 10 years, he has certified about 2,500 people from in-house training departments at U.S. companies.
In November, 20 meeting professionals took part in Phillips' first ROI training session, a weeklong workshop in New York where they got a chance to test the formula on their own events. The goal is to have these 20 certified users train other planners. Phillips hopes to conduct at least two of these workshops annually. From the fall session, a dozen case studies will be published this year by MPI. The first case study was conducted on MPI's Professional Education Conference in late January. A guidebook, customized for the meeting industry, is also in the works, as is ROI software. Trainees left the session “inspired and motivated” after they saw their meetings run through the process, says Myhill.
Planners who got a taste of the Phillips methodology at the July ROI Summit came away intrigued by its potential, but full of questions about its complexity and application.
“I think it's the ideal model,” says Angie Pfeifer, director, travel and meeting management, Investors Group, Winnipeg, Manitoba, but she feels she needs more in-depth training to put it to use in her company. “It's very sophisticated for us right now,” she admits, “but it's enough to whet your appetite.”
Pfeifer has been closely monitoring meeting costs at her company for about four years. She looks at the cost of a meeting from year to year and determines a per-person cost. If the cost goes up or down, she finds out what drove the changes. She also produces a post-event report after every conference that looks at attendance, demographics, and the objectives of the meeting (which she helps to determine in a pre-event roundtable with management), and assesses whether the goals were met.
While she would be interested in taking one of Phillips' weeklong seminars, she did not apply any of the ideas generated at the ROI Summit. One reason is that her department does not have the internal resources and systems to collect the necessary data from meetings and translate it into dollars.
In his seminars, Phillips teaches 10 techniques to convert data into dollars. He also employs 10 different methods of isolating the impact of a meeting on sales generation, effectively tracking whether a sale came from the meeting or from some other channel, such as advertising, promotion, or a competitor dropping out of the market. The key to it all starts with data collection, which can be done in a variety of ways, primarily surveys.
But Phillips admits that some meetings are more difficult to measure than others. For a state-of-the-company employee meeting, for example, the objective might be time savings. The planner might be able to streamline the meeting, track the amount of time saved for all employees, and convert that into dollars. That said, Phillips doesn't suggest doing an ROI calculation for all meetings. “I'd only take one up to the ROI level — my most visible, expensive, time-consuming meeting — the one that ultimately raises the most concern with management.”
“When it is a program that does not have sales leads associated with it — such as shareholder or training meetings — there's no way to track it,” agrees Sharon Marsh, former team leader, event marketing, Peoplesoft Inc., Pleasanton, Calif. Even at lead-generating meetings with external customers or clients, Marsh says it's difficult to isolate the impact of the meeting and track a sale to a particular meeting. The sales process can take months, even years, and other factors affect why sales occur, such as the economy or the competitive landscape.
“My frustration is no one out there has been able to help me understand how to put numerical values on these types of meetings,” says Marsh, who attended the ROI Summit.
Those planners who are adopting Phillips' ROI elements appear to be doing so piecemeal. For example, Lauren Halpern, program manager, event marketing at Office Depot, Delray Beach, Fla., plans to introduce a pre-event survey to her meetings in addition to the post-event survey that she already does. In the pre-event survey, she will ask attendees what they hope to get out of the meeting (from an education and performance enhancement perspective) and compare it to the responses on the back end. She will then analyze the answers to see if attendees' objectives were met, recommending changes to the program if they weren't. This is all she plans to implement at this point, but she may try to take the next step of monetizing the value-added “down the road.”
It's the same with Thomson RIA's Smith, who added a question to the post-event evaluation for her November user conference about what attendees had taken away from the conference. She will use the responses to market the conference in the future and to assess its value to attendees. “That's a huge tool for us in determining next year's program,” she says. As for taking her ROI to the measurement level, she doesn't plan to go any further right now because of time and resources. “We are not incorporating all aspects of the Phillips model, but we're trying to phase in those we can.”
Phillips is well aware of the frustrations and challenges that exist among planners looking to improve their ROI methods. While he has successfully adapted his model to the training business, meetings may be more difficult because planners and attendees are generally not accustomed to the type of follow-up that is required. “The mind-set of the attendees, the mind-set of the meeting planners, and just the culture of the accountability of meetings has got to shift from during the meeting to after the meeting. That's really a paradigm shift.”
Because planners have not always had responsibility for follow-up attendee surveys and have not typically worked closely with senior executives and CFOs to compile financial data, this method may be difficult to implement at first — which is why Phillips suggests analyzing just one meeting per year.
“It's going to be a journey,” he admits. “To get into this kind of credibility, this kind of rigor, and this kind of accepted, credible process, it's going to take us a while.”
Whether the new ROI becomes an industry standard or fizzles out because of lack of interest or because it's too hard to implement remains to be seen. But one thing is clear: Whether it's the Phillips' model or some other methodology, planners are anxious for tools to help them gauge the ROI of meetings.
“I've been saying this for eight years: Someone, please give me something tangible,” Peoplesoft's Marsh quips. “And each time I go to a presentation, I feel like it's closer.”
The Phillips ROI model's data collecting can be done in a variety of ways, including pre- and post-event surveys, focus groups, and follow-up interviews. The data is then plugged into a six-step process called the “chain of impact,” which leads to an ROI calculation. Here's how the chain breaks down with each level leading to the next:
Reaction, satisfaction, and planned action: What was the perception of the meeting? How did people react?
Learning: What did they learn from it? Did their perceptions change?
Application Implementation: How will they apply what they learned to their jobs?
Business impact: How will implementation affect the organization?
ROI: What is the monetary value of that impact?
Intangibles: If the impact cannot be converted to a monetary value, what are the intangible benefits?
The next question is: What can be monetized? Sales, productivity improvements, quality improvements, time-savings, employee and customer retention? “If it's in hard data categories of output, quality, cost, and time, we can convert it to monetary value,” says Jack Phillips, PhD, founder of the ROI Institute, Birmingham, Ala., and creator of the methodology. Things such as job satisfaction, stress reduction, teamwork, improved public image, and employee engagement fall into the intangibles column. While it may be difficult to convert these benefits to dollars, their impact should be analyzed and reported to management.
The tricky part is converting this data to dollars. Phillips uses 10 different data-conversion techniques, depending on the circumstances. The first step is to look for the standard value of items to be measured. Time saved, quality improvements, and new accounts may already have established values at a company or in the marketplace. There may also be linkages to value; for example, an increase in employee or customer satisfaction might lead to better sales numbers.
The final piece of the puzzle is isolating the impact of the meeting. This is the process of determining if the meeting was the catalyst of change as opposed to sales and marketing efforts or a competitor dropping out of the market. There are several ways to do this, one of which is asking attendees what drove them to purchase a product or improve their output.
For more information, go to www.roiinstitute.net.