The acronym ROI appears regularly in industry articles and is widely used by meeting planners and suppliers — yet there is confusion around the true meaning of the term.

Many use ROI to mean cost savings or cost avoidance, while others use it interchangeably with the word benefits. But the true definition for ROI is earnings or net income divided by the investment. It is best expressed as a formula: The ROI percentage equals meeting benefits minus meeting costs, divided by meeting costs, times 100.

Determining the monetary benefits of the meeting is the most difficult part of this. But it can be accomplished through careful planning, data collection, and analysis.

In the 1970s, Jack J. Phillips, PhD, first developed an ROI methodology. Today, more than 2,000 organizations worldwide use a refined version of this methodology in areas from human resources to performance improvement — and it's being used to calculate the return on investment of meetings and events.

The Phillips ROI Methodology uses five levels of evaluation, all which are essential in determining ROI:

• LEVEL 1 — REACTION AND PLANNED ACTION: measures attendee satisfaction. Almost all organizations evaluate at Level 1, usually with a generic end-of-meeting questionnaire. While important as a stakeholder satisfaction measure, a favorable reaction does not ensure that attendees have acquired new skills, knowledge, opinions, attitudes, or professional contacts.

• LEVEL 2 — LEARNING: focuses on what attendees learned at the meeting using tests, role playing, group evaluations, and other assessment tools. It helps to ensure that attendees have absorbed the meeting material and know how to use it properly, but a positive measure at this level is no guarantee that what was learned will be used on the job.

• LEVEL 3 — JOB APPLICATIONS: determines if attendees applied on the job what they learned or used professional contacts made at the meeting. Various methods can be used. The frequency and use of skills are important measures at Level 3. While a good gauge of the meeting's success, it still does not guarantee that there will be a positive business effect for the organization or the attendee.

• LEVEL 4 — BUSINESS RESULTS: focuses on the results achieved by attendees as they successfully apply what they learned from meeting material, messages, or contacts. Typical Level 4 measures include output, sales, quality, costs, time, and customer satisfaction.

• LEVEL 5 — RETURN ON INVESTMENT: This ultimate level of measurement compares the monetary benefits from the meeting with the meeting costs as expressed in the ROI formula.

The data should show a chain of effect through the levels as the skills and knowledge learned (Level 2) are applied on the job (Level 3) to produce business results (Level 4).

It is not appropriate to conduct a Phillips ROI study on all types of meetings. The best meetings for an ROI study would be linked to the operational goals and/or strategic objectives of the organization and would incur significant costs and staff/participant time.

Monica Myhill, CMP, is president of Meeting Returns, a Littleton, Colo., company that provides return-on-investment impact and evaluation studies for meetings and events. Contact her at (303) 220-1920, or send an e-mail to monica@meetingreturns.com.