How do you evaluate the success of online activities as compared to live education?
Why is it that in CME we look at live and online activities differently? Educational activities are educational activities; they just differ in methodology and delivery platform. However, when we evaluate the success of live and online CME activities, we do seem to be using different measures.
Live educational meetings, which have long been the cornerstone of CME activities, have most consistently been evaluated using levels of participation, among other criteria. We have all heard our colleagues say, “What a great meeting; we filled the room,” and “You can’t beat that! We needed an overflow room!” We have always equated high levels of participation with success in the live CME environment. We rarely if ever hear people say, “Wow, there were 500 people in the room but only 100 asked for CME certificates.”
With online CME, we hear about the number of certificates issued at least as often as we hear about the number of learners who participated in the educational activities. In online CME, the focus also often is on how many learners actually complete the educational activity; some consider this factor to be more important than the total number of people who viewed or participated in the activities.
I would argue that it is equally important to consider total participation and completion/certificate redemption in both live and online CME activities. There are many factors that come into play when determining why some learners complete activities and either request or do not request credit, and why others participate only for a portion of the activities.
When considering live activities, many participants stay in the room but do not remain engaged in the education. It is not uncommon to see people in the room e-mailing, texting, and even—dare I say it?—sleeping. One can surmise that these folks stayed only because they had some need to be in the room (e.g., part of the education was of interest, or they need the credits). Others stay for the portions of interest and then leave. I have no problem with the latter group, except when they request credit for having attended the whole activity.
As for online activities, there are many factors that might cause partial participation:
• The activity duration is too long.
• They are interested in only a portion of the activity.
• They are using the activity for information-gathering rather than for education per se.
• The programming is just not engaging.
At the time that I was writing this column, Joseph Kim, MD, MPH, started a discussion on the LinkedIn CME Group questioning the average percentage of physicians completing online CME activities; the results of this discussion will be of great interest to many in CME—I suggest you stop in and take a look at the conversation.
It is important for the CME community to really look at the various delivery formats and judge them separately but equally. Use criteria for success that really measure reason/rationale for participation, not just those that look to quantify levels of participation. Look at the total numbers of participants as a separate measure from the numbers of those who complete the course—and both of these factors should be considered separately from the number of certificate requests the provider receives.
At the end of the day, the success of CME activities is really based on addressing needs and gaps, and each learner has his or her own level of requirements. Some activities will fully meet these needs; other activities will meet them only partially.
That said, we should never lose sight of the need to address the needs of as many learners as possible, with the ultimate goal being to improve patient care. That is the true sign of success and one toward which we should all continue to strive.
Lawrence Sherman, FACME, CCMEP, is senior vice president, educational strategy, with Prova Education, an affiliate of Omnia Education, Fort Washington, Pa. He is a frequent lecturer on topics related to the strategic development, dissemination, and evaluation of CME activities. Reach him at LS@provaeducation.com.
More of Lawrence Sherman’s columns: