Marcie Minor, CME director, is a seasoned professional at Freedom Medical School in Texas. She has been gratified that most participants in CME activities at the school have indicated lack of bias on recent post-activity evaluation forms. In concert with the associate dean for CME, she recently revised her evaluation instrument to expand the number of items related to fair, balanced, and scientifically rigorous content to include the following statements:
After a recent day-long symposium on lowering cholesterol values, Marcie was puzzled by the aggregate data from the four new evaluation items. Respondents answered based on a 5-point scale, in which 5 represented “strongly agree” and 1 represented “strongly disagree.”
Mean values for the new items were:
Marcie was very confused by the inconsistency in responses and discussed the findings with Raymond Cash, the business development director. He suggested that Marcie report only the values for scientific rigor and unbiased content (only the positive responses). Alarmed by Raymond's advice, she asks the associate dean to further help her analyze the data and improve the evaluation process yet again.
Overstreet: I applaud Marcie for seeking more information about bias in her activities. I think this case highlights some of the difficulties with measuring bias and interpreting results — something we are all struggling with.
Parochka: Karen, this may indicate that multiple questions related to the issue of biased content are confusing to participants, and that CME professionals may be better served by defining terms to clarify intent for the learners. Marcie may want to talk with the faculty to determine their interpretation of the confusing evaluation data.
Overstreet: Another important point is that just as we have to work with faculty to ensure complete and fair data in certified activities, providers should never misrepresent their own data. Marcie made the right decision to ignore bad advice from Mr. Cash and instead go to the dean for help.
Parochka: Another possibility here is that participants who don't want to insult colleagues, friends, and fellow professionals may be unwilling to rate content as biased (the last item on the list) because they fear that evaluation data may not be anonymous. Even though all four criteria are related to bias, the final one most obviously concerns bias — and we all know bias is bad. CME providers should guarantee that data collected is confidential and separate from records of participation.
Karen Overstreet, EdD, RPh, FACME, is president, Indicia Medical Education LLC, North Wales, Pa. Reach her at Karen.Overstreet@indiciaed.com.
Jacqueline Parochka, EdD, FACME, is president and CEO, Excellence in Continuing Education Ltd., Gurnee, Ill; and partner, PTR Educational Consultants.Reach her at JacquelineParochka@comcast.net.
To share your comments on this case, or if you have an “ethical hypothetical” you'd like reviewed, send an e-mail to Editor Tamar Hosansky at thosansky@meetingsnet.com.