Measuring the Results

The outcomes of the initial workshop at IUSM were measured to Moore's Level 5, performance changes, and that level of outcomes will be measured at the added sites. (See box on page 18 for Moore's seven-level outcomes pyramid.)

A workshop scheduled for late 2010 on the IUSM campus is designed to take the initiative to Moore's Level 6, patient health, by specifically investigating evidence of:

1. An increase in the number of QI activities at each participating clinic or hospital;

2. An increase in the engagement of healthcare professionals in QI activities;

3. An improvement in selected clinical performance measures.

Participants in the IUSM intervention are being recruited from healthcare practices with assistance from Wishard Health Services, a central Indiana health service network of about 20 medical clinics, and the Indiana Health Information Exchange through its Quality Health First program. The partners will survey each participating practice monthly on the number of QI activities initiated that month, the number of its healthcare professionals involved, and the number of patients achieving target outcomes measures.

Winicur explains that the survey will use an interrupted-time-series experimental methodology, which allows a researcher to distinguish the effects of unaccountable variables from planned interventions across time. This allows the partners to differentiate change that stems from QI activities from unrelated causes, while also accounting for the complexity inherent in large healthcare systems, a factor often ignored by randomized controlled trials. IUSM and CMEE will ask practices opting out of the intervention to complete the monthly QI surveys to act as a comparison group. They also will invite learners to participate in the surveys via e-mail. A complementary healthcare LSS outcomes research study, “Researching Implementation and Change While Improving Quality,” has been submitted by the partners to the Agency for Healthcare Research and Quality as an R18 grant.

As noted in the December Medical Meetings' cover story, “If you plan to continue educating physicians and other healthcare workers in this increasingly QI-focused healthcare system … you're going to have to erase some old ways of thinking, planning, and conducting it.” In other words, you need a new direction. The Bridge to Quality initiative aims to help all groups within healthcare organizations head in that new direction together.

Bruce J. Bellande, PhD, FACME, CCMEP, is president of CME Enterprise, an ACCME-accredited medical education and communication company based in Carmel, Ind. He previously served as executive director of the Alliance for CME.

Sidebar: Moore's 7 Levels for CME Outcomes Measurement

1. Participation

2. Satisfaction

3. Learning

   3a. Declarative knowledge

   3b. Procedural knowledge

4. Competence

5. Performance

6. Patient health

7. Community health

Source: Moore, D.E. Jr.; Green, J.S.; Gallis, H.A. “Achieving desired results and improved outcomes: integrating planning and assessment throughout learning activities. “Journal of Continuing Education in the Health Professions 29, no. 1 (Winter 2009):1-15