Developing the Exam

One of the objections to CME certification that had been raised over the years was that CME professionals' roles were too diverse, their jobs spread across too wide a spectrum, to be accommodated under one credentialing umbrella. The team approached this challenge in the same way that undergraduate medical education addresses the diverse roles that physicians perform — by identifying core competencies that all members of the profession can be expected to know, according to Kues.

The first step in creating the exam was to conduct a job analysis with the aid of test development company Schroeder Measurement Technologies. NC-CME convened a group of CME experts in 2008 to develop the outline for the job analysis, starting with the competencies developed by the Alliance for CME as the foundation. “The group listed every possible task and type of knowledge that a CME professional needs to have. We then had them rank the importance of those tasks to their jobs. There were a few tasks that didn't rank very highly and so we deleted those,” says Overstreeet.

Building on that groundwork, the psychometricians at SMT helped the NC-CME team design a job analysis survey. “The goal of the survey was to get input from all types of constituents regarding the relative importance of various tasks and knowledge to their jobs. It's an objective process that has been done many times by SMT,” Overstreet says.

The online survey was sent to the more than 1,000 individuals whose names NC-CME had collected in its database. “They represented every region of the United States, every type of provider,” says Overstreet. Nearly 300 responses were analyzed to determine the final content outline for the exam. Next, a team held a three-day workshop to write and edit test items for the initial exam. Then, the beta testers met prior to two national conferences to sit in a proctored room to take a paper-and-pencil exam. Like those who came to the initial retreat, these 47 volunteers participated at their own expense.

NC-CME gathered feedback from the beta testers via an evaluation form. “And some of them took it upon themselves to send detailed comments to Judy or speak with her,” says Overstreet. “So we found out if they thought a particular question was ambiguous, or if they thought there could be two correct answers, or whether some parts of the exam were more difficult than others. The beta testers provided valuable feedback that helped us assess the relevance and accuracy of the items.”

SMT worked with NC-CME to evaluate the scores on each item on the beta test as well as the feedback from the testers. A group of CME experts participated in a lengthy item-by-item review process, and the NC-CME team deleted questions from the exam that were not effective. Altogether, five examination teams plus SMT test developers and psychometricians were involved in creating, reviewing, and validating the exam.

They finally had the first test ready to go — a three-hour, pass-fail exam including fewer than 200 multiple-choice items. “In certification exams, the primary goal is a pass-fail discrimination. Every candidate who achieves the minimum score is deemed to be competent,” says Kues.

The First Graduates

A total of 79 CME professionals took the beta test in May and the first official exam in June, and 70 passed. Of the 37 who took the exam in September, 33 passed. The beta tests were done with pencil-and-paper; beginning with the June exam, the tests were taken on computers at proctored testing centers, with instantaneous electronic scoring, enabling proctors to print the results and give them to the candidates on-site.

Kues notes that a number of the newly certified CME professionals are among those leaders who initially expressed skepticism about certification. Some had questioned the value of what might be perceived as a general certification to those with highly specialized jobs or with terminal degrees and many years of experience.

“When NC-CME had the first discussion about who the target audience is, we never assumed that the people with terminal degrees would take this exam because we agreed with them that if, for example, you have a doctoral degree in adult education and you've been in CME for 15 years, we don't think that the certification credential is going to add much,” says Kues. “But apparently we were wrong. And I think that it goes to show that there truly is a need for certification, and even the people at the top of our profession recognize that they need to verify that they can do this job.”

Moving Forward

NC-CME will construct a new version of the exam for the June 2009 testing period to reflect developments in the profession, such as new regulations or guidelines. “While many questions are likely to be repeated, each new exam will be unique,” says Ribble. Meanwhile, test items in the current exam are scrambled to prevent candidates from sharing answers. Certification is valid for three years, a standard for certifying organizations.

Candidates who fail the exam may reapply to take it as often as desired, provided they continue to meet the eligibility requirements and submit a new application and fee for each attempt.

Once NC-CME has given the exam for one year or 500 candidates have been tested, the organization may apply for certification by the National Commission of Certifying Agencies, which would give the program increased credibility. “We are working toward that goal and will apply for NCCA certification as soon as we are eligible,” says Ribble.

Another aim of the NC-CME is to become financially self-sufficient. “As with any nonprofit organization, fundraising continues to be a challenge,” says Overstreet. “Of course, once we get a critical mass of candidates being certified and recertified on a rotating basis, that will cover the expenses of the organization.”

NC-CME is also working on building collaborations with other organizations. In July, the Alliance for CME issued a statement noting that the Alliance has met with NC-CME, supports its program, and is providing an Alliance liaison. “We appreciate and want the liaison. What we are working out now is the exact role that the liaison would play, whether as a board member, or an observer, or some other role,” says Kues. “We will be approaching other key organizations to provide liaisons as well.”

It was evident at the CCMEP celebration in October that NC-CME leaders, charter donors, and graduates are thrilled about their accomplishment, and appreciate the support they received from organizations and hundreds of people that made it possible. “Certification can be the glue that holds the CME community together and gives it coherence,” says Ribble. “I really feel that this is a concept whose time has come and that it is needed and welcomed by all aspects of the CME community. It has been the dream of a lifetime to actually see this happen.”

---

Dave Kovaleski contributed to this article.

Sidebar: Taking the Plunge

To apply to take the Certified CME Professional exam, you first fill out a form detailing your education and continuing education, CME experience, and leadership activities. If you qualify, you then may register and sit for the exam.

The CCMEP exam covers five areas of competency, with a percentage of questions assigned to each:

• Adult learning principles — 15 percent
• Educational interventions — 30 percent
• Relationships with stakeholders — 10 percent
• Leadership/administration and management — 25 percent
• Knowledge of CME environment — 20 percent

The next NC-CME exam will be available from December 1-31 at 200 testing centers throughout the United States and Canada. In 2009, exams will be available March 13-30, June 13-30, September 13-30, and December 1-31.

The fee is $400 plus a $50 processing fee.

The Candidates Handbook, registry of CCMEPs, an explanation of exam scoring, and other information is available at www.NC-CME.org.

Related Article

Earning my Certified Continuing Medical Education Professional Credential

Sidebar: What the Grads say

BRUCE BELLANDE, PhD, FACME, CCMEP, president, CME Enterprise, Carmel, Ind.; former executive director, Alliance for CME.

Years in CME: 25

Benefits: “As an employer, I would consider hiring someone coming to CME with certification to have a real leg up versus someone without it.”

Value to CME profession: Physicians and healthcare professionals are being held more accountable for their actions, so it's only appropriate that the CME professionals who are educating them are held accountable, Bellande says. “We've got to be a different type of professional than we were even five years ago.”

Taking the test: “It was a good opportunity to self-assess after many years in CME.”

Limitations: The exam could delve deeper in areas such as quality and performance improvement, Bellande says. With so many subspecialties within CME, the credential might not make sense for individuals focused in one particular area, such as compliance or outcomes.

PAMELA MASON, CCMEP, director of medical education grants at AstraZeneca, Wilmington, Del.; member, board of directors, National Commission for Certification of CME Professionals.

Years in CME: 20-plus years in the pharma industry, mostly in medical education.

Taking the test: “I thought it was important for me to take the exam early so that I wasn't just talking about it. [The exam] was fair, it was thorough. There were items that touched all stakeholders, and that's important because each stakeholder needs to know about the other.”

LINDA CANN, MSEd, CCMEP, managing director, professional education, conventions, and international affairs, American Diabetes Association, Alexandria, Va.

Years in CME: 17

Taking the test: “It's been a very long time since I've sat down and taken a three-hour exam with a No. 2 pencil.”

Benefits: “It is a guarantee to a future employer that one does understand CME, and that's a bonus if you are job hunting. It tells internal stakeholders that I know what I'm doing, so it's less likely that ADA would go down a path or develop relationships that might not be appropriate for our organization.”

Value to CME profession: “This makes us feel a little bit better about where we are and where we are going. [The certification process] validated my knowledge and experience.”

MELISSA NEWCOMB, CCMEP, MBA, assistant director for certification, continuing professional education at the University of Rochester, New York

Years in CME: 7

Exam prep: Studying was difficult because there's no book or study guide that captures all the information in one place, Newcomb says. Even if there were, the field is changing so rapidly, the information would be outdated by the time the book was printed. Keeping abreast of the latest developments via articles, meetings, and networking, along with practical experience in the field are the keys to prepping for the exam.

Value to CME profession: “It's hard to pinpoint what it is we do. When people ask, I say we're a little bit medicine, a little bit education, and a little bit business. It's exciting to me that the field has made a commitment to us by offering this exam and saying, instead of being a little bit of all those things, we're going to make our own category. I now have a definition for what I do. I am a CME professional. There might be some folks newer to the field who are enticed to stay because it's not just a job — they can make this their career.”

KATHY ANN SMITH, CCMEP, CMP, coordinator of CME, Fox Chase Cancer Center, Philadelphia

Years in CME: 28

Taking the test: “CME is a new field, almost, with all the changes that have taken place. We've seen the roles change. CME professionals need a different skill set than we've needed in the past, so I took it as a self assessment.”

Exam prep: Smith opted to take the exam without doing any sort of extensive review, to see where she stood. “Given all the changes, it makes you wonder if you're getting it all.”

Benefits: “I would hope that it would bring a certain level of confidence in not just my abilities, but in the abilities of our CME office.”

Limitations: Smith doesn't think the CCMEP should become a requirement for employment in the field, given the fact that there are so many different roles within CME. “I'm not sure that certain people who specialize in one area or another are going to feel like they need to have it.”