Smartphone Automation of RSS in Academic CME

The continuing medical education community was quick to jump on smartphone applications, especially after the release of the Manhattan Research 2010 report stating that 72 percent of U.S. physicians use smartphones, compared to fewer than 20 percent of the general population. Now smartphone applications for the delivery of CME programming and personal data management have become an expectation rather than a novelty, with new entries appearing in the CME marketplace daily. However, the CME community has been slower to adopt free open-source software (FOSS), which together with mobile smartphone technologies can provide continuing medical education providers a wide range of cost-effective models for automating and streamlining their CME enterprise. Despite the low deployment costs, scalability, and popularity of FOSS applications (e.g., Drupal content management system, the Moodle learning management system, WordPress, MySQL) across the U.S. government, academic, not-for-profit, and corporate sectors, it just hasn’t caught on with CME providers. This is unfortunate, because integrating the two technologies can increase compliance and reduce costs associated with delivering high-quality CME activities.

At the 2011 Annual Meeting of the Alliance for CME, held in January in San Francisco, we presented the results of a pilot study examining the effectiveness of integrating DLC Solutions’ EthosCE open-source learning management system (LMS) with smartphone short-message service (SMS) technology to streamline University of Pennsylvania School of Medicine Office of CME regularly scheduled series (RSS). Our initial efforts focused on overhauling all application, approval, and attendance-tracking processes for grand rounds and case presentations. EthosCE combines Drupal CMS and Moodle LMS for CME administration and delivery. SMS technology enables standards-compliant smartphones to share 160-character messages and is the most popular smartphone feature in use today.

2002–2010: Setting the Stage
In 2002, when Zalman Agus and Mila Kostic joined the University of Pennsylvania School of Medicine to redesign and lead the Office of CME, the office was conducting 45 RSS annually with just five full-time employees, no online presence, and a barely functional database designed to manage CME activity data. Attendance records were manually entered into the database.

As the Office of CME grew over the next several years, it hired five new employees and developed a large online presence that was housed in a separate content management system. By 2009, it had leased a proprietary database to manage its CME activities data, which included conducting 90 RSS, accrediting 5,431 activities, and issuing 734,815 certificates. The database was administered manually and, even with the introduction of a bar code–scanning system, it still required extensive data entry.

2010: The Breaking Point
By 2010, the Office of CME was accrediting a large number of Web-based enduring activities housed by third-party education partners, and it was experiencing a major increase in the amount of participant data being stored in off-site databases and not included in its main database. And despite having temporary workers and one dedicated full-time employee to review RSS applications, monitor disclosures and handouts, and make on-site visits, the Office of CME was faced with a six-month backlog in entering attendance records and generating CME certificates. Kostic says, “The dramatic expansion of our program and lack of a comprehensive LMS resulted in increased labor costs and an administrative nightmare. We required a state-of-the-art LMS to support our CME enterprise.”

The Smartphone/FOSS Solution
After evaluating a number of different technology solutions, the Office of CME partnered with DLC Solutions (DLC-solutions.com) to consolidate its Web sites and databases under the EthosCE LMS framework. As part of our initial stage of collaboration, we conducted a strategic analysis of the Office of CME processes and technologies and interviewed key stakeholders to illuminate “pain points” that would benefit from automation. RSS were an immediate candidate for automation.

We wanted to streamline the RSS work-flow process to eliminate the six-month backlog of attendance records and minimize labor costs associated with manual data entry, training, and monitoring. Together, we decided to program a custom RSS work flow within EthosCE and leverage “point-of-entry” technologies, which put attendance management in the hands of attendees.

At first we considered using the institution’s staff key card readers. However, this model required each meeting room to have the institution’s key card–reading hardware installed and interfaced with University’s multiple authentication databases as well as EthosCE. While this initially seemed logical, we were concerned that the amount of time required to gain permission by the Department of Information Technology to access the different databases would be prohibitive. So we turned our focus to leveraging the staff’s mobile smartphone capabilities.

To develop a custom module that would enable the EthosCE LMS to integrate SMS text messaging technology, we did the following:
1. configured the EthosCE Attendance Module to validate participants’ smartphone and credentials and update their CME transcript,
2. created a Mobivity SMS account at www.mobilemarketing.net,
3. modified the Mobivity SMS Module for Drupal to support incoming and outgoing text messages on a shortcode through an SMS framework. (DLC Solutions’ modified source code is a free download at http://groups.drupal .org/node/67723), and
4. designed a custom application programming interface between Mobivity SMS Module and EthosCE Attendance Module.

At the onset of the project, our team mapped out the existing RSS application, and peer-review and approval processes, which primarily consisted of completing, reviewing, and routing PDF and PowerPoint documents via e-mail. We then identified all “touch points” and triggers that required administrative action and began to identify areas for automation.

Upon concluding that the entire RSS application and approval process could be administered online, we programmed an automated workflow module within EthosCE that encompassed the following steps:
• Each department’s grand rounds coordinator completes and routes the Web-based RSS application and content to the Office of CME for review and approval.
• The Office of CME’s RSS coordinator is notified by e-mail and, upon login, is able to review the application and content in the RSS dashboard.
• If required, the application and content is routed for peer-review.
• The peer-reviewer logs in to EthosCE, reviews the application and presentation content, and completes an online form either recommending that the application be approved or rejecting it, with qualifying commentary.
• The Office of CME RSS coordinator is notified by e-mail when peer review is completed, including recommendations. The RSS then is either formally approved or rejected in the LMS, and e-mail notifications go to the applying department’s grand rounds coordinator.
• Accreditation and SMS shortcode information for the newly approved RSS program is created in LMS and presented to the department grand rounds coordinator within EthosCE.
• The SMS shortcode and instructions are then provided to the grand rounds attendees during the session. The attendees send a simple message to the Office of CME’s EthosCE database, which records, validates, and confirms their attendance. Nonregistered attendees are notified via a text message with instructions for creating an account within the EthosCE database. Once they are registered, their transcript for the event is marked as “attended.”

In addition, we built a robust attendance administration interface that allows each department coordinator to manually record attendance for any learners who either were not able or not inclined to complete the SMS process.

Finally, an XML export process was built based on the Accreditation Council for CME’s Program and Activity Reporting System specification. This process will allow us to export annual data from the Office of CME database, which can then be uploaded directly to the Web interface for ACCME Program and Activity Reporting System annual reporting.

Is It Working?
The Office of CME conducted an initial pilot study with four departments to assess the feasibility and effectiveness of using a mobile text messaging application for RSS-attendance tracking. We sent a survey to attendees of an RSS grand rounds given by one of the pilot departments between September and December 2010.

The results showed that 78.5 percent of the 736 total attendance records were recorded via SMS. Accordingly, more than 75 percent of survey respondents reported using SMS. Seventeen percent of non-SMS attendees reported that, while they did have a smartphone, they either did not enable SMS or didn’t pay for a text messaging plan. Just 2.4 percent said they did not have a mobile smartphone.

Respondents who used SMS for RSS attendance-tracking rated their experience using a five-point Likert scale (1 = poor to 5 = excellent). The average rating for ease of use was 4.45, the clarity of instructions averaged 4.45, and credit monitoring scored a 4.6 average rating.

Time to Take Advantage
Significant advances in smartphone and open-source Web technologies provide the CME community with unique opportunities to simultaneously reduce costs and increase accreditation compliance. More specifically, these technology advances allow for major changes within the CME accreditation process for RSS now and, in short order, for live courses and enduring materials.

The Office of CME and DLC Solutions were able to successfully partner to implement a work-flow and technology framework that has been well-received by attendees and administrators and has markedly improved efficiency, accuracy, and documentation of the entire RSS accreditation process. This model should be considered as an adjunct to CME offices with a large RSS program. With full implementation of the new LMS system pending, we anticipate it will further streamline the CME administrative process and allow for much easier collection and analysis of the outcomes of our educational initiatives and the CME program as a whole.

Sidebar: Benefits Gained
The University of Pennsylvania School of Medicine Office of CME realized several benefits from having an integrated open-source learning-management system that 78.5 percent of docs used to enter their attendance records via real-time SMS:
■ One point of entry for all activities and all learners, integration with multiple data sources, and links to internal and external resources
■ One integrated CME administrative process
■ No recurring licensing fees because the University of Pennsylvania owns the EthosCE LMS source code.
■ Easy access for multiple users to activity applications, disclosures, approval submissions, and documentation approvals prospective for all types of activities including regularly scheduled series.
■ All educational formats can be built by staff and faculty and displayed uniformly to learners.
■ Learners have easy access to the whole process, from registration to certificate and transcript.
■ Easy data collection and reporting to the Accreditation Council for CME, as well as for internal annual program analysis and reviews

Automating RSS using this technology also resulted in some significant savings in time and effort. It allows learners to control the automated attendance records, so there’s less room for data-entry errors. It provides transcript records instantly and gives administrators and coordinators instant reporting capabilities. It also is proving valuable to other regulatory bodies at both the departmental and institutional level, and there are fewer issues with session documentation-approval timelines now that the Office of CME can provide limited administrative privileges to internal and external collaborators, automate the peer-review process, and keep the CME coordinator in the loop. It also makes it easy to collect, analyze, and report assessment and evaluation data, and overall allowed for a much more efficient, accurate system to manage RSS and assign office staff to more creative and valuable activities. In addition, since it integrates with the ACCME’s Program and Activity Reporting System, it will make life easier during the ACCME reaccreditation process and while preparing the annual reports.

Jeremy C. Lundberg, MSSW, LSW, is the chief executive officer of DLC Solutions LLC, a firm located in Philadelphia and Washington, D.C., that specializes in Web and e-learning technologies for CME organizations. His firm’s clients include the American Psychiatric Association, the American Society of Anesthesiologists, the American Academy of Physician Assistants, United Healthcare, and the University of Pennsylvania School of Medicine. Lundberg received the 2011 Alliance for CME President’s Award for his leadership contributions in emerging technologies and social media. He can be reached at jlundberg@dlc-solutions.com; (267) 234-7401.
Zalman S. Agus, MD, is the associate dean for continuing medical education and emeritus professor of medicine at the University of Pennsylvania School of Medicine. An accomplished scientist, author, and educator, he has served in various capacities for the American Society of Nephrology, the National Kidney Foundation, and the American Heart Association and is a member of the American Society for Clinical Investigation and has been a fellow in the American College of Physicians.
Mila Kostic is the director of CME at the University of Pennsylvania School of Medicine and has championed innovation and growth of this CME program since 2002. She is an active member of the CME community and regularly presents at national meetings and contributes to professional literature.
Rodman Campbell is the CME data and systems coordinator at the University of Pennsylvania School of Medicine. Having joined the Office of Continuing Medical Education in 2008, he manages all information technology initiatives, including online educational activities and data administration.

You Also Might Be Interested In
Alliance for CME Joins Social Networking Sites
Survey of How CME Providers Use Social Media
Case Study: ARS for Medical Education