Hospital accreditation is a poor predictor of the quality of patient outcomes, according to a new study by two University of Michigan School of Public Health researchers.
A study by Profs. John R. Griffith and Jeffrey A. Alexander, to be published in the January issue of the peer-reviewed journal Quality Management in Health Care, shows no relationship between Medicare-based measures of mortality and complications and the scores assigned to hospitals by the Joint Commission on Accreditation of Hospitals (JCAHO). Griffith and Alexander each hold funded chairs in the SPH Department of Health Management and Policy.
Griffith and Alexander's study suggests a serious need to review the system that assures quality in 95 percent of U.S. acute care hospitals, and which is used for Medicare certification and often for state licensure. The JCAHO system, in use for 50 years, almost exclusively uses structural and process measures and does not track newer, more direct performance measures such as the number of deaths or unexpected complications, the hospital's financial strength, or its ability to adapt to the latest treatment approaches.
Griffith and Alexander examined Medicare outcomes by comparing JCAHO scores, submitted by JCAHO, against Medicare inpatient data prepared annually for "The 100 Top Hospitals: Benchmarks for Success" study by Solucient LLC.
According to Griffith and Alexander, Solucient mortality and complications indexing, adjusted for differences in the kinds of patients and cases treated, shows patients' chances of having serious adverse events, such as death or complications, are about twice as great at the bottom 20 percent of hospitals as at the top 20 percent of hospitals. On other measures, the top quintile is about one and half times as good.
The "Balanced Scorecard" theory, increasingly popular among leading companies, suggests that a successful organization should achieve in a range of performance areas, including safety, stable financing, and efficiency, as well as in relations with its customers and workers, Griffith said. Such an organization also would work at the kinds of things JCAHO criteria emphasize, such as staffing in care units and compliance with codes.
Performance data like Solucient's suggest that America's hospitals need real improvement, Griffith said. "The differences between best and worst are too big to ignore. Most hospitals make a huge effort to raise their accreditation scores. If they put that energy in a smarter direction, we'd all be better off."
Part of the problem, the researchers said, is lack of public knowledge. JCAHO scores are translated to a "failure count" of the 48 areas studied, and hospitals get a list of "Type I Recommendations" to correct. About one hospital in 12 gets a nearly perfect score, with no recommendations, and only three of 100 get "conditional" accreditation.
Click here for details from the JCAHO site.
With so many hospitals receiving similar scores, the system does not encourage competition, they said.
Similarly, the Solucient performance scores are used in a "Top 100 Hospitals" program that recognizes only the 100 hospitals that achieve the highest benchmarks but fail to provide ranking information on the more than 5,000 other hospitals studied and ranked annually. Click here to see the "Top 100 Hospitals" site.
Another problem is understanding exactly what does contribute to high performance. JCAHO criteria are a consensus of what seems to work, rather than practices tested against real performance, they said. "JCAHO scores four dozen separate activities in hospitals, calculates a weighted overall score and makes a final decision to accredit. We expected to see 'good' JCAHO hospitals get 'good' performance scores---be safe, well financed, efficient and progressive. The data show the hospitals with the worst JCAHO scores have as good performance as the group with the best," Griffith said.
The disconnect suggests either something left out or something wrong. "I believe it's something left out," Griffith said. "It could be problems with the way JCAHO's inspectors assign the scores, but it's more likely the things JCAHO does not measure, particularly employee and doctor learning and enthusiasm."
Evanston, Ill.-based Solucient, which does not compete with JCAHO, provided $10,000 in funding for a detailed analysis of the stability of its "100 Top" criteria and recommendations on changes.
For more information on Griffith, author of the textbook "The Well-Managed Healthcare Organization," click here
For more information on Alexander, who recently finished a term as acting dean of the School of Public Health, click here
For more information on "Structural versus Outcomes Measures in Hospitals," the article by Griffith et al, contact Aspen Publishers Inc. at (800) 638-8437 or click here for the Aspen site.