Hospital Safety Scores
The Leapfrog Group
The Leapfrog Group is a national not-for-profit organization established by coalition of health care purchasers, sought to address the need for a standardized method to evaluate patient safety in US hospitals through creating a composite safety score. Leapfrog Group was founded and launched in Nov 2000.
Scope of Report
Leapfrog Hospital Safety Scores are based on surveys submitted voluntarily by hospitals across the country for the purpose of transparency and improvement. The survey assesses hospitals on three key areas: how patients fare, resources used in caring for patients, and leadership and structures that promote patient safety.
Report’s Intended End Users
The Hospital Safety Score benefits purchasers, providers and hospitals by supporting informed health care decisions by those who use and pay for health care; and promoting high-value health care through incentives and rewards.
Understanding the Score
The Hospital Safety Score is a letter grade of A, B, C, D, or F reflecting patient safety within a hospital. Participation and submittal of data is free and voluntary for each hospital.
The Leapfrog Hospital Survey is distributed to the hospitals annually from April 1st to December 31st. Data for each report is derived from the previous Calendar Year. The report offers a national-, state-, and hospital-level report for each measure.
- Nation’s general acute care, urban, Rural, Critical Access, and Children’s hospitals
- Leapfrog applies the minimum reporting requirement for using the measure in the safety score: the number of cases in the denominator must be >= 30.
The Leapfrog Group is not able to calculate a safety score for the following types of hospitals:
- Critical access hospitals (CAH)
- Long-term care and rehabilitation facilities
- Mental health facilities
- Federal hospitals (e.g., Veterans Affairs, Indian Health Services, etc.)
- Specialty hospitals, including surgical centers and cancer hospitals
- Free-standing pediatric hospitals
- Hospitals in U.S. territories
- Maryland hospitals, as they do not participate in the Center for Medicare and Medicaid Services’ (CMS) Inpatient Prospective Payment System (IPPS)
- Hospitals that are missing data for more than 9 process/structural measures or more than 4 outcome measures
- Leapfrog Hospital Survey
- CMS Hospital Compare
- AHA Annual Survey
The score includes data that patient safety experts use to compare hospitals. Publicly available data from the Centers for Medicare and Medicaid Services (CMS), the Leapfrog Hospital Survey, and secondary data sources are weighted and then combined to produce a single composite score that is published as an A, B, C, D or F letter grade.
Hospital performance on each measure was converted to a ‘z-score’ and then aggregated using measure-specific weights. Each individual measure included in the Hospital Safety Score is assigned a weight. The methodology to assign weights includes three criteria that reflect the quality of the measure. These criteria are: (1) Impact, (2) Evidence, and (3) Opportunity. These three (3) criteria are then combined using the following equation to compute a relative importance score for each measure: [Evidence + (Opportunity x Impact)]. The score computed from this calculation is then used to calculate an overall weight for each measure.
The Evidence Score for each individual measure is assigned a value of one (1) or two (2) using the following criteria:
1 = Supported by either suggestive clinical or epidemiological studies or theoretical rationale
2 = Supported by experimental, clinical, or epidemiological studies and strong theoretical rationale
The Opportunity Score for each individual measure is based on the Coefficient of Variation (Standard Deviation/Mean) of that measure, using the following formula: [1 + (Standard Deviation/Mean)]. The Opportunity Score is on a continuous scale that is capped at three (3). Any measure with an Opportunity Score above three (3) is assigned a three (3).
The Impact Score for each individual measure is comprised of two (2) parts, each of which is assigned a value from one (1) to three (3):
Number of patients affected
Severity of harm
The number of patients affected score is determined by the following:
1 = Rare event (e.g., Foreign Object Retained After Surgery)
2 = Some patients in hospital affected (e.g., ICU Physician Staffing)
3 = All patients in hospital affected (e.g., Hand Hygiene Safe Practice)
The severity of harm score is determined by the following:
1 = No direct evidence of harm or harm reduction (e.g., Hand Hygiene Safe Practice)
2 = Clear documentation of harm or harm reduction; adverse events (e.g., Foreign Object Retained After Surgery)
3 = Significant mortality reduction (more than 1,000 deaths or a 10% reduction in hospital wide mortality) (e.g., ICU Physician Staffing)
The values from each part are then added together to arrive at the overall Impact Score using the following criteria:
1 = Score of 2 (Low Impact)
2 = Score of 3-4 (Medium Impact) (e.g., Foreign Object Retained After Surgery; Hand Hygiene Safe Practice)
3 = Score of 5-6 (High Impact) (e.g., ICU Physician Staffing)
Once all data elements have been collected for a given hospital and all missing data have been scored appropriately, the Hospital Safety Score can be calculated using the methodology described below.
Z-Scores are used to standardize data from individual measures with different scales. Z-Scores can tell a hospital whether their score on a particular measure is above, below, or equal to the mean.
In the Scoring Methodology, a Z-Score is calculated for each measure that is applicable to a hospital. A z-Score is calculated using a hospital’s actual (raw) measure score, the national mean, and the standard deviation for that measure. The z-Score for each measure is calculated using the following formulas:
- For Process/Structural Measures: [Hospital Score – Mean) / Standard Deviation]
- For Outcome Measures: [(Mean – Hospital Score) / Standard Deviation]
Weighted Process Score
To find the weighted process score, first multiply the z-Score of each process measure by the weight assigned for that measure to get the weighted process measure score.
For more information on methodology refer to http://www.hospitalsafetyscore.org/media/file/HospitalSafetyScore_ScoringMethodology_Spring2014_Final.pdf
The Leapfrog Hospital Survey assesses hospital performance based on three key areas: (a) how patients fare, (b) resources used in caring for patients and © leadership and structures that promote patient safety. The survey consists of twenty-eight individual measures that cover a range of hospital processes, outcomes, and structures in many areas of care as below:
The survey consists of twenty-eight individual measures that cover a range of hospital processes, outcomes, and structures in many areas of care as below:
Improvement tools and resources can be found at http://www.hqinstitute.org/tools-resources