For the nearly 30 years that EMS has been involved with quality – improvement, assurance or management – there’s been talk of benchmarking.
“We could do some benchmarking.”
“We’ve set a high benchmark for our response time performance, ROSC, protocol compliance, etc.”
“We’ve finally got the data so we can do some benchmarking.”
In EMS, when people talk about benchmarking they are generally looking to rank themselves compared to others. In the rest of the healthcare world, benchmarking refers to a practice of looking to see who’s doing what really well and copy them. I’d like us EMS folk to put our old way of thinking about benchmarking in the grave and adopt the improvement-oriented approach to benchmarking. Here’s why:
A couple of decades ago, I was consulting with a group of well-regarded EMS systems. They were hell-bent on doing a benchmarking project. I tried unsuccessfully to talk them out of using the standard comparison model.
To get started, they wanted to gather data from the 12 organizations in their group on 25 simple areas including intubation and IV success rates, trauma scene times, cardiac arrest resuscitation rates, successful seizure control and time to answer the 911 phone in dispatch. They wanted to use the exact same definition for each measure for the exact same time period so they could compare across the organizations.
After a half a year’s worth of the kind of frustration associated with trying to push a bowling ball through a soda straw, we were able to have data on trauma scene times (back then we were confident that this measure was vitally important) and emergency response times using the same criteria for only four of the systems. The client finally realized after paying the consulting bill that they had essentially purchased the equivalent of an exceptionally expensive aerial ladder truck for a fire service that only serves structure-less campgrounds.
Benchmarking is an improvement tool
In the rest of the health care world, benchmarking is an improvement tool. Its job is to help an organization that’s working on an improvement project gather testable ideas that might make things better. The recipe for this approach to benchmarking is:
1. Aim statement
Start with a clear aim for something you’d like to improve in your system. Like shortening the first ring of the 911 phone to balloon inflation times for people with STEMI, improving customer satisfaction survey scores, decreasing employee turnover or increasing the percentage of people who have suffered a sudden cardiac arrest that survive and are discharged from the hospital neurologically-intact.
2. Identify success in other organizations
Then you look for a place that does what you’re trying to improve really well. You can do this by looking in the literature, asking people in your network or checking in with EMS industry consultants. If you need help, send me an email at mtaigman@firstwatch.net; chances are good that I can put you in touch with a good place.
3. Study successful organizations
Study what they do and how they do it. This step often involves a visit to the organization.
4. Identify change ideas
Create a list of change ideas from what you learn that are testable in your own system. If you’re going to swipe, it’s always good to swipe from the best.
5. Test ideas with PDSA cycles
Test the ideas in your system using plan, do, study, act (PDSA) cycles. Keep in mind that for clinical improvement projects, we’re talking about safe/evidence-based improvements like monitoring and shortening the length of compression pauses in CPR or CPAP for pulmonary edema. This is not the place to try crazy, potentially harmful ideas.
6. Share
Share what you learn from your experience with the folks you benchmarked so they can learn/improve too.
My favorite EMS benchmarking practice is the Resuscitation Academy in Seattle. Mickey Eisenberg, MD, PhD, Peter Kudenchuk, MD, Thomas D. Rea, MD, MPH, Michael Sayre, MD, King County Medic One and the Seattle Fire Department have dedicated the last 40-plus years to studying and improving cardiac arrest resuscitation. I suspect that they have contributed more knowledge to the field of prehospital resuscitation than anyone. Over the years, they have conducted rigorous scientific studies on nearly every reasonable theory that might improve resuscitation including a wide variety of ways to provide CPR including all kinds of assistive devices, different tools and approaches for airway management, a variety of pharmacological options for resuscitation, hypothermia and more.
During the two-day Resuscitation Academy, they generously share what they have learned, what they are currently researching and the many things they still do not yet understand. You’d be hard-pressed to find a smarter, kinder group of medical professionals anywhere on the planet. I know leaders from EMS systems all over the United States including EMSA in Oklahoma, MEDIC in Charlotte, AMR in Ventura, Santa Barbara, Calif., and more, that have attended the academy, brought ideas back to their system and have used them to significantly, sometimes dramatically, improve resuscitation rates in their communities.
Expertise outside of EMS
Some great improvement concepts can also be found outside of the EMS profession. One of the benefits of being a fire department or ambulance service provider of EMS is that normal people think we are cool. So they are usually willing to make time to share what they do with us.
When the folks at Sarasota (Fla.) Fire Department wanted to improve their customer service, they asked the Malcom Baldrige National Quality Award-winning Ritz Carlton hotel in Naples, Fla. if they could come for a visit. Joe Penner and the folks at MEDIC in Charlotte wanted help improving their teamwork, so they reached out to Boeing, another Baldrige Award winner. When we wanted to improve our selection of candidates for the field supervisor position at MAST in Kansas City, I flew to Memphis and spent a week with the folks at Federal Express.
I’d love to hear where you’ve swiped good change ideas from and how you’ve used them in your own organization. And I think that it’s time that we hold a funeral for the old mental model of, “we will all measure the same thing the same way and see who is best and who sucks” approach to EMS benchmarking. I’ll buy the beer.
This article, originally published in February 2017, has been updated.
