FirstWatch, supported by Prodigy EMS, has run a web series titled “Conversations that Matter” (CTM) for nearly two years, and over time it has had many discussions, using an open mic Zoom format, with many great industry thought leaders. At a recent session, I was joined by David M. Williams, PhD, senior improvement advisor of the Institute for Healthcare Improvement (IHI) and lead faculty for the Improvement Advisor Professional Development Program; and Mike Taigman, improvement guide for FirstWatch, to benchmark our industry benchmarks.
The collective force of Taigman and Williams are the genesis of performance improvement EMS, and the latest CTM encourages all to find and deploy benchmarks that are meaningful and contribute to overall system improvement.
The premise for the session was based on the original thoughts of Jack Stout, the father of high-performance EMS, who used to encourage EMS leaders to compare their performance to the best systems in the country. He said, “If you only compare to the systems in your area, how will you know if you’re not just the cream of the crap?” In this interactive session, Williams, who has applied the science of improvement worldwide, shared his thoughts.
Q. What is benchmarking, and when may it be helpful?”
Williams: The term benchmarking comes from land surveying and refers to a marker of a previously determined position, which can be used to set up a new position. A popular book by Robert Camp defines benchmarking as “the process of identifying and learning from best practices or best performance from any industry to identify potential changes for improvement.” Back in the 1990s, EMS systems expert Jack Stout wrote an article in the Journal of Emergency Medical Services advocating for the practice of benchmarking. He made the case that benchmarking offered two opportunities:
- It helped you see where your results compared with peers
- It helped to find exemplars who we can go learn from
Stout described these as lateral benchmarking and best practice benchmarking. Cares, AIMHI, Press Gainey and Viszient are all examples of lateral benchmarking. Michael Dell cautioned that these can be useful, but what if the benchmark isn’t the best? The Institute for Healthcare Improvement’s Research and Development process is an example of best practice benchmarking: it focuses on finding systems that have exemplar performance in an area and learning about the causal mechanisms (why something works well) more than the what. They develop a prototype to test and replicate results.
Q. Leaders believe that they don’t want to “reinvent the wheel” and that they should look for the best practice and copy that. You teach people to look for the best and be cautious that replicating results isn’t easy, and often the best answer isn’t known.
Williams: In best practice benchmarking, we want to go learn from the place where leaders are producing the best results. In healthcare, there are examples all over the place of systems able to nearly eliminate infections, medication errors, etc. These are the places we want to learn from. Most leaders do not want to waste time. They want to get results. If someone is getting the results, we should go learn from them. Where this often fails, is in two places:
- The system getting the results may not fully appreciate the key drivers that are contributing to their results – the causal mechanisms.
- Leaders try to copy the solution, which very often does not reproduce the results when installed in a different context and with leaders who were not involved in developing the changes.
What’s missing is a method for learning in a structured way. I use an approach developed using the Model for Improvement which creates the conditions for leaders to learn, extract key characteristics, and set them up to test them at home to develop an adapted version that works in your context and with your team. A good example of this is sudden cardiac arrest and Seattle. There are decades of evidence better survival is possible. Copying the system’s approach to another system is not likely to replicate the results. We do have examples of a host of communities that extracted the key drivers and then worked to develop their contextualized approach and achieved similar results over time.
Q. You have been an EMS consultant worldwide and an advisor to school systems, large health systems, and national governments in the United Kingdom, Europe and the Middle East. What themes do you see across your work in trying to make a large-scale improvement?
Williams: Common positive themes include motivated professionals, who want to do their best work, and who want to help people have positive outcomes. Common issues include:
- Second and third generation folks who grew up in the system of today with limited knowledge of the foundations of EMS systems … many don’t know why things are the way they are
- Wide variation, but mostly limited appreciation of evidence-based (research) and best practice
- Stuck in status quo mental models (speed, transport, etc.)
- Data-saturation with limited measurement knowledge
- Lots of best effort and reactionary practice and no method for problem-solving to results
- Groupthink that every EMS system is unique and getting caught in the complexity of their system … the fundamentals are often their biggest lever for design or redesign.
Q. You have been working on a book with two of the biggest names in quality. The book is built on improvement science and includes five activities for leaders as a method to pursue organizational excellence. How is this method unique, and what does it enable leaders to do differently?
Williams: Quality improvement starts at and is executed at the project level. We find systems that aren’t doing what we want them to do, and we design or redesign them to get the results we want. Many organizations and industries have adopted the Model for Improvement as their method for doing projects and a segment of those folks can execute constantly to get results.
Many leaders get fired up by mastering a method to get project results and want to expand that to their method for leading their organization. Sometimes this starts with organizations that are struggling and need to work their way out and sometimes this starts with organizations that are benchmarking well but know they can do better. Our method, Quality as an Organizational Strategy (QOS), is built on 100-plus years of quality and has been used by several organizations that have sought and been awarded the Malcolm Baldrige Award. QOS uniquely combines five leadership activities to create a system of improvement:
- Create clarity about the need the organization is here on earth to fulfill, then align purpose and vision, and develop practical tenants or values for how they will run.
- Learn how to understand and map the organization as a system of linked, inter-dependent processes and develop a vector of measures that serve as the vital signs.
- Develop a system for gathering information from the community, the industry, the customers and partners that is collected and analyzed to support learning and change.
- Use a planning process where a family of inputs is reviewed, and a method helps sort the vital few opportunities for impact. Leaders charter these for improvement projects.
- Learn and use rigorous improvement methods to design and redesign the organization and get results.
The process is not easy, but those that embark on the journey describe it as transformational. Leaders learn about their organizations in ways they had not before; they reveal lots of opportunities for improvement and prioritize tackling the big rocks; and they finally have a method for learning, problem-solving and getting results.
Q. This sounds like a lot. How do you start this with organizations? This sounds very different than the traditional expert model consulting journey we see in EMS where a consultant studies the system and generates the report.
Williams: There are many approaches to consulting. My approach blends bringing a foundation of improvement science – systems, variation, learning and problem solving, people – and the method of Quality as an Organizational Strategy with the local expertise of the leadership team of any kind of organization. I usually start light with a co-produced mini-learning experience with a leadership team to consider their purpose, review their current results and how successful they are at fixing problems, and then pick a handful of projects to start improving. It’s a little engagement at first, built so everyone learns. Leaders decide from there if they are ready to change.
The Conversations That Matter series takes place every month and further details of upcoming sessions and on demand viewing of previous sessions can be found at https://firstwatch.net/conversations/.
