EMTs and paramedics’ responses to an NAEMT survey on ePCR software points to opportunities to improve usability and training. The survey report, 2017 National Survey on EMS ePCR Usability, describes the factors impacting perception of ePCR usability, the importance of training and the need to train EMS providers toward proficiency.
The 18-question survey adapted questions from the Computer System Usability Questionnaire, an assessment created by IBM in 1993, which is used to assess the usability of computer software. Alexander Garza, MD, led the collection effort and data analysis on behalf of the NAEMT. The results were based on 3,160 completed questionnaires, with nearly three-quarters (74 percent) of the responses coming from paramedics. More than one hundred different software programs from 73 different vendors were reported in the survey, but just six vendors (not identified in the report) were used by 75 percent of the respondents.
Garza reported an overall usability score, across all software and all respondents, of 4.5 out of 7. Usability scores were also examined based on the specific software, system usefulness and interface quality. Only two user traits — type of training and social media experience — predicted with higher usability scores.
Top quotes on ePCR usability
“Usability is the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specific context of use.”
- International Organization of Standardization.
“This is strong evidence that EMS professionals who routinely engage in electronic social media platforms find ePCR software more usable because they are more familiar with how software works and have more skills at using software for their needs than those who do not.”
- Alexander Garza, MD
“I have not been made aware of any EHR (electronic health record) implementation program that failed because of too much training, but I know of a number that have occurred because of too little.”
- Jason Mitchell, MD, director for the Center for Health Information Technology
Top takeaways to improve ePCR usability
The 12-page report is well-organized, expertly written and worth reading in full. Garza introduced the project and the adaptation of computer system usability at EMS World Expo 2016. After listening to his presentation in New Orleans and reading the report, here are my top takeaways.
1. Champions predict success
Any software implementation is more likely to succeed if it is supported by field personnel that understand the application, how to best use the application, advocate for adoption and can provide just-in-time training to their peers.
“These ePCR ‘champions’ shouldn’t be expected to do this work in addition to all of their other responsibilities, but should have time set aside for helping others,” wrote Garza.
The best champions aren’t selected because they have the highest rank. Much more important is to find the go-to technology person that likely already exists on many shifts and in many stations.
2. Training is good, necessary
Garza found that training type impacted usability scores. Not surprisingly, the lowest usability scores came from respondents who had no training. The best usability scores were from respondents who had classroom instruction and software interaction.
“It’s important for providers to be able to interact with the software so that they feel more comfortable with it once they go out to the field, and they will feel like the software is more usable,” wrote Garza.
Usability of ePCR is impacted by the type of training given to EMTs and paramedics. (Photo/NAEMT)
Software educators and training officers need to apply what we already know from clinical education. The best predictor for field success is building toward competency through self-directed learning, lecture, case discussion and facilitated high-fidelity simulation. Apply a similar sequence to software instruction to improve usability.
3. Train toward a goal
More than a third (38 percent) of respondents said they were certified before using the ePCR software in the field, which may have meant completion of class or passing a test. In any discipline, a specific skill, such as ePCR usage, exists within a larger need. Usability of ePCR software has a dramatic impact on the quality of data captured by an EMS agency for its quality improvement, billing and reimbursement efforts.
“The fact that so few EMS agencies have designated a specific level of proficiency for ePRC use indicates that not enough emphasis is being placed on quality data collection,” wrote Garza.
For EMS data collection the axiom, garbage in equals garbage out, means that our efforts to improve quality of care, provider proficiency and patient’s experience are likely stymied by the data captured with each patient contact.
4. Substantial room for improvement
Like a lot of surveys, many EMS providers probably agree that we didn’t need a national survey to tell us ePCR software has substantial room for improvement. Garza has it right when he describes the need for software vendors to improve the user experience, conduct usability studies and constantly improve the products for EMTs and paramedics.
