P0018 - Heuristics, Usability, and Workflows, OH MY! Insights into the Migration of a Paper Based Workflow to a Digital Platform.

Backround: Borycki, Kushniruk, & Carvalho (2013) proposed a methodology for validating their safety heuristics using clinical simulations. This work was a followup to Carvalho, Borycki, & Kushniruk (2009) work on developing evidence based safety heuristics for evaluation of health information systems. In their study they proposed that heuristics were limited and simulated clinical situations were needed to better understand how the system would be used in the flow of a healthcare professional. This present study applies this strategy. Using heuristics, usability and then workflow analysis, a formerly paper process was transitioned electronic EHR documentation.

Methods: The primary objective of this study was to compare findings from heuristic evaluation, usability testing and workflow analysis of an electronic documentation process for a clinical team huddle.

Phase 1
Heuristic evaluation by 4 evaluators. The heuristic evaluations were then compared and reconciled to identified usability issues, assigned violations, and heuristic severity ratings .
Phase 2
Usability evaluation was performed through direct observation and screen recording via Morae. 4 hospitalists performed the 5 tasks to complete the electronic documentation. The Single Ease Question (SEQ) was asked after each task set. A comparison was then made with the results of heuristic analysis looking for predicted issues.
Phase 3
Observed Workflow simulation: 8 interdisciplinary teams, real time direct observation of new EHR workflow, video recorded and reviewed for usability observing for predicted and non-predicted events. NASA-TLX given to all participants with group debrief after each session.

Results:
Phase 1 Heuristics
Multiple level major (3) usability problems were found with heuristic analysis. No catastrophic (4) usability problems were found.
Phase 2 Usability Study
The usability confirmed 5/6 of the issues predicted through heuristics.
Phase 3 Clinical Simulation
The clinical simulation validated 4/6 of defined usability issues. The 2 unanticipated related to team dynamics.

Conclusion: The heuristics and usability allowed early mitigation of usability issues. However, additional issues were uncovered during simulations not predicted by heuristics or usability. These issues were around team communication and dynamics workstation access. All three phases of evaluation were useful. The usability testing identified additional issues. The workflow analysis through simulation help identify additional issues surrounding team dynamics and unintended consequences of the team’s workflows. The three phases of evaluation allowed for improvement of the usability of the documentation tool and strategies for rollout that would minimize end user frustrations and increase early acceptance. The next stage of this project will look at the adoption of the tool into daily clinical work.