(54) Protocol for Implementing and Evaluating Online Physician Prescribing Portraits: Scalable Low-Cost Audit & Feedback Delivery and Improvement

Publication Number: 385

Background: Audit and feedback are core components of quality improvement, but trials have been criticized for being uncoordinated, lacking innovation, and underpowered. To date, in British Columbia (BC), Canada, physician prescribing portraits have been sent by mail, a costly process that is difficult to repeat efficiently and to evaluate.

Objectives: To assess how the stages in operationalizing a secure online portal to provide up to 5,000 family physicians (FP) with personalized prescribing portraits, influenced evaluability.

Methods: Stage 1: To get data access permission, a security threat and privacy risk assessment was needed of a method to track when FPs accessed their portraits, without revealing their identities, yet allowing linkage to anonymous databases needed for impact evaluation.
Stage 2: In developing the portal infrastructure, including website design, server architecture, and data transfer processes, we needed a way to delay access to randomized control groups.
Stage 3: Online accessibility added new dimensions to iterative design of portraits via an interdisciplinary team with evidence review usability testing by a select panel of physicians: these processes should be able to continue after publication, with clarifications aided by feedback.
Stage 4: A simple authorization process for physician registrants (https://secure.ti.ubc.ca), creates a yes-no metric of engagement (registration), leading to a quantitative metric – number of webpages accessed.
Stage 5: Randomized promotions by email or mail (with each new Portrait) provide opportunities for low-cost trials of testing engagement innovations: e.g. 300 FPs were randomized to 3 groups, 2 with 6-month delays in access to their online portraits. Impact on accessing 4 other pre-loaded portraits will be assessed. Nonrandomized impact evaluations will compare FPs who access their Portraits during in-person educational sessions with non-participants.

Results: Online portraits created new channels for qualitative evaluation of ongoing feedback from FPs when they access Portraits by themselves or in educational sessions with their peers.

Conclusions: Online Portraits pave the way for combining randomized trials with cycles of quality improvement in audit and feedback methods.