Peripheral nerve blocks (PNBs) are an appealing technique in day surgery patients because of their efficiency, reduced cost, and excellent perioperative pain management.1 Nevertheless, a recent study showed PNB patients were more likely to experience severe pain and to return to hospital for pain control than those who received a general anesthetic,2 findings consistent with other published results.3,4 Although patients are instructed to begin oral analgesia before their PNB resolves, many struggle with medication management; this may be because of an incomplete understanding of instructions, dosing difficulties, or poor motivation to take medication while a residual blockade remains.2 Critically, our clinicians report that many patients have poor recall of when the PNB wore off, confounding follow-up assessment.

Smartphone applications (apps) show promise in increasing therapy adherence.5 Patient-facing apps have been described and implemented for reducing anxiety during induction of anesthesia for children,6 self-management of asthma7 and diabetes,8 adherence to an enhanced recovery after surgery protocol,9 and burn recovery.10

Panda (pain assessment via a novel digital app), an app developed by the Digital Health Innovation Lab at British Columbia Children’s Hospital, is usable and feasible for parents managing their child’s postoperative pain.11,12 This app has undergone extensive testing with patients’ parents, patients, and nurses. A version of the app, Panda-Nerve Block (hereafter “Panda”), was created for adult PNB patients and includes features for monitoring a PNB. The app includes both the visual analogue scale (VAS) and numeric rating score (NRS) pain scales13,14; accordance between the paper and smartphone versions of these scales has been previously shown.15 We aimed to assess the usability and feasibility of Panda in adult day surgery patients who received a single-shot PNB.

Methods

Study design

The primary objective of this study was to show the feasibility of Panda with patients at home; the app was considered feasible if at least 50% of alerts were responded to by users before the next alert occurred. Additionally, alert response time within one hour was used as a measure of adherence. Our secondary objective was to assess usability and identify issues that arose at home to make iterative improvements to the app between rounds of testing. Feasibility is an assessment of whether or not patients will use Panda in practice. In this study, feasibility was measured by adherence to alerts and patient-reported satisfaction with the app. Usability is a measure of how easy and intuitive Panda was to use and whether there were issues that led to user confusion or undesirable outcomes.

Three separate rounds of testing were conducted in December 2017, February 2018, and May 2018, lasting for approximately two weeks each. This protocol was approved by the University of British Columbia – Providence Health Care Research Ethics Board (REB Number H17-01365). We aimed to recruit ten patients per round for a total of 30 patients. This sample size was selected as it has been used in similar studies.11,12,16

Selection and description of participants

Patients were recruited in the surgical daycare unit and surgical procedure room, an ambulatory surgery unit at St. Paul’s Hospital, a tertiary academic site in Vancouver, Canada. The inclusion criteria were American Society of Anesthesiologists physical status I–III, age 19–75 years, undergoing ambulatory surgery under PNB, and anticipated post-surgical pain. Patients were required to use their own iOS or Android device, to be discharged on the same day as their surgery, and have at least two days’ planned post-discharge analgesic medication. Patients were excluded if they were unable to follow study instructions and complete questionnaires in English, did not have an iOS or Android device, or if there was a significant cognitive impairment, hearing or visual impairment, neurologic injury, or psychomotor dysfunction that precluded app usage. Eligible patients were approached by study personnel after being identified on the operating room slate. Written informed consent was obtained from each participant.

Panda app

The original Panda app has been described previously.11 It allows parents to schedule alert reminders in concordance with their child’s prescription (Fig. 1). Several new features were designed specifically for Panda.

Fig. 1
figure 1

Panda-Nerve Block home screen. PANDA = Pain Assessment via a Novel Digital App

Full size image

Initial configuration of the app includes demographic information, surgical information, and, specific to Panda, block information (location, presence of a catheter, concentration and volume of local anesthetic injected, and any adjuvant medication). Compared with the pediatric app, the Panda user is the patient rather than a parent. Both the NRS and VAS are available to users for pain assessment.

The pain check function was modified from the original Panda app to include questions about the quality of the patient’s PNB. At each alert, patients are asked if their operative site has motor weakness and if the sensation is numb, tingly, or has returned. Patients are then asked where their pain is and to rate their pain. Patients may perform a pain and block check in between scheduled alerts, as well as log any breakthrough medications. These questions were determined by expert consensus at our institution and were based on our routine follow-up questions.

A tutorial and demo mode were available for practice. The upload function uploaded all the patient’s Panda usage data to a Research Electronic Data Capture (REDCap) database on BC Children’s Hospital Research Institute servers.17

Data collection

Patients completed a pre-study questionnaire pertaining to past experiences with smartphones and apps (Appendix 1). Participants were assisted in downloading, installing, and setting up the app on their own device. To assure roughly equal numbers of patients testing the VAS and NRS, patients were assigned to either scale in a non-randomized, alternating fashion; however, users were still able to switch scales if they preferred. To be consistent with our institution’s discharge directions given to PNB patients, all prescribed analgesic medications were scheduled as a regular medication for the first 24 hr post-discharge regardless of how the prescription had been written; medications were subsequently scheduled as either regular or PRN as prescribed by the surgeon. All participants watched a tutorial video and practiced responding to a mock Panda alert.

Participants used Panda at home to manage their pain and track the regression of their PNB, including the presence or absence of complications such as paresthesias. Participants were asked to use Panda for a period between two and seven days. The identifying information, selection of pain scale, and details of the block were locked from changes by the participant; however, participants could add, change, or remove medications from the schedule. Alerts scheduled at inconvenient times, such as during sleep, could be cancelled. All user data and activity within Panda were tracked and uploaded to REDCap.

Participants were contacted by phone or email 48 hr after discharge as a check-in. Additionally, patients could contact the research assistant for help throughout the study. The anticipated last day of usage was determined by the patient on postoperative day 2, but they could contact the research assistant if they finished using Panda sooner than expected. A standardized 14-question telephone interview was administered by the research assistant on the patient’s last day of app use; a shorter five-question version was available for patients who did not have enough time for the full questionnaire and was used with one patient (Appendix 2). These questions were selected to identify major barriers to use and the perceived utility of the app for postoperative pain management. Participants additionally completed a standardized Computer Systems Usability Questionnaire (CSUQ) to assess the app’s ease of use.18 Upon completion of the study, Panda was remotely deactivated and patients were instructed to delete the Panda app from their device.

Data analysis

After each round, telephone interviews and CSUQ responses were analyzed for common themes. Themes were used by the Panda software developer to direct modifications to Panda before the next round of the study. Qualitative analysis was performed using NVivo 12.2.0 (QSR International Pty Ltd., Melbourne, Victoria, Australia).

Each user’s audit log was analyzed for the following values: the total study time, the median response time, the percentage of alerts to which a response was logged prior to the next alert in both the first 48 hr of app usage and throughout the entire period of app usage, the proportion of logged responses that occurred within 60 min, the median pain score (either VAS or NRS), the percentage of pain scores that were logged as no pain (either a response of “no pain” at the pain check prompt or a value of 0 on the scale), the number of times an analgesic medication was taken (defined as a non-steroidal anti-inflammatory drug, acetaminophen, or an opioid medication), and the median interval between analgesic medication administration. A one-hour response time was selected as it was felt to be the maximum allowable delay to still be considered compliant with the medication regime; this cut-off has been used by our group in the past.11,12 These data were collated for each individual round; the median [interquartile range (IQR)] of each parameter was calculated and the Kruskal–Wallis test was used to compare data between rounds. A P value of less than 0.05 was considered significant. Calculations were performed using R Studio 1.1 (RStudio Inc., Boston, Massachusetts, USA).

Results

Cohort characteristics

Forty-one patients consented across three rounds (Fig. 2). Twenty-nine participants were analyzed in full.

Fig. 2
figure 2

Patient recruitment across all three rounds

Full size image

Group characteristics can be seen in Table 1. Most patients reported having high or medium smartphone knowledge. The largest age group consisted of users over the age of 50. No patients used an electronic method to keep track of medications already taken.

TABLE 1 Study cohort characteristics
Full size table

App usage

Across all three rounds, the median [IQR] proportion of alert responses before the next alert was 68 [34–93]%. During the first 48 hr of app use, the median [IQR] proportion of responses before the next alert was higher at 83 [54–92]% (Table 3). Of the recorded responses, the median [IQR] proportion of responses within one hour was 87 [75–95]%. There were no significant differences between groups for any usage parameter nor were differences found in median pain scores, pain scores recorded as no pain, doses of analgesic medication taken, or median interval between analgesic medications taken (Table 2). Only one patient required app assistance.

TABLE 2 Audit-trail log data analysis and calculated usage parameters per study round
Full size table
TABLE 3 Compiled audit-trail log data analysis and calculated usage parameters across all three rounds
Full size table

Twenty patients (69%) completed the CSUQ. Across all three rounds, the median [IQR] response to “Overall, I am satisfied with this system” was 2 [1–3.5], equivalent to “agree”. The most negative response was to the statement “This interface gives error messages that clearly tell me how to fix the problems” (4 [3–4.5]), equivalent to “neutral”. The most positive response was to the statement “The interface is pleasant” (1.5 [1–2]). All other responses ranged between 2 and 3, corresponding to “agree”.

User feedback and user-directed changes

Panda elicited positive responses from users (Table 4). Patients found the app easy to use and would use it again for a future surgery, with many specifically commenting that they liked the scheduling and reminder functions. Most patients felt they could set Panda up independently.

TABLE 4 Qualitative analysis results
Full size table

Responses to the layout and aesthetics of the app were mixed. Several requested changes to the app could not be made, including a self-adjusting, dynamic medication schedule that accounted for PRN doses and modifications to the pain scales, such as graduations to the VAS.

Several barriers to use were identified. Some patients required extensive assistance with installing the app on their device (e.g., resetting passwords, connecting to WiFi). One patient had difficulties using Panda and stopped after two alerts. A small number of technical glitches were also reported by patients, such as freezing or problems with saving entries.

These themes formed the basis of the changes made to Panda between rounds. One persistent concern was the language of the nerve block check. While patients could originally choose that their limb felt “normal”, several found this confusing as it did not feel normal compared with their pre-surgical baseline. The question was changed to ask if sensation in the operative limb was “returning” for the second round, and further changed to “returned” for the third round with good satisfaction.

Several changes were made to the set-up process. The option to add multiple PNBs was added at the end of the first round to account for combinations of blocks. The ability to enter a custom medication name was also added after the second round to better accommodate patients with multiple medications.

Discussion

Smartphone apps are appealing for ambulatory surgery as they guide patients through their postoperative recovery from home. Our study shows Panda is feasible and useable for patients to manage postoperative pain and monitor the regression of their nerve block following day surgery. We attribute this to the rounds of testing and iterative improvements conducted across multiple studies.11,12

Most patients found the app useful and responded appropriately to alerts. Additionally, response rates were high in the first 48 hr post-discharge, capturing the critical transition from blockade to oral analgesia, including documentation of opioid usage and real-time nerve injury assessment.

Our app met our threshold response rate of 50% and had improved adherence rates over the pediatric Panda app, showing Panda’s feasibility.12 This may be because the user population was adult patients rather than parents or caregivers. Patients are inherently more aware of their symptoms and conceivably more motivated to use the app.

Several works on perioperative apps have been published.19,20,21 Warren-Stomburg et al. describe a postoperative pain app for ambulatory surgical patients. In contrast to our study, the implementation was complicated by low user adherence and high attrition.22 The authors hypothesized that this may have been due to a lack of feedback to the patient as well as user-side technical glitches.

Despite also not including a feedback system, usage of Panda was not affected by this. Conclusions about the differing adherence rates are challenging to draw because different metrics were used. The attrition rate may have been lower because Panda is a more comprehensive app that includes more features. Panda is also specifically developed for ambulatory PNB patients rather than all ambulatory patients, tailoring to specific needs of that population, again potentially improving buy-in. Additionally, Warren-Stomburg et al. did not perform follow-up, whereas we completed both a 48 hr check-in and a final study interview.

Barriers to usage

As also observed in the pediatric study,12 some patients requested a version of the app that would automatically shift the future medication schedule when they took their medication late. This would require a smart algorithm to prevent complications such as unsafe dosing intervals and medication overdose, which was beyond the scope of this initial usability/feasibility study. Requests to modify the pain scales, such as adding graduations to the VAS, could also not be accommodated as deviation from these scales would require re-validation.

Iterative improvements to the app did not significantly change its usability. The most common user critique, the phrasing of the nerve block check, was resolved by the last round. Most patients were able to install, set-up, and use the app; nevertheless, there were several patients who required extensive assistance in the initial set-up. While feasible in the context of a study, this degree of assistance is impractical for a busy clinician. With respect to the two patients who could not be enrolled because of technical issues, we were unable to reproduce these glitches, which may have been device-specific.

The lack of significant differences in usage metrics between rounds suggests that the initial version, though not completely optimized, was itself useable for patients, probably because many of the challenges in implementation were addressed in the original pediatric Panda study. User criticism was generally centred around aesthetic concerns rather than functionality, but this may have impacted user adherence and motivation. While our study is underpowered to detect subtle differences, statistically significant changes were seen between rounds in the similarly sized pediatric study.12

Limitations of data

A limitation of this study is the reliance on self-reported data. Attrition bias is also a concern: patients may have been lost to follow-up because they found Panda uninteresting or unhelpful. While not analyzed in full, multiple days of data were still submitted for these patients, suggesting they continued to use the app. The Hawthorne effect must be considered; outside of a study, patient adherence could diminish in clinical practice.

Although the VAS and NRS are well-established, and we have shown agreement between the paper and electronic versions of these scales in the recovery room, the electronic versions of these scales have not specifically been validated for at-home patients.13,14,15

Future directions

Our work is a step toward providing improved pain management at home for ambulatory surgical patients via apps. As we move toward implementing the app into routine postoperative care, we anticipate publishing Panda to the public iOS and Android app stores, simplifying installation. Moreover, we are developing additional features including a two-way communication system between clinicians and patients. Patients will be able to communicate directly to a clinician either through text, voice, or video messaging, including photographs of their surgical sites and peripheral nerve catheters.

In addition to its primary goal of improving pain management for ambulatory patients, Panda has potential for assisting in diagnosing PNB-related complications. Real-time postoperative follow-up does not just provide detailed records for clinicians but facilitates early diagnosis of complications. Through regular screening for concerning symptoms following a PNB, Panda can assist the early diagnosis of nerve injuries, compressive hematomas, and abscesses, on which more timely interventions can be delivered.

While effects on clinical patient outcomes were beyond the scope of our study, Panda could potentially play a role in reducing periods of severe pain and decreasing readmissions to hospital; we anticipate testing these hypotheses in a future randomized-controlled trial.