A Development and Evaluation Process for mHealth Interventions: Examples From New Zealand

"This article aims to describe the steps and themes in the mHealth development process, using the examples of a mobile phone video messaging smoking cessation intervention and a mobile phone multimedia messaging depression prevention intervention, to stimulate discussion on these and other potential methods."

From the abstract of this Journal of Health Communication: International Perspectives Supplement article: "The authors established a process for the development and testing of mobile phone-based health interventions that has been implemented in several mHealth interventions developed in New Zealand. This process involves a series of steps: conceptualization, formative research to inform the development, pretesting content, pilot study, pragmatic randomized controlled trial, and further qualitative research to inform improvement or implementation. Several themes underlie the entire process, including the integrity of the underlying behavior change theory, allowing for improvements on the basis of participant feedback, and a focus on implementation from the start. The strengths of this process are the involvement of the target audience in the development stages and the use of rigorous research methods to determine effectiveness. The limitations include the time required and potentially a less formalized and randomized approach than some other processes...."

As stated here, "There is a growing body of evidence suggesting that mobile phones can support behavior change." University of Auckland mHealth interventions include:

• text messaging for smoking cessation
• STUB IT, a video messaging intervention based on social cognitive theory
• video diary vignettes from role models (which provided opportunities for observational learning of behaviour change techniques, social support, and altering norms); and
• a depression prevention intervention for adolescents developed using multimedia mobile phone messaging, based on cognitive behavioural therapy and previous depression prevention work.

The development and testing began with a process similar to intervention mapping (a protocol for selecting theoretical foundations and translating theory into health promotion materials) that included the development of SMS (text)-length messages, 30-second video clips, scheduling and alerts, and mobile websites. This process included input from marketing experts. Focus groups of appropriate audiences were used in the formative research to comment on the messages. An online survey was used in this stage for determining both message content and frequency.

Student short videos were solicited. They were pretested among various populations. As a result researchers found that not all participants related to each story context or message presenter, leading to the conclusion that messages could be repeated in different contexts and formats as long as they adhered to: realism, positivity, and clarity.

The purpose of qualitative follow-up was to obtain a more in-depth understanding of how participants used the intervention, aspects that were helpful, whether individual messages were understood and how they affected behaviour, and feedback indicating whether the theory underlying the intervention was consistent with the effect of the intervention for participants. "For example, STUB IT participants reported they felt less isolated and more supported, like they were not quitting alone, and better equipped to quit next time. This was considered to be supportive of the use of role models providing observational learning, social support, and enhancing self-efficacy. This was used to inform subsequent mHealth development, including a mHealth exercise support program for people with cardiovascular disease."

The follow-up was used not only to adjust message programmes, but also to inform funders, stakeholders, and providers about the potential for implementation. If scaled up implementation is planned, researchers advise building in impact evaluation, including measuring reach: who takes up the service and whether or not it is reaching those who need it. "The implementation of STOMP as a national service included a 3-year evaluation plan. A report on the first year outlines uptake by different groups and includes quit rates measured in a way that allows comparison with other cessation services."

The article concludes that more process evaluation would benefit mHealth intervention development.