Dissertation (Doctoral) Innovative Research Methods to Evaluate Effective Engagement with Consumer Mobile Health Applications for Chronic Conditions


Quynh Pham


Consumer mobile health applications (mHealth apps) hold enormous potential to provide effective, efficient, economical, and scalable chronic health care. The opportunities afforded by these interventions to positively disrupt traditional chronic care models are significant, but require a cumulative evidence base to inform policy and practice. Evaluating these interventions, however, presents unique challenges. Traditional research methods are poorly suited to accommodate the rapidly changing technology landscape, which requires mHealth apps to constantly evolve and update just to remain usable, let alone effective. Moreover, they cannot characterize the complex, multilevel, and temporally dense patterns of effective engagement required to realize improved outcomes. This program of research sought to (1) investigate the complexity of conducting innovative evaluations of consumer mHealth apps for chronic conditions, and (2) build a research analytics platform to streamline evaluations and optimize mHealth apps. We established in Study 1 that mHealth evaluation methodology has rarely deviated from common methods, despite the need for more relevant and timely evaluations. This finding led us to explore the potential of analytic research methods in Study 2, where we discerned that digital health innovators were gaining nuanced insights into engagement, but had not yet characterized effective engagement for improved outcomes. To support innovators with this effort, our research in Study 3 focused on designing, developing, and evaluating a resource to consolidate, analyze, and visualize analytic indicators of effective engagement. The Analytics Platform to Evaluate Effective Engagement with digital health interventions—APEEE for short—provided innovators with a means to characterize the breadth and depth of mHealth app engagement required to achieve intended outcomes. Motivated by the potential for APEEE to power data-driven research methods, we directed Study 4 toward studying the implementation of the platform—and research analytics more broadly—in digital health evaluative practice, and surfaced the rich complexity of engaging in this practice to evidence mHealth apps. Raising the standard of mHealth evidence may enable greater confidence in the causal effect of apps on improved chronic health outcomes. It is this opportunity afforded by innovative research methods to close the gap between promised and realized health benefits that is most meaningful.


Joe A. Cafazzo