The unprecedented spread of mobile technologies, along with recent advances in their sophistication and innovative health-related applications, led to the development of a new electronic-Health field, known as mobile-Health (mHealth).
MHealth technologies offer real-time monitoring of health outcomes to detect changes in health status and support the adoption and maintenance of a healthy lifestyle. Furthermore, they provide a great platform to deliver Just-In-Time Adaptive Interventions (JITAIs) facilitating behavioural changes and supporting the development of precision medicine.
Despite the growing popularity of JITAIs, there is a lack of guidance on constructing high-quality evidence-based JITAIs, and mHealth poses some unique challenges that preclude direct application of existing statistical methods.
The first attempt to bridge the gap between the enthusiasm for delivering mHealth interventions and the lack of adequate statistical tools was proposed by Lei, Tewari and Murphy (2017). They modelled the decision-making problem as a contextual multi-armed bandit (MAB) problem, assuming a linear reward model. Motivated by the untenable assumption of time-invariant linear model, Greenewald et al. (2018) relaxed the MAB to a non-stationary and non-linear reward. However, its performance is guaranteed under restrictive conditions on the action choice probabilities, it is not suitable in a delayed feedback scenario, which is a key feature of mHealth, and the learning process is relatively slow.
In an attempt to contribute to the immature mHealth statistical literature, with this proposal we aim to extend the contextual MAB model to a delayed reward setting, speed up learning by pooling data across multiple users, and eventually evaluate its performances in comparison with the exiting methods in both a simulation study and a real data analysis based on the Yahoo! news recommendation data. Overall, our work might benefit both the healthcare and statistical community.
We are entering a new era of healthcare, in which personalized medicine along with electronic and mobile health occupies a central role, and technological sophistication and capacity has now provided the platform for delivering JITAIs. Thus, it is critical that researchers develop nuanced health behavior theories capable of guiding the construction of such interventions, and, at the same time, that the statistical, and more generally the quantitative community, build sophisticated approaches adapted to the specific mHealth field.
Many statistical challenges arise when constructing high-quality JITAIs and, being a very immature ground, these challenges have been poorly or even not yet addressed, and many advances are still to me made.
With our proposal, we aim to extend the current statistical mHealth literature, prevalently based on contextual multi armed bandit, by developing a novel approach which integrates the scheme of a contextual MAB with specific modelling tools meant to address one of the emerging challenges in mHealth, i.e., the delayed reward. Optimizing JITAIs in such a way will not only encourage users to make the right decision "in the moment", impacting thus their near future, but will also balance between proximal outcomes and delayed outcomes or rewards, ensuring a healthy behavior in the long run.
In addition, we propose to speed up the learning procedure by pooling data across multiple users, as to our knowledge all learning procedures proposed so far are based on the solely user's history.
Overall, our work will benefit the healthcare community, clinical and non-clinical, by providing a refined (preventive) healthcare education, a better assistance and monitoring of healthy user behaviors, an improved management of chronic diseases and a better support to clinical decisions, including both diagnostics and treatment recommendations.
Moreover, it will contribute to the general literature where sequential decisions are to be made, such as website optimization, clinical trials, adaptive routing and financial portfolio design.
Eventually, we would also like to point out that mHealth is being touted as a cost-effective solution to a number of the problems commonly associated with traditional chronic disease management interventions. As a such, an improved strategy for delivering JITAIs, will not only translate in better long-term health outcomes, but also in a reduction of the cost associated with care.