Ricerc@Sapienza

The unprecedented spread of mobile technologies, along with recent advances in their sophistication and innovative health-related applications, led to the development of a new electronic-Health field, known as mobile-Health (mHealth).
MHealth technologies offer real-time monitoring of health outcomes to detect changes in health status and support the adoption and maintenance of a healthy lifestyle. Furthermore, they provide a great platform to deliver Just-In-Time Adaptive Interventions (JITAIs) facilitating behavioural changes and supporting the development of precision medicine.
Despite the growing popularity of JITAIs, there is a lack of guidance on constructing high-quality evidence-based JITAIs, and mHealth poses some unique challenges that preclude direct application of existing statistical methods.
The first attempt to bridge the gap between the enthusiasm for delivering mHealth interventions and the lack of adequate statistical tools was proposed by Lei, Tewari and Murphy (2017). They modelled the decision-making problem as a contextual multi-armed bandit (MAB) problem, assuming a linear reward model. Motivated by the untenable assumption of time-invariant linear model, Greenewald et al. (2018) relaxed the MAB to a non-stationary and non-linear reward. However, its performance is guaranteed under restrictive conditions on the action choice probabilities, it is not suitable in a delayed feedback scenario, which is a key feature of mHealth, and the learning process is relatively slow.
In an attempt to contribute to the immature mHealth statistical literature, with this proposal we aim to extend the contextual MAB model to a delayed reward setting, speed up learning by pooling data across multiple users, and eventually evaluate its performances in comparison with the exiting methods in both a simulation study and a real data analysis based on the Yahoo! news recommendation data. Overall, our work might benefit both the healthcare and statistical community.