Ricerc@Sapienza

In recent years, advancements in human-computer interaction (HCI) have enabled the development of versatile immersive devices, including Head-Mounted Displays (HMDs). These devices are usually used for entertainment activities as video-gaming or augmented/virtual reality applications for tourist or learning purposes. Actually, HMDs, together with the design of ad-hoc exercises, can also be used to support rehabilitation tasks, including neurocognitive rehabilitation due to strokes, traumatic brain injuries, or brain surgeries.

International contributions on advanced human-computer interaction with a special section on game-human interaction

Hand rehabilitation is fundamental after stroke or surgery. Traditional rehabilitation implies high costs, stress for the patient, and subjective evaluation of the therapy effectiveness. Mechanical devices based approaches are often expensive, cumbersome and patient specific, while tracking-based devices are not affected by these limitations, though they could suffer from occlusions. In recent works, the procedure used for implementing a multi-sensors approach, the Virtual Glove (VG), based on the simultaneous use of two orthogonal LEAP motion controllers, was described.

In previous papers, we proposed an extension of Spider Diagrams to object-oriented modelling, called Modelling Spider Diagrams (MSDs), as a visual notation for specifying admissible states of instances of types, and for verifying the conformance of configurations of instances with such specifications. Based on this formalisation, we developed a notion of transformation of MSDs, modelling admissible evolutions of configurations.

In this paper, a fully immersive serious game system that combines two Natural User Interfaces (NUIs) and a Head Mounted Display (HMD) to provide an interactive Virtual Environment (VE) for patient rehabilitation is proposed. Patients\textquotesingle~data are acquired in real-time by the NUIs, while by the HMD the VE is shown to them, thus allowing the interaction.