Project title

Assessment of multi-sensory environments interventions by means of neurometry and multimodal-extended reality (XENSORY).

Overview

MSEs are isolated spaces where controlled sensory stimulation (e.g., lighting and sound effects, sensory activities and others) is presented to users. They are intended for a broad range of impairments (cerebral palsy, ADHD, Down syndrome, stroke, post-traumatic stress, Alzheimer, Parkinson, autism, etc.); professionals (psychologists, occupational therapists, neuro-rehabilitators) and it is majority present in schools for children with special needs, associations and institutions related to disabilities, physical and cognitive impairment and well-aging at large.

Despite literature and professionals have reported, by means of surveys and standardized observation scales, multitude of empirical evidences of a wide range of positive improvements associated to SMS interventions, little has been reported about the subjacent brain activity that supports the improvements. Moreover, MSE interventions lack of objective quantification (e.g., by means of automatic tools) and methodology. In addition, MSEs are not exempt from other important limitations, such as they are expensive, not available everywhere and not practicable for people with reduced mobility (e.g., people prostrated on bed)

Research objectives

In this project, we propose two main research actions:

The first one is the use of Neurometry techniques for the online assessment of the cerebral activity of MSE users during interventions. A closed-loop wearable Brain-computer interface (BCI) will be implemented to report endogenous and valuable information from brain activity to better understand the behavioural improvements, thus providing scientific evidences of the rationale of SME effectiveness and the capacity of online monitoring.

The second one is related to the limitations outlined in the previous paragraph. We hypothesize that interventions with a highly-immersive-virtualized MSE could produce similar results as those of a real MSE. For this purpose, we propose the use of a multimodal approach based on Extended Reality (XR) (HMD and eye-tracker) and a wearable BCI to reproduce MSE interventions and compare with the traditional ones. After providing scientific evidence of the convenience of the multimodal XR-based-MSE approach, the basis for a low cost, globally available and practicable by people with reduced mobility will be established.

Collaborators

PhDs and experts from San Rafael School for Special Education, Hospitaller Order of St. John of God (SR-SJG) and the technology-based company Eneso, with expertise in MSE interventions and the design and commercialization of MSEs respectively, will participate defining the intervention protocols and the technical requirements of the XR-based MSE.

Impact

Institutions (early care centers, schools for special needs, public and private associations, and geriatrics) and also home-based-therapy users can benefit from the outcomes of XENSORY, either as virtualized interventions or by means of new paradigms generated as a result of the generated knowledge.

XENSORY is proposed for Strategic action 1, Health (National Plan for Scientific and Technical Research and Innovation 2021-2023) and covers these two main priorities actions: Health throughout the entire life cycle (“especially including populations in stages vulnerable groups”), and Tools, technologies and digital solutions for health and care (“promoting the development and use of innovative techniques, technologies and tools to improve the quality of life”).

Datasets

Data are available from the principal investigator upon request.

VR-COL: Dataset containing EEG recordings acquired during an auditory sustained attention task performed within a Virtual Reality environment featuring a single-color thematic scene (blue or red). The task comprised five Spanish C2-level listening tests, with pre- and post-task resting-state baseline data recorded. Event markers and participant response data collected during the study are included in the dataset. These data enabled the generation of comparative tables and figures between the two color conditions (blue and red), as well as tables and figures detailing extracted EEG features. The studies utilizing this dataset are presented below:

• Ávila-Muñoz, G., López-Gordo, M. A. & Rodríguez-Álvarez, M. Influence of Color on Academic Performance: A Studio with Auditory Sustained Attention Within a Virtual Scenario. In Artificial Intelligence for Neuroscience and Emotional Systems (eds Ferrández Vicente, J. M., Val Calvo, M. & Adeli, H.) 528–538 (Springer Nature Switzerland, Cham, 2024). doi:10.1007/978-3-031-61140-7_50.
• Ávila-Muñoz, G., López-Gordo, M. A. & Rodríguez-Álvarez, M. Influence of Red and Blue Virtual Environments During a Listening Evaluation: A Preliminary Study of Academic Performance and Subjective Self-Perception. In Innovative Technologies and Learning (eds Wang, W.-S., Sandnes, F. E., Lai, C.-F., Sandtrø, T. A. & Huang, Y.-M.) 317–323 (Springer Nature Switzerland, Cham, 2025). doi:10.1007/978-3-031-98185-2_33.
• Ávila-Muñoz, G., Lopez-Gordo, M. Á. & Rodríguez-Álvarez, M. Exploratory Study of Color Influence During an Academic Auditory Sustained Attention Test and the Implications for Educational Environments. Int. J. Neural Syst. 36, 2650040 (2026). doi:10.1142/S0129065726500401.

Click here to download the descriptive document.

VR-MSE: Dataset containing EEG data of a Virtual Reality replication of a multisensory intervention conducted in previous studies [Poza, J., Gómez, C., Gutiérrez, M. T., Mendoza, N. & Hornero, R. Effects of a multi-sensory environment on brain-injured patients: Assessment of spectral patterns. Med. Eng. Phys. 35, 365 (2012) and Gómez, C. et al. Characterization of EEG patterns in brain-injured subjects and controls after a Snoezelen intervention. Comput. Methods Programs Biomed. 136, 1–9 (2016)]. This dataset includes EEG data recorded during the pre- and post-intervention resting states, as well as during the four phases of the intervention: bubble tubes, fiber optics, disco ball, and cloud projector. These data enabled the preparation of a manuscript presenting tables and figures that analyze the effect of the intervention through within-subject relative changes anchored to the pre- and post-intervention resting states. However, the data collected throughout the intervention can also be utilized to examine the precise temporal evolution of responses to each stimulus. In addition to the EEG recordings, the dataset incorporates continuous event markers logging all experimental events. This dataset was utilized in a manuscript accepted for publication in the International Journal of Neural Systems, which is currently ahead of print.

• Ávila-Muñoz, G., López-Gordo, M. A., Rodríguez-Álvarez, M., Gómez, C. & Poza, J. Development and EEG Assessment of a Virtual Reality Replica of a Prior Real Multisensory Environment Intervention. Int. J. Neural Syst. 2750035 (2027). doi:10.1142/S0129065727500353 (accepted).

Click here to download the descriptive document.

Principal investigator

Dr. Eng. Assoc. Prof. Miguel Angel Lopez Gordo (malg@ugr.es)
Brain-computer Interface Lab (CITIC-UGR)
Smart Wireless Applications and Technologies Group (SWAT)
Dpt. Signal Theory, Telematics and Communications
University of Granada