NECO Lab | Universidad de Granada

CONECTA

Project title

Training, assessment and enablement of people unable to communicate effectively (CONECTA).

DIPRAL

Research objectives

Eye-gaze communication boards have been a major step forward for people with very reduced mobility, but they only work if the user keeps the head in a specific position and orientation in front of the sensor. This requires functional head control and an understanding of how eye tracking works. Some children and people with intellectual disabilities find it very hard to learn this, often over years and with an uncertain outcome, and people without head control currently have no alternative at all.

CONECTA proposes an immersive communication system: a communication board displayed in a virtual or augmented reality headset with eye tracking. The board follows the user’s head and gaze even when the user cannot control them, and intelligent algorithms distinguish the intention to look at a cell from involuntary or erratic movements. The project pursues three objectives:

  • O1. Design and develop a system able to train, assess and enable communication in people with communication challenges and restricted motility or impaired regulation of movement in space, who cannot use current augmentative and alternative communication (AAC) systems.
  • O2. Evaluate and validate the prototype with students of the partner school.
  • O3. Disseminate and transfer CONECTA to the community of users, families and professionals through publications, outreach events, multimedia material and social media.

The system

CONECTA is built from two paired subsystems that run on affordable commercial devices:

  • Board virtualization subsystem. An application for headsets with eye tracking (currently Meta Quest Pro, developed with Unity and OpenXR for portability across devices) that presents the communication boards in the immersive environment, reads the selected cells aloud and runs the training exercises.
  • Board management subsystem. An application for any standard tablet or phone from which the professional or caregiver loads the boards the user already works with, either from a photograph or exported from AsTeRICS Grid, and sends boards and exercises to the headset in real time over the local network.
CONECTA system

Together they provide three core functions:

  • Gaze and head training focused on stability, tracking, dwell and choice, designed with the school’s professionals to shorten the learning curve of communication boards and ease an eventual transition to conventional eye-gaze devices.
  • Communication enablement for users whose physical conditions doesn't allow them use the traditional AAC systems, for whom CONECTA becomes a stand-alone alternative communication system.
  • Assessment, recording the most relevant metrics of every session for longitudinal analysis, with a level of detail that helps professionals discover each user’s patterns.

Collaborators

CONECTA is a multidisciplinary project. Researchers from the Neural Engineering and Computing Lab (NECO Lab, CITIC-UGR), with long experience in biosignal processing, human-machine interaction and studies with people, work together with the professionals of the special education school of Hospitalarias Fundación Purísima Concepción in Granada, a reference and support centre recently awarded the Princess of Girona Foundation School of the Year prize. Its physiotherapists, speech therapists, occupational therapists and special education teachers define the design requirements and take part in the validation with their students.

Impact

CONECTA is aimed at people with communication challenges together with problems of head control or learning difficulties, a profile present in conditions such as cerebral palsy, amyotrophic lateral sclerosis, Rett syndrome, Angelman syndrome or acquired brain injury, among others.

An impact study carried out at the partner school among its 113 students found that CONECTA would give 20 students who currently have no gaze-based access the chance to train it, and would improve the training of 9 more: 26% of all students. Students with autism spectrum disorder or attention difficulties could also benefit by working on attention and task persistence, so nearly 70% of the school’s students could benefit from the project.

Principal investigator

Dr. Eng. Assoc. Prof. Miguel Angel Lopez Gordo (malg@ugr.es)
Neural Engineering and Computing Lab (NECO Lab, CITIC-UGR)
NEWTech research group (TIC-268)
Dpt. Signal Theory, Telematics and Communications
University of Granada