Physiological-based Dynamic Difficulty Adaptation in a Theragame for Children having Cerebral Palsy
Defended by: VERHULST Adrien
Internship co-ordinator: YAMAGUCHI Takehiko (TUS)
Internship supervisor: RICHARD Paul (UA)
This research have been done in the context of the Children Rehabilitation Project (CRP), for LARIS (UA), at Harada’s laboratory (TUS)
The purpose of this research is to provide a physiological-based Dynamic Difficulty Adaptation (DDA) able to be used in a CVA rehabilitation application, and more especially in the Children Rehabilitation Project (CRP). The CRP is an adaptive modular interface destined to CVA’s rehabilitation for Children, and has for purpose to offer a state-of-the-art rehabilitation platform to improve CVA’s rehabilitation through rehabilitation applications. During this research, we successfully managed to implement a physiological-based DDA (using Anxiety state and Boredom state) able to direct the user toward the Flow state. We still need to experiment it on children having CVA’s rehabilitation.
In this report, we go through all the steps of this research, by going through (1) the state of the art done on DDA and on physiological-based DDA, (2) the approach based on physiological signals, (3) the acquisition of physiological signals, (4) the extraction of the physiological signals’ features and (5) the training of a learning algorithm able to dynamically change the difficulty.
Keywords: Theragame, Cerebral Palsy, Rehabilitation, DDA, Children Rehabilitation Project.
This one-page report just go through the important steps of the research. It has not for purpose of being exhaustive in any ways.
To do a physiological-based DDA we decided to focus on the emotional state of the user, and more especially on the Anxiety and Boredom emotional state. Through Anxiety / Boredom states (and the Flow theory) it is possible to know "where" the user is regarding the Excitement state. Our approach was to propose a DDA able to (1) recognize, in real-time, if the user was in Anxiety / Boredom; (2) adjust the difficulty to avoid Anxiety / Boredom and meet Excitement.
Physiological signals acquisition
The step #1 of our research was to capture physiological in real-time, in Unity3D. We decided to use the BITalino Board, and developed a C# SDK (.NET 2.0) to use BITalino in Unity3D. We successfully implemented it in Unity3D. We show bellow some of the physiological signals captured by our Unity3D scripts.
Physiological signals' features extraction
The step #2 of our research was to extract the physiological signals' features, in real-time, in Unity3D. Based on our state-of-the-art, we developed several MATLAB scripts to extract EDA / ECG features from EDA / ECG physiological signals. We then exported the MATLAB scripts in a C-shared library and successfully implemented it in Unity3D. We show bellow a sample of the features.
Classification of labelled features in affect-state
The step #3 of our research was to (1) label the physiological signals; (2) train a learning classifier to classify non-labelled physiological signals. We captured the users' physiological signals when they were in the Anxiety or Boredom state (through ProvokeAffect). We then extracted the labelled physiological signals' features and used them to train a SVM to recognize Anxiety / Boredom state (we managed to have a success rate of 67%). We successfully implemented it in Unity3D.
Implementation in the CRP
The last step of our research was to develop a physiological-based DDA, and to implement it in the CRP. We decided to develop a simple DDA (which basically increases the difficulty when there is Boredom and decrease the difficulty when there is Anxiety) able to "move" the difficulty from 0% to 100%. When then simply linked the difficulty's variable of the CRP to the difficulty's value. For example, if the user is anxious at a difficulty of 50%, the difficulty decreases from 50% to 40%, over 30 seconds. While the difficulty decreases the: (1) speed of the object decreases, (2) size of the object increase, (3) interval decreases, (4) distance increases.
We saw how we choose to implement our AFFECT-DDA into the CRP. We successfully managed to links all our steps in the application, and therefore to do our approach. However, in order to answer our research question (what is the incidence of a physiological-based DDA on a Children-with-Cerebral-Palsy rehabilitation application?) and to validate our approach it is necessary to test the system with Children with Cerebral Palsy, probably in the Hospital of Angers.