Interactive sonification is an effective tool used to guide individuals when practicing movements. Little research has explore the affordances of continous and discrete sound structures in supporting motor therapeutic interventions for children with autism who exhibit motor impairments. The goal of this research is to study if children with autism understand the use of such sounds during motor therapeutic interventions, its potential impact of interactive sonification in the development of motor skills in children with autism, and the feasibility of sonifying an elastic display and a mobile phone to be used in specialized schools for children with autism.
Hypothesis: We hypothesize continuous sounds will provide better guidance for the trajectory of the moment while continuous sounds will encourage repetitions independently of the technology being used including an elastic display and a mobile phone.
Keywords: Elastic display, depth camera, accelerometer, mobile sensing, sensors, interactive sonification, movement, ASD, user studies, children, motor coordination, rehabilitation
Sound structures
Push movement
- Initial position: The child brings his/her arms to her chest
- Trajectory: The child stretches an arm forward in a horizontal way (i.e., parallel to the floor)
- End position: The child has an arm fully extended forward in a horizontal position (i.e., parallel to the floor)
Lateral movement
- Initial position: The child is standing with his/her arms down
- Trajectory: The child moves each arm from down to up “swiping” in a lateral movement and drawing a half-circle in a trajectory
- End position: The child is standing with the arm raised on the same side where the movement began
Cross-lateral movement
- Initial position: The child is standing with his/her arms down
- Trajectory: The child moves each arm from up to down up crossing the right arm from right to left and the left arm from left to right
- End position: The child is standing up raising his/her arm on the opposite side where the movement began
Evaluation methods: We conducted two deployment studies in Mexico using the mobile sensing and the Bendable sound with the designed sounds. In the first study, six children with autism were asked to perform the forward reach and lateral upper-limb exercises while listening to three different sound structures (i.e., one discrete and two continuous sounds). In the next study, nine children with autism were asked to perform upper-limb lateral, cross-lateral, and push movements while listening to five different sound structures (i.e., three discrete and two continues) designed to sonify the movements.
Outcomes and results: Results showed that children with autism exhibit awareness about the sonification of their movements and engage with the sonification. In particular, the use of discrete sound structures engage the children in the performance more repetitions while continous sound structures increase their ability to perform the movements correctly. We found out BendableSound better provides support to perform the push movement while the mobile phones outperform in supporting the practicing of lateral movements.
12,583 thoughts on “Interactive sonification of movement trajectories during motor therapies”