Interactive and Personalised Listening Using AI Within Physical and Sonic Environments

The PhD aims to create an interactive music system that allows users to manipulate and explore data through sound using deformable interfaces using AI techniques, enabling individuals to explore and experience complex data, such as ecological and historical phenomena, through immersive sound powered by tactile interfaces and personalised interactions. The system will consist of several modular building blocks so each component can be developed in isolation. The spatiotemporal data will be sonified by applying AI techniques to traditional methods so that the system can map sounds in ways that align with different individuals' perceptions. This approach will use techniques found in music recommendation systems. The deformable interfaces will use interactive AI to map the physical interface to sounds, taking into account a user's perceptual links between sound and shape. This idea will be expanded further by exploring dynamic mapping to cater to different accessibility needs. The project draws inspiration from soundwalking, where individuals or groups walk together in a space while focusing on the sounds of their environment. The goal is to broadcast the audio within a larger space by using, e.g. acousmoniums or line array speakers. However, these approaches are often cost-prohibitive; thus, affordable audio broadcasting alternatives will also be explored.

Engaging diverse audiences with complex topics has become challenging in a time characterised by overwhelming distractions. Traditional methods of presenting data can struggle to capture individuals' interest even in particular contexts, such as exhibitions within museums. Immersive spaces can tackle this by providing focused environments to promote deep engagement with specific contexts. Incorporating tactile and collaborative interfaces in these spaces encourages further engagement by allowing individuals to navigate data exploratively.

Allowing individuals to broadcast their sonic interpretation of data into a...