ADRIA M. CASSORLA

  • Home
  • INICI
  • Hola Món!
  • Hello world!
  • Portfolio
  • Contact
  • Contacte
  • Home
  • INICI
  • Hola Món!
  • Hello world!
  • Portfolio
  • Contact
  • Contacte

master thesis

DAW based spatial audio creation using mobile phones

PROJECT OVERVIEW

3/5/2019

0 Comments

 
This project is my Master thesis of the MSc in Audio and Music Technology at the University of York, and it has Abbey Road studios as Industry Partner. I started to think about it approximately one month ago, when I was assigned to this topic. However, now I'm starting to research full-time, and I will be working on it until the end of August. 

If I tell you that the formal title is: "DAW based spatial audio creation using smartphones", probably your reply will be something like: "Sounds cool, but... what does it mean?" Let's start from the begging then:

background

"DAW based..."
A Digital Audio Workspace (DAW) is a generic software categorisation within we can find commercial software like ProTools (image), Cubase, Reaper, LogicPro etc... With them, the user has an environment to sequence, record, edit, and mix audio or music.

​This kind of software can manage inputs and outputs of an external soundcard (to work with analogue equipment, for instance), or use third-party effects (also called plug-ins) to generate or modify sound within the same environment. Therefore, a DAW is the base of all digital studios, from the smallest home-studio to Abbey Road's studio one. A DAW-based tool, thus, is whatever tool that can interact with a DAW (generally in a plug-in).
Picture
Pro Tools | First's GUI - Avid
"...Spatial audio creation..."
Spatial audio is a really vague name but, for now, just assume that spatial audio means the sound is perceived in the space (as the real life). Other ways to say it is 3D audio or immersive sound. Spatial audio can be reproduced using an array of speakers (like Dolby Atmos), but it is also possible to reproduce through headphones. Do you want to try it? What about going to the virtual barbershop?

​Several commercial brands like Waves, Facebook, or SSA, as well as independent developers like Matthias kronlachner or Ambisonic ToolKit, have released different tools to work with (and create) spatial audio. Some of these tools are used, for instance, to place sources in the space, to add spatial reverberation, or to apply other effects. Moreover, most of these tools are also DAW based, and then, why and what do I want to research in this area?
​
Picture
Font: ambiX - Ambisonic plug-in suite

​"...using smartphones."
Indeed, the last part of the sentence is the answer. At it is shown in the image above (from the ambix plug-ins suite) all of the previously presented tools are using 2D user interfaces to work with 3D audio. Even though they simulate 3D objects, such as the ambix encoder (bottom-right), sometimes it is difficult to work with them, especially concerning elevation, width or distance.

​"And what's happening with the smartphones?" Well, nowadays almost everyone has at least one smartphone with them; And, among others, it has lots of sensors like the gyroscope, the accelerometer or the camera that could be used to control spatial audio. An app then, can interpret these sensors information, and send OSC messages (a standard network communication for sound) to a plug-in in the DAW. After that, the plugin can translate the received information to place a sound in the 3D space, change the reverberation, the EQ, or whatever else.

​Now that we know the background and hopefully the title is better understood, let's talk about the specific aims and objectives of the project:​

aims and objectives

​The main objective is developing an iPhone app which lets the user control spatial audio parameters in a DAW. Therefore, also a plug-in needs to be developed to receive the app messages and process the audio.
With these objectives, the project has two research areas:

Spatial audio workflow
Before controlling spatial audio parameters with a smartphone, we have to know which parameters we want to control in our plug-in. Therefore, the first research area is the spatial audio workflow, where we can find questions like:
  • Which are the typical Fx used to work with spatial audio? (Eq, Comp, Reverb...)
  • Are they used individually in each source? Or as a global effect? In series or parallel?
  • Is there any pattern in the order they are used?
  • Which Fx are necessary and which are used sporadically?
  • Are the users happy to use generic effects, or they prefer to use third-party effects?

Spatial audio control using smartphones
Finally, the second research area is related to gesture control and smartphones. An iPhone has lots of sensors and Augmented Reality capabilities to explore and thus, there are also several questions to ask:
  • ​(Basic) Could a smartphone be used to control spatial audio? And how?
  • ​​Which Fx are more likely to be controlled using these capabilities?
  • ​Is it practical to have an app on the smartphone, and a plug-in in the computer, or is it better to use just one of them at once?
  • Which is the best way to control spatial audio? Using AR and the camera? Using small movements? Moving the iPhone around the user? Just an extension of the plug-in user interface?

summary

To summarise, the main concept of the project is to enable the user to control spatial audio parameters in a DAW environment, but without using only the mouse and the keyboard. Instead, investigate new and easier ways to control it using the different capabilities of smartphone sensors.

A good example of spatial audio creation without the keyboard and the mouse is this product called DearVR Spatial connect. It is using a VR headset and VR controllers to move sound sources in a VR environment. This is what they call 'the mixing console of the future'.
0 Comments



Leave a Reply.

    blog:

    This blog aims to share the progress of my Master thesis project with anyone who might be interested. Maybe someone could be interested in all the project, perhaps someone else could be interested in a post of a particular topic or, why not, maybe you are a friend of mine, and you just want to keep in touch with me during these busy days (sorry about that... I'll be back soon!). Whatever of the previous reasons, you are really welcome to read as many posts as you like and, of course, feel free to comment, discuss or ask anything you want!

    project:

    I’m currently researching on new DAW-based tools to work with Spatial Audio creation, using gesture control with smartphones. This project has Abbey Road as an industry partner, and it is supervised by Dr Gavin Kearney and Prof Andy Hunt.

    (Click here to read the project overview)

    Author:

    I am passionate about Audio and Music Technology, and I really love the world of sound, audio and music. After finishing the Sonology Bachelor, and working for two years in the media industry, I decided to progress to a more scientific and engineering-based career. That is why I am currently stuying an MSc in Audio and Music Technology at the University of York, where I discovered my passion for academic research, especially in the field of Spatial Audio and Virtual Acoustics.

    View my profile on LinkedIn

    Blog:

    Aquest blog pretén compartir el progrés del meu treball de final de màster amb tothom qui hi pugui estar interessat. Potser hi ha algú que està interessat en tot el projecte, potser algú altre està interessat en algun tema en concret o, per què no, potser ets algun amic meu que vol simplement vol estar en contacte amb mi durant aquests dies que no hi sóc (perdó, tornaré aviat...). Sigui quina sigui la raó, sou tots molt benvinguts a llegir tants posts com vulgueu i, per descomptat, no dubteu en comentar, discutir, o preguntar el que us vingui de gust!

    Archives​

    May 2019
    April 2019

    Categories

    All
    Demo
    Overview
    Reviews

    back to portfolio

​© 2020 Adrià Martínez i Cassorla