Informing sound art design in public space through

soundscape simulation

Valerian Fraisse 1 ; Nadine Schütz 2 ; Marcelo M. Wanderley 3 ;

Catherine Guastavino 4 ; Nicolas Misdariis 5

1,2,5 STMS IRCAM-CNRS-SU, Paris 1,3,4 Schulich School of Music, McGill University, Montreal 1,3,4 Centre for Interdisciplinary Research in Music Media and Technology, Montreal

2 Institute of Landscape and Urban Studies (LUS), ETH Zurich, Zurich

2 (((Echora))), Zurich 4 School of Information Studies, McGill University, Montreal

ABSTRACT Urban sound management often amounts to reducing sound levels with the underlying assumption of sound/noise as a nuisance. However, a reduction in sound level does not necessarily lead to a more pleasant auditory experience, especially in urban public spaces where vibrancy can be sought after. A proactive design approach that accounts for the human experience of sound environment is needed to improve the quality of urban spaces. Recent studies in soundscape research suggest that added sounds and particularly sound art installations can have a positive influence on public space evalu- ations. Yet, the role of added sounds in urban context remains understudied and there is no existing method to date to prospectively inform soundscape interventions. We present here a research-crea- tion collaboration around the design of a permanent sound installation in an urban public space in Paris: Nadine Schütz’s Niches Acoustiques. We report on the development and validation of a Higher-Order Ambisonic soundscape simulation tool for adding sounds to recorded sound environ- ments. This tool is part of a methodology for elaborating listening tests involving the evaluation of added sounds prototypes to inform the sound artist’s composition to optimize the quality of public space experience in the presence of the sound installation. 1. INTRODUCTION

Cities often consider urban sound as “noise”, an epidemiological burden that should be miti- gated [1]. Still, sounds play a critical and complex role in the way we experience urban spaces and reducing sound levels may not necessarily result in better sound environments [2]. Sound can instead be regarded as a resource in relation to other urban design considerations through the soundscape approach [3]. Specifically, recent studies show that added sounds in urban spaces can have a positive influence on the perceived quality of public space, be it in encouraging positive behaviors (e.g. [4],

1 valerian.fraisse@ircam.fr 2 nadine.schuetz@ircam.fr 3 marcelo.wanderley@mcgill.ca 4 catherine.guastavino@mcgill.ca 5 nicolas.misdariis@ircam.fr

worm 2022

[5]) or in improving soundscape evaluations (e.g. [3], [6]). Most of these studies assess sound inter- ventions on site, through questionnaire deployments or behavioral observations. However, the eval- uation of soundscape interventions in laboratory settings can be beneficial to control and manipulate conditions in a rigorous and repeatable way [6]. Further, the simulation of soundscape interventions allows for prospective evaluations and could help inform the composition of added sounds at the early stages of the creation process. Nevertheless, laboratory conditions can substantially differ from real- life situations, which raises the question of ecological validity, especially in the presence of artificially added sounds [7]–[9]. While an interactive soundscape simulation tool has recently been proposed for co-design processes between urban professionals [10], we extend this line of research to provide a physically accurate simulation tool for soundscape interventions involving added sounds in labora- tory settings, as a way to inform the design of sound installations in public spaces.

The research presented here was conducted in the context of a collaboration bringing together a composer and academic researchers around the permanent public space sound installation Niches Acoustiques . Carried out by the composer Nadine Schütz, this laureate project will result in the de- ployment of the sound installation in the parvis of the Court of First Instance in Paris, France. The research-creation collaboration aims at informing the composition through soundscape evaluations as well as to evaluate the installation’s impact on the auditory experience of the parvis. It involves a longitudinal collaboration between the composer and the researchers, from the creation stage to the public reception. Over the course of 2021, we deployed measurement campaigns to characterize the acoustic environment of the parvis and to take field recordings (Section 2). More recently and in order to evaluate the relationship between Niche Acoustique’s sound design parameters and their influence on soundscape evaluation through listening tests, we developed and validated a soundscape simulator tool involving Higher-Order Ambisonics (HOA) [11] diffusion of field recordings from the measure- ment campaign together with 3D modelling of monophonic added sounds. The development of the tool underwent several iterations that we will presented here. We first developed the tool from the background sound environment reproduction to the added sounds’ auralization (Section 3). We then invited four experts in spatial audio to tune-in the auralization parameters (Section 4). Finally, we validated the tool and selected one of the auralization for future listening tests in a preliminary labor- atory study (Section 5).

2. MEASUREMENT CAMPAIGN

In spring 2021, we collected spatial recordings in the parvis’ for the soundscape simulation. We con- ducted punctual Higher-Order Ambisonic (HOA) recordings and sound level measurements through- out the public space across five sessions covering different activity levels (weekday morning, week- day afternoon, weekday evening, weekend morning and weekend evening). A map of the parvis of Paris’ Court of First Instance is shown Figure 1, with the position of the sound installation Niches Acoustiques ’ speakers and the recording positions. The measurement campaign is briefly outlined below, see [12] for a detailed description.

To capture the parvis’ sound environment in all its variety, the square was gridded in 18 meas- urement points (see Figure 1). These positions correspond to typical usage patterns (e.g., transit flows, seats and benches, subway entrance), sound sources (e.g., roadway traffic, quiet zones) and relevant distances to the installation’s poles. For each measurement session and at each point, 5-minute re- cordings were made synchronously with a reference position in the middle of the parvis (position 11, see Figure 1). On the mobile position, we measured equivalent sound pressure and third-octave level with a B&K 2250 sound level meter together with HOA recordings with a mh Acoustic Eigenmike 32 [13]. On the reference point, we captured the equivalent sound pressure level with a B&K 2238

worm 2022

and mono recordings with a DPA 4060. All measurements were oriented towards the opposite direc- tion to the Court of First Instance building, and the distance between the measurements location and the ground was 1.3 m.

worm 2022

Figure 1: Map of the parvis of the Court of First Instance of Paris. The Niche Acoustiques’ speakers

are mounted on four poles across the parvis. Punctual HOA recordings and sound level measure- ments were made across 18 positions with a reference measurement at position 11. Position 11 is

also the listening point chosen for the soundscape simulation tool. 3. DEVELOPMENT OF A SOUNDSCAPE SIMULATON TOOL

We designed a methodology for informing the composition of a public space sound installation prior to its on-site deployment through soundscape simulations. The simulation consists in HOA diffusion of environmental sounds that form the ambient sound environment to which are inserted auraliza- tions of added sounds that form prototypes of the sound installation (see Figure 2).

Figure 2: Flowchart of the soundscape simulation tool. The ambient sound environment (upper part) is simulated from concatenated excerpts from fields HOA recordings. A 3D Modelling of the

scene is sent to an auralization unit that converts monophonic added sounds (lower part) to HOA

streams. Both streams are diffused in a listening room for soundscape evaluation.

3.1 Ambisonic Reproduction

All stimuli (including those used in the experts’ tune-in and in the preliminary listening test) were presented at the IRCAM’s studio 4 listening room over a hemispherical dome of 24 Amadeus PMX 4 speakers. The choice of encoding and decoding parameters is the result of multiple joint listening sessions including three of the authors. Specifically, we compared in situ listening with the repro- duced soundscape within small intervals of time. The Eigenmike 32 signals were encoded into 4 th order HOA streams with MAX/MSP’s spat~ [14] using Tikhonov regularization [11]. The individual 5-minutes L Aeq values captured during the measurement campaign at the mobile position (see Section 2) were used to calibrate the reproduction levels in the listening room. All HOA streams (the Eigenmike recordings as well as the added sounds) were summed together and decoded using MAX/MSP’s spat~ [14]. 3.2 Sound environment

A sound environment made of concatenated 4 th order HOA excerpts [11] from the measurement cam- paign (see Section 2) is continuously playing during the soundscape stimulation. The excerpt selec- tion was made according to the following criteria: (1) the excerpts had to be representative of the parvis’ average level of activity: we excluded peak-

activity (weekday morning and afternoon) and low-activity (weekend morning) recording ses- sions and chose excerpts from sessions 3 (weekend evening) and 5 (weekday evening). (2) the excerpts had to come from recording positions close to each other and to the simulation’s

listening position (position 11, see Figure 1). For instance, we excluded positions too close to the road (e.g. position 18) or too far from position 11 (e.g. position 5). The retained positions were the following: 3, 6, 7, 8, 10, 11, 12, 13, 14, 15 and 16. (3) the excerpts should not contain too many salient sounds to allow the participants to focus on the

added sounds during the listening tests: excerpts were selected through joint listening sessions between the two first authors using Reaper [15]. We systematically excluded electro-mechanical sounds such as construction works, intelligible voices and sirens. We excluded any other sounds that significantly rose from the background environment. Excerpts ranged in duration from 30 seconds to around 2 minutes. A total of 38 excerpts were cross- faded in random orders with Reaper scripts that were generated using python’s library reathon [16] to create short yet smooth and unnoticeable transitions. A 3-second crossfade between excerpt was found to be optimal. 3.3 Added sounds auralization

The added sounds are spatialized using IRCAM’s EVERTims framework [17] integrated in MAX/MSP’s spat~ library [14]. First, the parvis’ geometry and the acoustic properties of the wall surface materials as well as the sources and listener positions are modelled with Blender [18] (see Figure 3). The listener is located at the center of the parvis (position 11) and the sources position correspond to the installation’s speakers’ position (see Figures 1 and 3). Upon reception of the scene geometry and the listener/sources position, EVERTims computes in real-time a list of image sources that correspond to early reflections and sends them to an auralization unit [17], [19]. The auralization unit then simulates the late reverberation using a Feedback Delay Network (FDN) [17], [20]. The output of the auralization unit is ultimately encoded into 4 th order HOA streams with spat~ (see [11]).

worm 2022

Although the parvis’ geometric model is based on scale drawings, many physical parameters necessary for tuning the auralization unit were missing (such as the late reverberation time, absorption coefficient of the surfaces, etc.). We therefore decided to rely on perceptual cues to fine-tune the auralization. We first invited four expert listeners in spatial audio to tune-in the auralization via ana- lytical listening (Section 4) and we validated and selected one of the auralizations with a preliminary listening test (Section 5).

Figure 3: 3D Modelling of the parvis for the auralization of added sounds in Blender. 4. TUNING OF THE AURALIZATION VIA ANALYTICAL LISTENING

worm 2022

Four experts in spatial audio and sound engineering were invited to fine-tune the parvis’ auralization parameters using a Graphical User Interface (GUI) designed with MAX/MSP (see Figure 4). Experts were able to compare HOA reference excerpts with the auralization of mono signals (Ambisonic reproduction parameters are described in Section 3.1). The HOA reference excerpts were gathered from the measurement campaign’s field recordings (see Section 2). The mono signals were selected from an open-source sample library [21] and from IRCAM’s sound library to be as close as possible to the reference, and were normalized using a python implementation of ITU-R BS.1770-4 loudness algorithm [22]. Excerpts consisted in three impulsive sounds allowing to clearly hear room parame- ters such as the early reflections while remaining varied in timbre (a truck honk, a metallic impact from nearby construction works and a whistle). During the task, experts were asked to tune-in the maximum propagation duration and walls’ frequency-wise attenuation for the early reflections and the reverberation gain as well as the absolute and frequency-wise reverberation times for the late reverberation. Note that experts were able to play/pause the background sound environment described in section 3.1, to loop the excerpts and to change the gain of the added sounds during tuning.

Experts were free to end the tuning session as soon as they were satisfied with the results. In total, the sessions took from around 10 to 30 minutes. During the sessions, all experts expressed the difficulty in finding a tuning that was suitable for all three types of sound source: a tuning that was satisfactory for one type of source was often not equally satisfactory for the other two sources. Experts typically addressed this issue by beginning the tuning with one of the sound sources of their choice and adjusting it so that the auralization was satisfactory for all sound sources.

Figure 4: MAX/MSP GUI provided to experts to fine-tune the soundscape simulator’s auralization unit. In the master area (left), users can change the added sounds’ gain. In the playback area (mid-

dle), users can play or loop the added sounds as well as the reference excerpts and can play/pause

the background sound environment. In the tuning area, users can adjust auralization parameters, from the early reflection (max propagation duration and surface attenuation) to the late reverbera-

tion (absolute and frequency-wise time of reverberation and reverb gain). 5. PERCEPTUAL VALIDATION

worm 2022

To validate the simulation tool and select the most adequate auralization between the four experts’ tunings, we ran a preliminary listening test with N=12 participants (9 PhD students and three re- searchers from IRCAM) including one of the above-mentioned experts. 5.1 Procedure

The test was implemented in MAX/MSP (Ambisonic reproduction parameters are described in Sec- tion 3.1). During the test, participants were invited to compare four auralizations (corresponding to tunings from experts E1, E2, E3 and E4) of five different mono signals with a reference HOA excerpt in presence of the continuous ambient sound environment described in 3.1 (see Figure 5). Each signal was presented in a random order and repeated twice so that participants evaluated a total of 10 sets of stimuli. Signals consisted in the three impulsive signals used in the tune-in listening sessions (a truck honk, a whistle, and a metallic impact from nearby construction works) in addition to two con- tinuous sounds (a passing scooter as well as a circular saw from nearby construction works). For each auralization, participants were free to play all signals as many times as they wanted and were required to indicate whether “the modelled source seems to come from the same space as the one heard in the reference excerpt (la source modélisée a l'air de provenir du même espace que celle entendue dans l'extrait de référence)” on continuous Likert scales from “totally disagree (pas du tout d’accord)” to “totally agree (tout à fait d’accord)” (see Figure 5). Note that the auralizations were presented in a different random order for each signal.

Extraits de rétérence [J J Sons ajoutés Premigres reflexions CO «w CO «= C «= CO

Figure 5: MAX/MSP interface provided to participants. For each type of sound source, participants

were able to play the reference HOA excerpt (left), and to compare it against four auralizations of the added sound signal (middle). Participants were then invited to rate each auralization on a Likert

scale (right). When they were done, participants were able to go to the next excerpt (bottom left).

5.2 Results

At the end of the listening test, participants spontaneously described the soundscape simulation as being realistic and immersive. However, most of them mentioned the difficulty of identifying notable differences between the auralizations, except for one that were often indicated as less convincing than the other three (most likely E2, see below). Especially, participants indicated that they were not able to distinguish auralizations of the two continuous sound sources (passing scooter and circular saw).

worm 2022

To measure the test consistency, we computed the test-retest reliability coefficients (Pearson’s correlation) between and within participants with python package scipy [23]. Result show individual scores varying from r = 0.24 to r = 0.82, with an overall correlation of r = 0.57 (p < .001). The mean ratings for all auralizations range from 34 (E2) to 56 (E4) and boxplots are shown in Figure 6. To assess whether differences of ratings between the four different auralizations and the five types of sound were significant, we conducted a factorial repeated-measures Analysis of Variance (ANOVA) using R package ezANOVA [24]. Mauchly’s test indicated that the assumption of sphericity had been violated for the main effect of the type of signal, therefore degrees of freedom were corrected using Greenhouse-Geisser estimates of sphericity. There was a significant main effect of the type of aural- ization on the ratings (F = 13.4, p < .001) and no significant effect of the type of sound signal (F = 1.4, p > .05). This indicates that participants rated differently the four auralizations while each type of sound had the same ratings at average. Holm post hoc t-tests revealed that ratings for the E2 aural- ization were significantly lower than ratings for any of the other auralizations (-7.5 < t < -5.5; p < .001, see Figure 6, left). However, there were no significant differences between E1, E3 and E4. In addition, there was a significant interaction effect between the type of auralization and the type of signal used (F = 3.7, p < .001). As shown in Figure 6, right, this can be explained by the fact that participants were able to differentiate the auralizations of impulsive sounds (honk, metallic impact and scooter) but not of continuous sounds (scooter and circular saw), confirming their spontaneous feedback of the experiment. Two repeated measures ANOVAs of the ratings for the four auralizations comprising one of the two types of sounds confirmed this hypothesis as there were significant differ- ences of ratings between auralization for the three impulsive sounds (F = 16, p < .001) but not for the two continuous sounds (F < .01, p = .99). Holm Post-hoc t-test including only the impulsive sounds showed again significant differences only between E2 and all the other auralizations (-11 < t < -7.5; p < .001).

The results of this preliminary experiment allowed us to exclude the auralization E2 and thus to choose the soundscape simulator’s auralization between E1, E3 and E4. We decided to choose E4 as it had the best mean rating (56/100) across all tunings.

Figure 6: Ratings and Holm post hoc t-test significances (***: p < .001) for each auralization (Left).

Mean rating and standard error for each type of sound across auralizations (Right). 6. CONCLUSIONS

worm 2022

While soundscape interventions are typically evaluated retrospectively, we developed a tool to inform the design of sound interventions before their in-situ deployment in a controlled 3D sound environment with artificially added sounds from monophonic sources. We believe that such an ap- proach should allow for a better integration of sound interventions, as it is often difficult to adjust an intervention once it has been carried out.

The development of the tool involved a lot of back and forth between listening sessions and experts’ guidance and is the result of a series of compromises between realism and immersion. The results of the preliminary listening tests confirm that the tool is fairly immersive and ensure that it is ecologically valid for the study of soundscape intervention simulations.

Overall, this research project is part of a large-scale research-creation collaboration from which we seek to inform the composition of Niches Acoustiques as well as to evaluate its impact on the soundscape of the parvis. The soundscape simulation tool presented here is being used in listening tests in which participants familiar with the parvis are invited to compare several composition proto- types of the sound installation Niches Acoustiques . The results of these listening tests will assist the composition of the installation by linking sound design parameters and their influence on soundscape evaluation. 7. ACKNOWLEDGEMENTS

We would like to thank Benoit Alary, who made it possible for EVERTims to be integrated in the simulator patch; Elise Nicolas, who contributed to the measurement campaign, and Antoine Le Dreff, who assisted with some of the listening sessions. We would also like to thank Olivier Warusfel, Clément Cerles and Thibaut Carpentier for their technical and scientific guidance. We would finally like to thank all participants of the study.

100 1 £2 E3 4 Rating as 100 0 60 B88 1 El £2 €3 &4 Impulsive sounds Truck Honk Wiistie Metalic Impact Continuous sounds Circular Saw Scooter

8. REFERENCES

[1] World Health Organization, “Burden of disease from environmental noise,” The WHO Euro-

pean Centre for Environment and Health , 2011. [2] J. Kang and B. Schulte-Fortkamp, Soundscape and the Built Environment . CRC Press, 2015. [3] C. Guastavino, V. Fraisse, S. D’Ambrosio, É. Legast, and M. Lavoie, “Designing sound instal-

lations in public spaces: A collaborative research creation approach.,” in Designing Interactions for Music and Sound , M. Filimowicz, Ed. Routledge, 2021. [4] L. Lavia, J. Kang, H. Witchel, and F. Aletta, “A preliminary soundscape management model for

added sound in public spaces to discourage anti-social and support pro-social effects on public behaviour,” presented at the Proceedings of the DAGA 2016, Aachen. doi: http://dx.doi.org/10.13140/RG.2.1.1392.4242. [5] E. Bild, D. Steele, C. Tarlao, C. Guastavino, and M. Coler, “Sharing music in public spaces:

Social insights from the Musikiosk project (Montreal, CA),” in Proceedings of Internoise 2016 , Hamburg, Germany, 2016, pp. 6834–6843. [6] T. Oberman, K. Jambrošić, M. Horvat, and B. Bojanić Obad Šćitaroci, “Using Virtual Sound-

walk Approach for Assessing Sound Art Soundscape Interventions in Public Spaces,” Applied Sciences , vol. 10, no. 6, Art. no. 6, Jan. 2020, doi: 10.3390/app10062102. [7] C. Guastavino, B. F. G. Katz, J.-D. Polack, D. J. Levitin, and D. Dubois, “Ecological Validity

of Soundscape Reproduction,” Acta Acustica united with Acustica , vol. 91, no. 2, pp. 333–341, 2005. [8] W. Davies, N. Bruce, and J. Murphy, “Soundscape Reproduction and Synthesis,” Acta Acustica

united with Acustica , vol. 100, no. 2, pp. 285–292, 2014, doi: 10.3813/AAA.918708. [9] J. Y. Hong et al. , “Quality assessment of acoustic environment reproduction methods for cine-

matic virtual reality in soundscape applications,” Building and Environment , vol. 149, pp. 1–14, Feb. 2019, doi: 10.1016/j.buildenv.2018.12.004. [10] C. Tarlao, “Soundscapes in context: Investigating in situ experiences and proposing a simulator

for urban professionals,” PhD Thesis, McGill University, 2022. [11] S. Moreau, “Étude et réalisation d’outils avancés d’encodage spatial pour la technique de spa-

tialisation sonore Higher Order Ambisonics : microphone 3D et contrôle de distance,” PhD The- sis, Université du Mans, Le Mans, France, 2006. [12] V. Fraisse, E. Nicolas, N. Schütz, C. Ribeiro, and N. Misdariis, “Évaluer l’impact d’installations

sonores sur la perception du paysage sonore urbain : cas d’étude d’une place publique pari- sienne.,” presented at the 16e Congrès Français d’Acoustique, Marseille, France, 2022. [13] https://mhacoustics.com/products (Accessed Apr. 29, 2022). [14] https://forum.ircam.fr/projects/detail/spat/ (Accessed Apr. 25, 2022). [15] https://www.reaper.fm/ (Accessed Apr. 27, 2022). [16] https://github.com/jamesb93/reathon (Accessed Apr. 27, 2022). [17] D. Poirier-Quinot, B. F. G. Katz, and M. Noisternig, “EVERTims: Open source framework for

real-time auralization in architectural acoustics and virtual reality,” presented at the DAFx-17, Edinburgh, United Kingdom, 2017. [18] https://www.blender.org/ (Accessed Apr. 27, 2022). [19] S. Laine, S. Siltanen, T. Lokki, and L. Savioja, “Accelerated beam tracing algorithm,” Applied

Acoustics , vol. 70, no. 1, pp. 172–181, Jan. 2009, doi: 10.1016/j.apacoust.2007.11.011. [20] M. R. Schroeder, “Natural Sounding Artificial Reverberation,” JAES , vol. 10, no. 3, pp. 219–

223, 1962. [21] https://freesound.org/ (Accessed Apr. 29, 2022).

worm 2022

[22] C. J. Steinmetz and J. Reiss, “pyloudnorm: A simple yet flexible loudness meter in Python,”

presented at the Audio Engineering Society Convention 150, 2021. [23] https://docs.scipy.org/doc/ (Accessed Apr. 27, 2022). [24] A. Field, J. Miles, and Z. Field, Discovering Statistics Using R . London: SAGE, 2012.

worm 2022