Sketching Architectural Sound Design Using Virtual Reality

Kristin Nedlich Engineering Acoustics, Luleå University of Technology, Sweden kristin.nedlich@ltu.se

Arne Nykänen Engineering Acoustics, Luleå University of Technology, Sweden arne.nykanen@ltu.se

Björn Hellström KTH School of Architecture, Sweden bjorn.hellstrom@tyrens.se

ABSTRACT A sketch is an important tool that architects use to visualize and formulate design decisions. The sketch process is a crucial stage where design iterations and decisions can be made fast, easy and abundant. Sketching facilitates idea spawning, design assessment, modification and communi- cation.

This exploratory study focuses on qualities with Virtual Reality that can be used for sketching sound in the architectural design process. The study was influenced by Contextual Inquiry Inter- views and used trained architects as respondents. The main result of this study is a summary of the pros and cons on how Virtual Reality work today and discusses how to develop a test bed for future research.

One of the main cons, as of now, is the bulkiness of the equipment and the fact that it is not in- tended for sketching sound, yet. Computing speeds and acoustic rendering is another issue on the technical side. A possible benefit of using Virtual Reality is that it gives architects the possibility of incorporating sound in the architectural design process, thus making it an integral part of the prac- tice.

1. INTRODUCTION – THEORY AND CONTEXT

The idea of using Virtual Reality as a tool for sound design, and not only as a tool for visual as- sessment and demonstration, is the basis for this study. Michael Fowler [1] states that the aural ex- perience is fundamentally reliant on computer modeling and aural rendering. By understanding in- ternal work flows and sketch processes in sound design for architecture, we can improve the tools available and make them more suitable for the task. More research should therefore be focused on improving tools that enhance the designer’s capacity in that respect [2]. By making the tools more suited for the task and more accessible we have a foundation on which to establish sound as a given, instead of consequential, factor in architectural design.

inter.noise 21-24 AUGUST SCOTTISH EVENT CAMPUS. ? O? ? GLASGOW

User experience is recognized in product design to be paramount to the success of a product and sound plays a large part in that success [3]. Architecture is, in a sense, also a ‘product’ who’s suc- cess relies on user experience, comfort and well-being. This points to a significant and important shift in how architects can work with sound design in living environments.

1.1 Why sound sketching for architectural design?

Instead of considering the sounds and noises in our built environments as something that needs to be mitigated, we could (should) turn sound into an integrated part of the architectural design pro- cesses. As Hellström [4] writes, “the term noise signifies something more than itself; it is a product triggered by a certain activity, which is either wanted or unwanted according to the listening con- text.”

Josep Llorca [5] discusses how the implementation of sound object design in the architectural field can be aided by temporal tools. If we consider the sound object as the primary factor for the design process then, by Llorca’s point of view, Virtual Reality would be a viable tool to use for this purpose.

1.2 Difficulties with sound sketching for architectural design In the field of sound design for architecture we deal with the problem of how to represent, commu- nicate, hear, and listen to, the sound in a sketch that is temporal [6]. The temporal and contextual aspect presents one of the biggest challenges in sound sketching as we want to keep the sketch mak- ing process like Buxton [7] describes it; quick, timely, inexpensive, disposable, plentiful, explorato- ry and ambiguous.

There are other difficulties, which are not related to technological issues, such as computing speeds or techniques for calculating sound effects, that need to be considered when developing sound sketching for architecture. One such difficulty lies in the human perception of sound in con- trast to our high tolerance for perceiving visual sketches as sketches and not finished products [2].

Sound sketching is therefore in need of a communicative vocabulary and an internal process of its own.

1.3 Towards sound sketching for architectural design When it comes to acoustic planning tools, there are some very sophisticated tools used in the acoustic field but there is still little in the way of sketching with sound for architectural design pro- cesses that allows for quick and easy assessment of sound qualities in architectural design. The acoustic planning tools are not yet integrated in the realm of architecture enough to be considered a part of the architectural design process.

This implies that the needed shift in how we design sounds in built environments is to let sounds be one of the driving factors behind the design decisions and subsequently generated morphologies (i.e. shape, configuration and plan) of the environments. With a deeper understanding of the sound design process and the design tools we use, we can move towards sound becoming an integral part of the design process.

inter.noise 21-24 AUGUST SCOTTISH EVENT CAMPUS. ? O? ? GLASGOW

1.4 Research questions

By testing a digital application on trained architects and observing their use of the application and how their process is affected we can learn more about how to think about sound design in archi- tecture. The overall line of query was based on the following questions: What qualities can VR offer to the architectural design process? What do we need to understand about the architectural sound sketching process, and how do we use that knowledge to create a usable working tool in which sketching and designing sound is feasible?

2. METHOD

2.1 Virtual environment

The design concept for the models was based on open office planning. The spaces were made very differently in size and proportion so that the participants would be able to separate the rooms visually as well as aurally (see Figure 1).

Three different models for the experiment were prepared, in such a way that the test leader con- trolled which room to be in, and when and how the sounds were played back. In this way, the par- ticipants did not need to learn any controls themselves and instead could focus on experiencing, and re-designing, the virtual spaces. Double headphones were used so that the test leader heard what the participants heard and any technical problems or issues with the sound could be easily detected and rectified. Because the project focuses on the internal sketching process the experiment was struc- tured as a walkthrough of ‘sketched’ iterations of open plan office spaces. The participants were asked to listen to the three different rooms, with a layout and furnishing similar to that of a com- mon, open plan office (see Figure 2).

inter.noise 21-24 AUGUST SCOTTISH EVENT CAMPUS. ? O? ? GLASGOW

Figure 1: Perspective top view of the virtual spaces of the experiment #1-3. The figure illustrates differences in height width and length. Sound sources are marked S1-S5. S1: ventilation, S2: male voice, S3: female voice, S4: printer, S5: music. LP stands for Listening Point.

Figure 2: View inside space #3, from the listening point (LP).

Consequently, the ‘poor’ architectural qualities in the test models led to constructive discussions with the respondents about architectural design processes in combination with sound.

inter.noise 21-24 AUGUST SCOTTISH EVENT CAMPUS. ? O? ? GLASGOW

2.2 Equipment

The software used in the study was the Unreal Engine 4.23 game engine with the Resonance Au- dio spatial audio SDK. The hardware used was laptop computer equipped with an Nvidia RTX 3070 GPU, an RME Fireface UC audio interface, a Head Acoustics PEQ V headphone amplifier with Sennheiser HD 600 headphones, and an HTC Vive Pro 2 VR headset.

2.3 Audio levels and acoustic effects Sounds, sound and sound pressure levels in the virtual environments are listed in Table 1. The background level in the studio was less than 20 dBA. Reverberation times were: Room #1: 0.4 se- conds, Room #2: 0.6 seconds and Room #3: 0.7 seconds.

Table 1: Sound pressure levels in the listening positions.

Type of sound Room #1 Room #2 Room #3

Female voice 54 dBA 41 dBA 48dBA

Male voice 58 dBA 40 dBA 46dBA

Music 39 dBA 38dBA 38dBA

Printer 60 dBA 47dBA 45dBA

3. STUDY – PARTICIPANTS, METHODOLOGY AND METHOD

The methodology for this study is based on Contextual Inquiry Interviews that are most com- monly used in User Experience and contextual design [8]. The most significant parameter from CI that was used was the relational interview where the participants were asked to describe what they were thinking and doing, while they were performing a design specific task.

3.1 Participant selection

The number of participants in the study was in total four. A low number, but since this study was explorative with the intention of creating a test bed on how to investigate how to do research in this subject, it was considered appropriate. Therefore it seemed like an adequate number at this point of the research. The participants were not required to have any previous knowledge of this particular program. A background as an architect, with a current status of professionally active in practice or academia, was required for taking part in the survey. Participants were invited to the study and par- ticipated by their own volition. The professional experience ranged from 1 to 20 years. All partici- pants had self-reported normal hearing.

3.2 Methodology and method The goal with the interviews was to elicit the responses from the respondents following a structure resembling Contextual Inquiry [8]. The test leader used a semi-structured interview with the rese- arch questions in mind to navigate the interview per the research questions listed in Section 1.1.

The participants were taken through a virtual tour with the VR gear and external headphones on. They were then given the option to take the headset off and free hands to alter, change, move or re- move sound sources from a room of their choice. They were also given the option to change liste- ning positions. Throughout the experiment the participants were asked to verbalize what they were thinking, how the workflow felt, and what they wanted to do with the design of the floorplan with respect to the sound environment.

4. RESULTS – ANALYSIS OF RESPONSES

The results are presented in sections 4.1–4.4. Text in italics are quotes from respondents. The four participants in the study had very individual responses to how they reacted, and interacted, with the VR environment. One thing that stood out as a commonality though, was that they all re- quested the possibility to be able to add acoustic material properties to the walls, ceilings, floors and furniture. The different sounds were received differently depending on the type of sound, quality of recording and how it behaved in the simulated rooms. The printer sound in particular was very dif- ficult to pinpoint both in origin and whether or not the loudness was correct. One of the respondents said that it felt ‘like the sound was on top of my head’. The cause for the problem with the printer sound was later discovered to be a faulty attenuation setting, which hade inadvertently been set to panning instead of binaural. An unfortunate mistake, but it shows how important the binaural liste- ning is and how strong the human sound perception is.

The music was the easiest to identify where it was coming from, in all rooms.

inter.noise 21-24 AUGUST SCOTTISH EVENT CAMPUS. ? O? ? GLASGOW

4.1 Responses and quotes from the participants

One of the participants (R1) got really into re-designing one of the offices and, instinctively, the respondent and the test leader, as a team, started to use the listen-move-listen technique [2].The test leader acted as a ‘Wizard of Oz’ [7] where the participant had a computer screen on which they pointed or explained what they wanted to change and the test leader executing the instructions on the computer. This part of the experiment was done in screen-mode, as it was more comfortable for the participant to get an overview of the model and show where and how to change or move sound sources and listening points.

Q: How would you like to re-design this space, with respect to sound and space? R1: From an architectural point of view, this is a very challenging space [#3] what we have here at the moment because it is the most open and I think that in order to work some people need privacy. As an individual person, in a single office, you can choose when sounds will occur [wit- hin that space].

This last sentence regarding choice is one of the most important aspects of when noise becomes a disturbance or not. When presented with the ability to choose what kind of aural environment we want to be in, the perceived stress from noise is greatly reduced.

Q: What do you think are the benefits of using VR to sketch with sounds in architectural design? R4: Just to have this feeling, that you are in the environment. I think I prefer this [versus the more detailed analysis that you can get with more statistically accurate, analytical programs]. With this VR environment you put yourself inside it [the proposed space] and you want to have an overview of the whole space. I can change my position and listen to the sounds from different positions of the space, and I will have the experience of distance, volume and frequencies. I can also experience how sounds move in the space. Figures is a more scientific way of understanding the facts, but peoples experience [of spaces] is more powerful.

R4’s answer points to a discrepancy and not necessarily a linear and productive relationship be- tween accuracy and perception when using a tool for designing sounds in living environments.

R1: I think I lack all knowledge on acoustics, I don’t understand how sound travel or crashes in planes. To my knowledge, I understand that parallel walls create echoes.

Q: How would you use VR for an architectural design project? Would you consider start- ing with designing the sound and adapting the spatial design to accommodate that? R4: That’s an interesting question. For different occasions, I would like different sound experi- ences or effects. You could have many different options [in geometry] that would create the de- sired sound effects, and if you could play around with that, that would be quite interesting.

R1: In terms of an architect, opposed to an acoustic expert, it would be helpful to use it more as a tool to help organize the space.

inter.noise 21-24 AUGUST SCOTTISH EVENT CAMPUS. ? O? ? GLASGOW

Q4: How much freedom would you (the respondent) like to have, when it comes to chang- ing geometry of spaces, shifting walls, ceiling heights and so on? R4: For this type of technology, I would like to have the possibility to move around in the space and change my position. Because right now, in this setting, I was sitting still and the sounds moved around me, but if I was able to walk around it would be a totally different experience. I would also like to be able to change things in an instant, put up barriers, or remove them. Then, I believe, you have this choice to do things and I think people would like to try that more. That could be an option for future office design of furniture design.

R1: If I would have a button here [in the virtual environment], I would like to be able to change dimension and angle in the walls and the height of the room. And if I could choose also to change the materials in and on the walls. I think it would play a big part in changing the room acoustics. I would like to work with the materials, shape, angle and height. If it could be more organic that would be great.

Respondent R3, wished for this technology to be compatible with BIM-applications to facilitate the co-working with acousticians. They believed that this work would be much facilitated with compatible computer models. One more thing that was common in the participant group was the request for a program that is easy to use, possibly even with the possibility to work with the design in the Virtual Reality mode. But still accurate enough in the acoustic analyzes to provide meaning- ful data for the architects as well as other team members.

5. SUMMARY AND CONCLUSIONS

The study, small as it was, pinpointed some very real pros and cons with using virtual reality for sound sketching. For one, it is quite easy to start using the program, compared to other similar CAD- or acoustical analysis software. The threshold to getting started and actually starting to use it is significantly lower compared to other applications. Perfecting it of course requires a bit more knowledge and experience, but since it is a quite large community there is plenty of documentation, tutorials and forums to learn from. The hardware paraphernalia, e.g headset, headphones, sensors etcetera, is unfortunately a bit bulky and uncomfortable to work with. The alternative, which would be a VR-room, has other limitations in that it cannot be used in the field. Mobile applications such as AR (augmented reality) or MR (mixed reality) is an option worth exploring, but unlikely to yet be nearly accurate enough in acoustic rendering.

When discussing with the respondents, all seemed positive to using sound more in the sketch process. However, it indicates that the architectural education has failed to raise the issue of sound and there is a knowledge gap between architects and acoustics.

The test bed in this experiment was simple, yet complex enough to stimulate a constructive con- versation about the relationship between architectural design and sounds in our living environment. For comparison in test beds where architects have been asked to work with simplified models of sounds and listening points in open office plans, a similar study was performed in Designing spaces and soundscapes. Integrating sonic previews in architectural modelling applications [9].

inter.noise 21-24 AUGUST SCOTTISH EVENT CAMPUS. ? O? ? GLASGOW

6. FURTHER WORK

The respondents’ answers and feedback about their experience of the experiment as a whole has stimulated new questions and ideas for future, in-depth research. They also concretized what archi- tects need to know and how they would like to work in order to design sounds in architecture.

Proposed research questions for further work: What is instrumental in a tool for sketching sound in architecture? What data do we need to achieve desired aesthetical and technical qualities when designing sound in a building project? What level of immersion and presence, in a tool, is appropri- ate for sketching sound in architecture? Should said immersion be less, or more, to achieve better results?

7. REFERENCES

1. Fowler, M. (2015) Sounds in space or space in sounds? Architecture as an auditory con-

struct. arq, 19.1, 61–72. Cambridge University Press, 2015, doi: 10.1017/ S1359135515000226 2. Nykänen, A., Wingstedt, J., Sundhage, J., Mohlin, P. (2015) Sketching sounds - Kinds of lis-

tening and their functions in designing. Design Studies 39 (2015) 19-47, http://dx.doi.org/ 10.1016/j.destud.2015.04.002 3. Jansen, R.J., Özcan, E., van Egmond, R. (2011) PSST! PRODUCT SOUND SKETCHING

TOOL. Journal of the Audio Engineering Society · June 2011 4. Hellström, B., (2003) Noise Design , Doctoral thesis (monograph), Royal Institute of Tech-

nology, KTH, Ejeby Förlag, Gothenburg, Sweden, ISBN 91 88316 38 6 5. Llorca, J., (2018) Virtual Reality for Urban Sound Design: A tool for Architects and Urban

Planners , chapter 9, Artificial Intelligence – Emerging Trends and Applications, http:// dx.doi.org/10.5772/intechopen.75957 6. Delle Monache, S., Rocchesso, D. (2019) Sketching Sonic Interactions , Foundations in

Sound Design for Embedded MediaEdition: 1st Chapter: 4Publisher: Routledge, DOI: 10.4324/9781315106359-4 7. Buxton, W. (2007) Sketching User Experiences: getting the design right and the right de-

sign. Elsevier Inc., Canada, https://issuu.com/marbaque/docs/sketching_user_experiences_- _bill_b (accessed february 2022) 8. Holtzblatt, K., Beyer, H. (2015) Contextual Design Evolved, Morgan and Claypool, ISBN:

9781627055581 print ISBN: 9781627055598 ebook 9. Milo, A., Reiss, J.D. (2019) Designing spaces and soundscape. Intergrating sonic previews

in architectural modelling applications , proceedings of the 23rd International Congress on Acoustics, 2019

inter.noise 21-24 AUGUST SCOTTISH EVENT CAMPUS. ? O? ? GLASGOW