Proceedings of the Institute of Acoustics

 

The Arup ExperienceLab

 

C. Ferreira, Ove Arup & Partners Limited
I. Knowles, Ove Arup & Partners Limited

 

1 INTRODUCTION

 

The methods by which engineers, designers and planners make decisions is by using experience, based on evidence and their perception of the information presented to them. Tools have been produced to create experiences that help better inform a design decision, by enabling greater engagement and participation in the design process. These tools aim to recreate an environment as accurately as possible, as it has been shown that the smaller the gap between expectations and actual experience, the more effective the design^[1].

 

This paper describes a facility that has been developed for an immersive environment which empowers clients and collaborators to make decisions and to understand complex issues by experiencing them. Since its prototype launch in 2018, the Arup ExperienceLab has developed a multi-sensory environment, using visual, haptic, and aural senses to establish an experience-led approach to design. The completed final facility was formally launched in April 2022.

 

2 EXPERIENCE-LED APPROACH TO DESIGN

 

2.1 Auralisation

 

Developed in 1998, Arup SoundLab is a facility that empowers clients and stakeholders to understand complex acoustic design issues and provides access to understanding standards and planning requirements. Initially the Arup SoundLab was used to convey the acoustic design for some of the world’s best arts and culture venues, and since then it has taken on a broader range of projects including large infrastructure schemes, transportation noise, and commercial building design.

 

The success of Arup SoundLab is founded on its accurate, realistic reproduction of an acoustic design in a carefully controlled acoustic environment, that undergoes calibration procedures to maintain accuracy for all demonstrations. By providing an accurate portrayal of an existing or future environment, the demonstrations provided in the Arup SoundLab are a piece of evidence, used to assist decision making.

 

Figure 1: SoundLab in London, UK

 

2.2 Applying Visualisations to Demonstrations

 

Whilst building the sound demonstrations created in SoundLab, it was identified that the visual element was an important factor in how people experience and react to sound. At first, a flat screen projection was used, as an aid for users to see an image that illustrated what they were listening to. This helped connect the listening experience to a visual cue.

 

The visualisations were further developed at its largest scale for the London Heathrow Airport consultations on future expansion and airspace change^[2]. For the consultation events, a fully immersive and interactive sound demonstration was created, combining head-tracked spatial audio and a Virtual Reality (VR) visualisation so the user could look up and view the aircraft as it flies overhead. This accurately demonstrated the ambient environment at a location relevant for the user, plus the sound level and visual cue of aircraft at different altitudes.

 

Figure 2: Interactive virtual reality simulation of aircraft passing overhead

For consultation events, using VR as a method to present visualisations was key to the success of creating conversations with members of the public^[3]. This method of VR was also used inside SoundLab, to make visual design decisions at key stages in a project. However, it was found that with groups of people who are collaborating and discussing with each other, using VR blocks communication between groups of people, creating an isolating experience. Learning from this, the motivation was to create a facility that enabled visitors to be immersed in the same experience allowing discussion and communication between stakeholders, without the need of head mounted (VR) displays.

 

3 ROOM DESIGN

 

Using knowledge gained from the Arup SoundLab, plus the development of bespoke visualisation technologies, the Arup ExperienceLab was designed to combine multi-sensory stimuli that create a fully calibrated experience for the user. This section describes the technical aspects that have been developed to create this facility.

 

3.1 Audio System

 

The audio system used in the Arup SoundLabs are currently based on an Ambisonic spatial sound algorithm, using an array of 16 loudspeakers. Whilst this remains a valid approach for demonstrating a calibrated and accurate spatial soundscape, the optimal experience is generally limited to just a single listener within the central ‘sweet spot’. This can result in the requirement to repeat presentations multiple times for different listeners.

 

For the Arup ExperienceLab, the key aim was to be able to give a group of up to six people the same listening experience at the same time. By having the speakers positioned further away from the listener compared to the Arup SoundLab, the listening area could be increased to a small central zone in the middle of the facility. The array in the Arup ExperienceLab uses a 26 channel ambisonic audio array, which includes 24 loudspeakers and 2 subwoofers.

 

The structure that defines the perimeter of the Arup ExperienceLab is a gridshell which allows for the speakers to be placed in a flexible location within the facility. Utilising this flexibility, other spatial audio formats such as wave field synthesis is planned to be explored in the future using a larger number of loudspeakers.

 

 

Figure 3: Loudspeaker arrangement presented on the Arup ExperienceLab screen

 

3.2 Visualisation System

 

3.2.1 Screen

 

Creating an immersive visualisation for the users, the screen was designed to strike a balance between VR but maintain a shared experience within groups. The following three elements were chosen to meet the requirements:

• A high-quality surface to allow projection of detailed high-resolution images
• To be acoustically transparent to allow for loudspeaker placement behind it
• Ability to be curved, to fill the field of the user’s view

To achieve this, an acoustically transparent stretched fabric bonded to a perforated steel frame was developed. The size of the screen was optimized by the width and angle of curvature required to fill the users view, plus height limitations within the office space. For each of the tested iterations, analysis was conducted to find the optimal projector number and positions. Each iteration used prototypes to view simulations of typical projected images, looking carefully and interfaces, distortion, and glare from the projector apertures.

 

The final solution used a 6m wide and a 2m high curved screen, using high resolution projectors with short throw lenses, with pixel matched blending to project onto the acoustically transparent fabric screen. The projectors were selected based on these following requirements:

• Minimum 4k resolution
• Curved screen correction
• Stereoscopic capabilities for 3D presentations, using 3D glasses

 

Having a stereoscopic mode allows the visualisations to have a realistic depth of field and convincing realism. Aside from greater immersion in a 3D scene, this has allowed the user to understand visual elements in more depth including topography, relative scale of buildings and finer details inside a model.

 

3.2.2 3D Motion Tracking

 

Incorporating gesture control into the facility, Arup ExperienceLab incorporates a motion tracking system which allows interaction with the experience using physical tools. The motion tracking applies to a single user in Stereoscopic mode, which allows the user to walk around with active perspective changing to enhance the sense of realism.

 

The most effective use of this tracking has been by adding a set of motion tracking nodes to the 3D glasses, allowing a single user to move their head and change the view. This has been particularly important for testing sightlines within an arts & culture venue, or testing accessibility issues for individual people within a modelled environment.

 

3.3 Room Specifications

 

The facility is built using a self-supporting plywood gridshell frame, which is inside a dedicated room in the Arup London office. The frame has a grid arrangement which holds the loudspeakers, and the curved fabric screen.

 

To control a low reverberation time, the floor is carpeted plus the inner gridshell walls and ceiling of the Arup ExperienceLab are covered in acoustic absorption. The void behind the absorption integrated into the gridshell ensures that the RT is controlled across the audio spectrum. This means that the sound experienced is not influenced by the acoustic inside the room.

 

To achieve a low background noise, the projectors are housed inside custom-built ventilated enclosures that reduce noise breakout inside the main space. All other equipment such as the computers, and loudspeaker amplifiers are located outside the main space to remove any other sources of noise. To reduce noise from the adjacent office spaces, the outside shell of the Arup ExperienceLab room is built with slab-to-slab partitions.

 

 

Figure 4: Room arrangement of the Arup ExperienceLab

 

4 DEMONSTRATION USES

 

By utilizing modelling and measurement technologies, the facility has the ability to reproduce an existing environment, fully virtual, or mixed reality (a real environment with overlaid animation and audio). Demonstrations can be a passive demonstration or interactive, placing the control in the hands of the visitor, allowing them to explore a virtual environment. This section describes each of these uses.

 

4.1 Reality Capture

 

The first demonstration mode category describes how a user can be immersed within an existing environment. This uses a calibrated spatial audio recording of the ambient noise and a visualisation of the moving environment using a high resolution, 3D camera recording. This is called a ‘reality capture’ demonstration, used to understand an existing landscape that has been captured using site recordings. The audio that is played back though the ambisonic system is calibrated to the sound level measured on site, and the visualisation accurately depicts the landscape using stereoscopic vision.

 

The visualisation capture is taken using a stereoscopic camera rig, which is set to record video that can be projected onto the aspect ratio of the Arup ExperienceLab screen. A ‘verified view’ is recorded, so that it captures a real perspective for the user and allows models to be accurately placed on the view (see Section 4.3).

 

The sound capture is taken using a 3D ambisonic microphone, plus a sound level meter which are placed next to the camera rig. The ambisonic microphone records the ambient noise occurring in the environment in three dimensions, and the sound level meter is used to record the noise levels during the measurement period. The sound level meter allows the audio recording to be played back at a calibrated level, so the user listens to a calibrated accurate environment.

 

 

Figure 5: Example of a reality capture demonstration

 

4.2 Three Dimensional Models

 

The second category is to project a three-dimensional model into the Arup ExperienceLab, so that the user can be inside a future design, bringing the model to life. Similar to the reality capture demonstrations, the models are viewed using stereoscopic projection so there is convincing realistic depth to what is being viewed.

 

The model is created using 3D modelling software, which is then used to transform it in a game engine format allowing playback using the Arup ExperienceLab system. This transformation applies the correct architectural features, textures, lighting, and colours in the demonstration, plus putting the model in stereo so it is viewed through 3D glasses. Calibrated adjustments are made to the aspect ratio and camera views in the model, so that what is viewed on the screen accurately depicts the correct dimensions. Finally, if required for the demonstration, the motion tracking is applied so that a user can move around inside the model to enhance the user interaction.

 

For the acoustics, the 3D model is placed into acoustics software that creates an auralisation of the model. This considers the geometry of the space, the internal materials, and the location of where the sound sources and receivers will be. This auralisation creates a B-format ambisonic audio file that is played in conjunction with the visualisation.

 

This type of demonstration has been important for architects, building engineers, or future users of a building to fully understand its design. Furthermore, it has been important for accessibility testing, as it allows understanding a user journey and the touchpoints moving through a building.

 

A successful example of this use has been for the design of a large broadcast studio, where a demonstration was created to let the key stakeholders and future users of the studio understand what the design will look, feel, and sound like.

 

The sound quality of the studio design was demonstrated by listening to an auralisation from the studio acoustic model. This auralisation used multi-source orchestral audio recording, so that individual instruments were modelled separately, then mixed together in the final auralisation. This allowed a sense of width and depth across the platform in the studio design.

 

The visualisation of the studio illustrated the vision from the design team, including the lighting, architectural and venue designers. Using the 3D head tracked technology, a user could test and experience sightlines at various positions around the studio to assess what view the audience and performers would get. Participants were able to move around the studio and experience the sound, lighting, and architecture from multiple positions.

 

 

Figure 6: Example of a 3D model demonstration of a broadcast studio design

 

4.3 Mixed Reality

 

Combining both the reality capture and 3D modelled environments, the mixed reality demonstrations are used to show a model of a new building or scheme in the context of an existing environment.

 

The use cases of this are to create a comparison between an existing environment, with a future environment that could be impacted by a proposed building or infrastructure development. For the future environment this could be a photo realistic 3D model, or a conceptual white card model of an early design.

 

This type of demonstration is particularly important in making planning decisions, by viewing how a new scheme may impact a city environment, landscape, or protected views^[4]. As the experience is enhanced by an immersive screen and the sound environment, it shows how planning decisions can be made using experiential evidence, rather than viewing a still 2D image. Taking a step further, moving videos have been produced that show a ‘walkthrough’ of an environment to demonstrate a journey along an impacted landscape.

 

 

Figure 7: Example of a mixed reality demonstration comparing before and after a rail scheme

 

5 DISCUSSION AND NEXT STEPS

 

Since the launch in 2022, the Arup ExperienceLab has undergone a marketing campaign to invite future clients and collaborators to the facility and showcase the capabilities of each type of demonstration use. This has generated valuable initial insight on what the facility can be used for in the future, that can help unlock a design, planning, or approvals issue. In its success so far, it has helped offer understanding and buy-in for future users of a large broadcasting studio, understand the impact of new proposed planning applications in a city environment, and the impact of a new rail noise line on rural locations.

 

User responses have been collated to improve the demonstration offering, from both a visualisation and acoustic perspective. For the audio, a wave field synthesis array can be explored as an alternative system, which has a key advantage of being able to increase the spatialised listening zone. Furthermore, real-time audio applications have been studied, so that a user can listen back to themselves in a modelled acoustic environment in real time, enhancing user interaction.

 

For the visualisations, studies are being made that can allow an import of large datasets so that they can be visualised and used to understand complex data patterns. An example of this is to import Computational Fluid Dynamics (CFD) or pedestrian simulations into a modelled demonstration.

 

These focused next steps will help continue to evolve the realism and user interaction within the facility, that will further demonstrate a holistic experiential design offering.

 

6 REFERENCES

 

1. E. Schwartz, Exploring Experience Design, Birmingham: Packt Publishing, 2017.
2. D. Hiller, A. L. Maldonado, V. Jurdic, H. Harris, C. Heggie, P. Hermans, C. Sharp, A. Thomas and J. Woodcock, “A 10 Year Retrospective and Look Ahead At The Use of Sound And Visualisations For Stakeholder Engagement And Understanding Perceptions Of Disruptive Technologies,” in European Rotocraft Forum, 2021.
3. Arup, “Arup.com,” [Online]. Available: https://www.arup.com/projects/virtual-reality soundbooths. [Accessed 1 October 2022].
4. Mayor of London, “London View Management Framework, Supplementary Planning Guidance,” Greater London Authority, London, 2012.