Proceedings of the Institute of Acoustics

 

Using cognitive psychology and neuroscience to better inform sound system design at large musical events

 

J. Burton, College of Science and Engineering, University of Derby, Derby, UK
A. J. Hill, College of Science and Engineering, University of Derby, Derby, UK

 

1 INTRODUCTION

 

Large musical events have become increasingly popular in the last fifty years. It is now not uncommon to have indoor shows in excess of 10,000 people, and open-air events of 30,000 people or more. These events, nevertheless, present technical challenges that have only begun to be solved in the last hundred years, with the introduction of sound reinforcement systems, electric lighting and now video/display technologies. However, these technologies present an artificial link to the performance that requires an understanding of both the audience's expectations as well as the technologies' abilities and limitations. Although many of these abilities and limitations are well documented, the audience's responses to them are less so. This paper introduces research primarily into audience auditory responses but at a subconscious level. By investigating these responses, it is hoped to find a commonality amongst audiences, from which better-informed metrics can be derived.

 

The work is intended to have a direct positive impact on the auditory experience of audience members at large open-air concerts and musical events. Events of these kinds had become increasingly popular in 2019 [1] and the post-Covid resurgence in live events is helping to re-establish the popularity of this type of public event.

 

This research looks at existing techniques, and by examining, collecting and collating examples of good practice helps reinforce that practice in a positive and practicable way. This will be of particular benefit in areas where the research is less well-coordinated; such as Music-Induced Hearing Loss (MIHL) [2]. The research builds on the authors’ ongoing research activities [3] [4]. The case study ‘The impact event sound level regulations have on sound engineering practice’ [5] is a good example of that work, where the research is based on data sets collected by the primary author over many years, which are proving valuable in helping to understand dynamic sound pressure levels at the type of events covered by this research. Initial analysis of the data has informed the current work of the World Health Organisation through the ‘Making Listening Safe’ initiative [6].

 

2 AUDIENCES AND HEARING

 

The archaic definition, dating from the 14^th century, links both ‘auditory’ and ‘audience’ to the same root word in Middle English, auditorie, a group of listeners or spectators, in turn, borrowed from the Latin audītōrius, from audīre "to hear". Through this interweaving of etymological roots, it is possible to see how the concept of audience and hearing have always been inextricably intertwined. To separate ‘audience’ from the ‘hearing’ experience would be considered by our forebears as inconceivable, so associated as they are with a shared complex human experience. The word ‘auditory’ in our modern usage is still defined as “of, relating to, or experienced through hearing” [7], but its use as a medical prefix has shifted the emphasis of this previously broadly encompassing term into one that has more specific connotations in terms of auricular hearing, pertaining to the ear as the primary source of hearing [8].

 

This research concerns itself with the auditory experience as part of a more holistic approach to perception that includes not just the ear but the entire somatosensory system. Audience perception of sound, and in particular music, is not limited to a single organ response, but instead to a more complex system of responses to changes at the surface, as well as inside the human body. By examining the processes by which we perceive music, both as individuals as well as part of a collective group, this research endeavours to define the auditory experience in terms of the factors that contribute to that experience, primarily when engaged in what could be considered the larger format style events that have become prevalent in recent years, such as open-air concerts and musical events.

 

2.1 The Evolution of the Large Open-air Concert and Musical Event

 

The audience experience has become increasingly multi-sensory at large open-air concerts and musical events. The use of light, in the form of theatrical lighting, video and pyrotechnics augments the overall experience of the modern audience; ‘the industry has undergone phenomenal changes, both in growth, consolidation, and amazing new products”. “We control smoke, haze, lasers, video graphics, strobe lights, moving lights, air graphics, conventional lights, and spotlights in a concert” [9] In this multi-faceted experience, the audience’s attention is now challenged on many fronts, however, the auditory experience remains integral.

 

3 BACKGROUND

 

Throughout the period that could be described as the modern era of open-air music (primarily 1920 onwards) and its development into the current range of events today, audiences have changed and developed in many ways. Cultural, social, economic, political and societal factors all influence an audience's response to an open-air musical event. They are, however, complex and ever-changing and therefore primarily beyond the scope of this work. What will be explored is the way humans perceive and respond to events on a physiological and experiential level. This in turn consists of conscious and subconscious experiences. The way we perceive and respond to sound, and in particular music, is helpful to our understanding of the body’s response to open-air events. Somatic and autonomic responses can help provide a quantifiable and less intellectually biased metric on which to judge aspects of the auditory experience. However, it is recognised that it is problematic to judge any auditory experience without considering the intellectual response alongside any physiological one. It is important to seek out commonalities that will be genetically long-standing and embedded in our neurological and psychological systems rather than our quixotic social consciousness.

 

The research consists primarily of three strands. The objective of each strand is to examine a particular aspect of the audience experience, the current knowledge and understanding of that area and, by doing so, identify any key areas that are missing, or where further research may be required. Each strand encompasses several different areas of science, outlined here under the following areas:

• Hearing/feeling
• Hearing/seeing
• Hearing/pleasing.

These divisions are purely for explanatory purposes and are inextricably intertwined, but they help to organise the main strands of the research.

 

3.1 Hearing/Feeling

 

Hearing is the ability, by the detection of vibrations, primarily within the ear to perceive sound. The ear is the primary source of auditory information, but not the only source. This work is on how people experience open-air events, with an emphasis on the auditory experience. This experience will be in the broadest sense of the word, encompassing not just information received via the cochlea but the entire body.

 

Vibrations, including those associated with the musical experience, are perceived by the body's somatosensory system. The somatosensory system consists of four main types of receptors: nocireceptors (pain), proprioceptors (body position), thermal receptors (cold and warmth) and mechanoreceptors (e.g. vibration, pressure, stretching). There are four different mechanoreceptor types in glabrous (hairless) skin: Merkel's Receptors, Ruffini's corpuscles, Meissner's corpuscles and Pacinian corpuscles as seen in Figure 1 [10].

 

 

Figure 1: Illustration of the mechanoreceptors in the hair and hairless skin of primates. Merkel's receptors and Meissner's corpuscles can be found close to the surface. Pacinian corpuscles and Ruffini's corpuscles lie deeper in the tissue [10] adapted from Schmidt [11]

 

Meissner corpuscles are most sensitive to vibration below 40Hz, at higher frequencies, the Pacinian corpuscles dominate perception [12]. Vibration detection in the skin by these mechanoreceptors, bone conduction and body resonances (such as in the chest cavity) [13] all contribute to the perception of airborne low-frequency noise. An investigation into sound and vibration perception of both hearing and profoundly deaf people showed an ability to perceive noise through the surface of the subject’s body, both deaf and hearing if the sound pressure level (SPL) was in excess of normal hearing thresholds. The research by Yamada, Ikuji, Fujikata and Watanabe [14] indicates that the perception of sound is not confined to the ear mechanisms or auditory systems. Moller and Pederson [15] summarise that “hearing becomes gradually less sensitive for decreasing frequency, but there is no specific frequency at which the hearing stops”. They saw a reasonable agreement between studies of hearing thresholds, represented down to 20Hz by the International Organization for Standardization (ISO) but they also proposed a “normal threshold one decade further down in frequency” to 10Hz.

 

Leventhall [16] noted that the majority of studies into low-frequency noise use a spring-mass model for mechanical vibration into the feet or seat. A spring-mass model is a simple harmonic motion with an assumption that the amount of stretch is proportional to the restoring force, and that the mass glides easily without loss of energy. As Leventhall states in his work on “Somatic Responses to Low Frequency Noise” the spring-mass model is not suitable for body vibration generated by low frequency airborne vibration as it is a compressive effect on the whole body and consequently a more complex scenario. In their study of music-induced vibrations, Merchel and Altinsoy [17] concluded that mechano-sensation effects of low-frequency content at music concerts, which can be felt through the feet or seated area, are predominantly airborne vibrations. In open-air festivals there are limited non-airborne vibrations as the audience is standing, the ground is solid, and is in all practical terms non-resonant at the sound pressure level experienced by the audience. Where audiences are standing, non-airborne vibration is limited to that through the feet, so whole-body vibration (WBV) is predominantly airborne. Physical responses to sound are an integral part of the listening process. The link between WBV and aural perception has been well established and is detailed well by Walker, Sungyoung, and Martens [18].

 

3.2 Hearing/Seeing

 

The link between sight and hearing is well documented and again a holistic viewpoint is beneficial to appreciate the audience's experience at events. S. Connor, in ‘Edison’s Teeth: Touching Hearing’ noted; “The senses are multiply related, we rarely if ever apprehend the world through one sense alone. Indeed, under conditions in which one sense predominates, closer inspection may disclose that the predominating sense is in fact being shadowed and interpreted by other apparently dormant senses” [19].

 

Modern events, as already mentioned, are multi-dimensional. Lighting, video and pyrotechnics are now regularly incorporated into the audience experience. The audience no longer expects a musical performance with little or no accoutrements, but rather a full production. It has been suggested on occasion that the music is no longer the primary feature of musical events, but takes second place to the visual production. “Lighting at a rock show is one of those things that most people don’t consciously think about but can strongly impact their experience. Through lighting techniques, a stadium show can feel as intimate as a small club show. The audience can be made to feel inspired, disturbed, moved, or impressed, depending on the emotions that the artist is trying to communicate”[20].

 

However, the spectacle of the production, the lights and the special effects, although important to the experience are again subject to our intellectual response. The perception of sound, however, and in particular the localization of sound is possible independent of sight. We do not need to see to hear, to be able to localize and focus on sound. Sound localization requires the integration in the brain of auditory spatial cues generated by interactions with the external ears, head and body. The relative weighting of these cues can change in relation to other stimuli; vision, touch (for proximal stimuli), and smell. However, vision tends to dominate localization judgements and can induce a ‘recalibration of auditory space’ if the auditory spatial cues and visual information become misaligned [21]. In their research, Kumpkin and Campbell noted “the remarkable plasticity of the auditory system. In practical terms this allows the audience to associate the sound they hear with the spectacle they see, despite the possible disparity of the two systems caused by the different arrival times of soundwaves and lightwaves.” [22].

 

The listener responds to aural clues as part of our innate ‘fight or flight response’. This reaction, first described by Cannon in 1929 [23] involves a series of neural and physiological mechanisms that rapidly activate the body to confront the threat (fight) or to escape it (flight) [24]. The auditory system, as part of this mechanism, is highly attuned to making decisions based on the cues available as to the source of any sound. It is also fallible, as anyone attempting to pinpoint the direction of an advancing siren knows, unable to localize the sound. In the concert environment, this potentially disconcerting aural disconnect requires further investigation.

 

3.3 Hearing/Pleasing

 

The link between music and well-being is well-established. Lonsdale and North [25] identified the various functions fulfilled by music, including pleasure, tension reduction, identification with a subculture, and socialising. Three studies compared the neurochemical effects of music, based on experimenter-selected criteria, including style, rhythmic properties, and arousal/valence dimensions. In one study, the experimenter selected classical music that the experimenter deemed relaxing (60-100 beats/min; soft sounds) this was compared to techno music that the experimenters deemed stimulating (130-200 beats/min; screeching sounds). Subjects in this study reported no preference or engagement with a particular musical genre. Techno music increased plasma cortisol, Adrenocorticotropic hormone (ACTH), prolactin, growth hormone and norepinephrine levels [26]consistent with heightened hypothalamic–pituitary–adrenal (HPA) axis and sympathetic nervous system activity [27]. The production of cortisol is associated with the ‘fight or flight’ mechanism and yet the music does not have the same inherent association with stress. A study by Papinczak et al [28] of young people found they strongly believed that music played an integral role in their daily lives and was an important aspect of their well-being.

 

Music, and musical performance, are the drivers in this investigation and it is that association with hearing/pleasing that underpins the research.

 

4 METHODOLOGY

 

The bulk of the early research for this project (which extends far beyond the scope of this introductory paper) consisted of a review of key literature with a focus on the three strands detailed in Section 3. By examining current knowledge and understanding, gaps in that knowledge have inevitably appeared. Many aspects of this research are inevitably beyond the scope of this study, however, where a key point is identified, future areas of research are explored and detailed.

 

The primary author, having been involved in this field for over 40 years, has identified and explored, through professional practice, many of the areas of interest. The wide scope of the subject area is therefore not one that is being explored for the first time, but rather being revised through the lens, not of a practising professional, but of an academic.

 

4.1 Initial Areas of Investigation

 

Some initial areas of investigation have been identified:

• Dynamic mixing: Calculating the Just Noticeable Difference (JND) for a rise or fall in sound pressure level in playback material, and its perceptive effect in terms of a positive or negative listening experience.
• Low-frequency content and preferred listening levels: effect of sound pressure level (SPL) and the effect of bandwidth extension (continuation of authors’ previous work [29]).
• Mechano-sensation: Examination of the JND of perceptible frequencies in the region 96 - 100dB SPL.
• Vertical localization: Vertical perception and the aural/visual disconnect when utilising flown loudspeaker systems.
• Sensory stimulus alignment: Alignment of synchronous audio, visual and tactile responses over extended audience areas.
• Sound reinforcement sub-systems: Audience perception of sound sources in multiple speaker sound reinforcement systems.
• Structure-borne transmission: Low-frequency sound transference through solid ground and its effect on audience perception.
• Crossover networks: Effect of spectral separation in multiple speaker system deployments.

 

5 INITIAL RESEARCH

 

5.1 Pilot Study

 

A pilot study took place in three medium-sized venues (2000 – 3000 capacity) in September 2022. The experiment investigated the preferred listening levels (PLLs) of eighteen self-declared experienced and five inexperienced listeners.

 

5.1.1 Methodology

 

A pre-prepared audio multitrack was mixed through a sound reinforcement system set up for a conventional rock show in each venue under investigation. Participants were asked to combine the six audio elements: drums, bass, keyboards, vocals and a brass section, to produce a balanced mix at an SPL they felt was appropriate for performance in that venue, one that could be enjoyed by a large audience. No further prompting was given. Instructions on how to combine the recorded elements were given but no guidance as to the SPL required. The levels were combined using a generic midi device controlling track levels in the software Ableton Live [30]. The controller provided no visual feedback to indicate the audio level and all tracks started at -∞ dB, silent.

 

The participants proceeded to adjust the track until they were happy with the mix, then using a master control they were able to move the level of the entire mix up and down until they were satisfied that the level was appropriate for a performance in that venue. The sound pressure levels were monitored and recorded using a 10Eazy sound level meter [31] and a note of the SPL as one-minute A and C weighted averages were recorded (L_Aeq,1min, L_Ceq,1min). The tracks were allowed to run for approximately 20 seconds after the participants preferred level had been reached to help produce a more accurate and representative Leq measurement.

 

By using a multi-track rather than a WAV, or similar stereo recording file format, it was hoped that the participants would become more invested in the critical listening process. This was reflected in many of the comments received during the debriefing. The experiment was found to be enjoyable, as well as challenging, all participants were felt to have engaged with the process well.

 

5.1.2 Data Analysis

 

An initial study of the data has shown a mean average of 85 dB L_Aeq,1min (Figure 2a) for non experienced listeners and 90dB L_Aeq,1min for those with more experience (Figure 2b), and 88.4 dB L_Aeq,1min for all participants (Figure 2c),. The background noise levels (BNL) during the testing, which took place in the working venue environment was 58 dB L_Aeq,1min. An average BNL of 84 dB L_Aeq,1min were recorded during the evenings when the audience was in attendance. Only the A-weighted results have been analysed, however, C-weighted data was collected for future study.

 

 

Figure 2: Box and whisker plots for the preferred listening level of inexperienced listeners (2a), experienced listeners (2b) and all participants (2c). The vertical axis indicates the A-weighted sound pressure level averaged over one minute (L_Aeq,1min).

 

5.1.3 Pilot Study Conclusions

 

The results of the pilot study appear to support the robustness of the testing method. The test has limitations in terms of only providing a preferred level for a single piece of popular western music. Participants who are familiar both with the style, and or the piece itself, may introduce bias, and preconceptions based on prior experience. However, the test has produced higher preferred sound levels (Figure 2c) than experiments that have relied on replaying stereo WAV files [32], possibly due to the more immersive experience of creating the mix.

 

The relationship between the BNL of the venue with an audience in-situ and the PLL of the mix level requires greater analysis. In the experimental situation the signal-to-noise ratio (SNR) between the BNL and averaged PLL was 30.4dB L_Aeq,1min. The mix level during the evening performances averaged at 100dB L_Aeq,1min. This equates to an SNR of 16dB. The evening mix level is in line with previous data collected by the authors [3]. An examination of the dynamic capabilities of the hearing system, in particular our ability to accommodate high SPLs, may help provide insight as to whether the increase in SPL during the concert was in line with PLLs in terms of the ratio of the increased background noise and subsequent elevated show level. Increased audience participatory noise [33], and the onstage noise produced by the musicians’ instruments both contribute to potentially greater SPL.

 

Greater study is needed to explore the findings and implications produced by this initial experiment.

 

5.1.4 Future Work

 

It is hoped, in future studies, to increase the number, and range, of participants. A larger study group with a wider number of novice or inexperienced listeners is required, along with a better gender balance. A long-term study will commence this autumn, looking at live event engineering undergraduate students to see if their PLLs increase as they gain experience during the course of their studies.

 

6 CONCLUSION

 

The overall research project is wide-reaching, focusing on a large range of factors that affect the experience of audience members at large open-air concerts and musical events, with an emphasis on the auditory experience. This project is the culmination of forty years of the primary author as a practitioner in the field of sound engineering, primarily as a live-concert touring engineer. During that career, the author developed dissatisfaction with the available ‘science’ upon which to base fully rounded decisions regarding the audience experience, and in particular that most closely associated with their practice, the auditory experience. By gaining a better understanding and creating a fuller picture of the specific scenario of a large open-air concert or musical event, such an understanding will become closer to being realised.

 

The multi-stranded approach of this investigation encompasses not just research but also experimentation. Many aspects of the audience experience have not been fully investigated, either due to their novelty or lack of interest in the scientific community. This work endeavours to fill some of those gaps. The work described within this paper offers early insights into the challenges faced and potential direction(s) of travel for the wider project over the following years.

 

The driver for this work is a love of music and a passion to share music with as many people as possible. Underlying that is a mantra of “democracy for listeners”, providing as equitable an experience as possible to every member of the audience, the aim of any good sound engineer or sound system designer. With this work, there is the hope that more knowledge and understanding will become available in the quest for that ‘democracy’ through an understanding of our ‘commonality’ as human beings, which transcends cultural and intellectual borders.

 

7 REFERENCES

 

1. Mintel. Raving Mad: UK music festival attendance at the highest level of four years. 2019.

2. Zhao F, Manchaiah V, French D, Price S. Music exposure and hearing disorders: An overview. International Journal of Audiology. 2010; 49:54–64.

3. Hill AJ, Mulder J, Burton J, Kok M, Lawrence M. Sound Level Monitoring at Live Events, Part 1 – Live Dynamic Range. JAES. 2021; 69(11):782–792.

4. AES Conference Papers Forum – Education and Certification in Sound Pressure Level Measurement, Monitoring and Management at Entertainment Events. Available from: https://secure.aes.org/forum/pubs/conferences/?elib=21227. Accessed 2022 Apr 2.

5. Hill A, Burton J. A case study on the impact list event sound level regulations have on sound engineering practice. IOA, Reproduced Sound, Bristol, Nov 2020.

6. WHO. WHO global standard for safe listening venues and events. 2022. Available from: https://www.who.int/publications-detail-redirect/9789240043114. Accessed 2022 Mar 14.

7. Merriam-Webster. Definition of AUDITORY. 2021. Available from: https://www.merriam-webster.com/dictionary/auditory. Accessed 2022 Jan 24.

8. Merriam-Webster. Definition of AURICULAR. Available from: https://www.merriam-webster.com/dictionary/auricular. Accessed 2022 Jan 24.

9. Moody JDP. Concert Lighting: The Art and Business of Entertainment Lighting. 4th ed. London: Taylor & Francis; 2016.

10. Merchel S. Auditory-Tactile Music Perception. Herzogenrath, Germany: Shaker Verlag; 2014.

11. Schmidt R. Physiologie Des Menschen. 32nd ed. Berlin: Springer-Verlag; 2019.

12. Mountcastle V. The Sensory Hand: Neural Mechanisms of Somatic Sensation. Cambridge, USA: Harvard University Press; 2005.

13. Peters A, Abrams A, Gerhardt K, Griffiths S. Transmission of airborne sound from 50 to 20,000 Hz into the abdomen of sheep. J. Low Freq Noise Vibration. 1993; 12:16–24.

14. Yamada S, Ikuji M, Fujikata S, Watanabe T. Body sensations of low-frequency noise of ordinary persons and profoundly deaf persons. J. Low Freq Noise, Vibration and Active Control. 1983; 2:32–36.

15. Møller H, Pederson C. Hearing at Low and Infrasonic Frequencies. J. Noise & Health. 2004; 6(23):37–57.

16. Leventhall H. Somatic Responses to Low-Frequency Noise. In: Proceedings 12th International Meeting: Low-Frequency Noise and Vibration and its Control. Bristol, 2006.

17. Merchel S, Altinsoy M. Music-Induced Vibrations in a Concert Hall or Church. Archives of Acoustics. 2013; 38(1):13–18.

18. Walker K, Sungyoung K, Martens W. Perception of Simultaneity and Detection of Asynchrony Between Audio and Structural Vibration in Multimodal Music Reproduction. Paris: AES; 2006.

19. Connor S. Edison’s Teeth: Touching Hearing. In: Erlmann V, ed. Hearing Cultures: Essays on Sound. London: Routledge; 2004. p.153.

20. Bonde EO, Hansen EK, Triantafyllidis G. Auditory and Visual based Intelligent Lighting Design for Music Concerts. EAI Endorsed Transactions on Creative Technologies. 2018; 5(15).

21. Wozny DR, Shams L. Recalibration of Auditory Space following Milliseconds of Cross-Modal Discrepancy. J. Neurosci. 2011; 31(12):4607–4612. doi:10.1523/JNEUROSCI.6079-10.2011.

22. Kumpkin D, Campbell C. Re-weighting of Sound Localization Cues by Audiovisual Training. Front. Neurosci. 2019; 13:1–22.

23. Cannon WB. Bodily Changes in Pain, Hunger, Fear and Rage. Oxford, UK: Appleton; 1929.

24. Teatero M, Penny A. Fight-or-Flight Response. In: Milosevic I, McCabe R, eds. Phobias: The Psychology of Irrational Fear. ABC-CLIO; 2015.

25. Lonsdale A, North A. Identify the various functions fulfilled by music, including: pleasure, tension reduction, identification with a subculture, and socialising. British Journal of Psychology. 2011; 102(1):108–134.

26. Gerra G et al. Neuroendocrine responses of healthy volunteers to “techno-music”: relationships with personality traits and emotional state. Int J Psychophysiol. 1998; 28:99–111.

27. Chanda M, Levitin D. The Neurochemistry of Music. Trends in Cognitive Sciences. 2013; 17(4):179–193.

28. Papinczak Z et al. Young People’s Uses of Music for Wellbeing. Journal of Youth Studies. 2015; 18:1119–1134.

29. Burton JG, Murphy DT, Brereton JS. Perception of Low-Frequency Content of Amplified Music in Arenas and Open-Air Music Festivals. Audio Engineering Society; 2017.

30. Ableton Live. What is Live? Available from: https://www.ableton.com/en/live/what-is-live/. Accessed 2022 Oct 1.

31. 10eazy Home. 2020.

32. Tereping A-R. Preferred Sound Level for Concert Listeners and Correlations between Sound Quality Dimensions. Audio Engineering Society; 2015.

33. Kok M, Hill AJ, Burton J, Mulder J, Lawrence M. The Influence of Audience Participatory Noise on Sound Levels at Live Events. Glasgow; 2022.