Using psychoacoustic parameters to select suitable sounds to augment soundscapes for people with dementia.

Arezoo Talebzadeh 1 Timothy Van Renterghem 2 Pieter Thomas 3 Paul Devos 4

Dick Botteldooren 5

Ghent University

Department of Information Technology, WAVES Research Group Technologiepark 126, B 9052 Gent-Zwijnaarde, Belgium

ABSTRACT

People with dementia have difficulties identifying time and space, and any disturbing noise or unfa- miliar sound can be agitating and annoying for them. Sound augmentation as an intervention was shown to improve mood and cognitive behaviour in people. In addition, this approach has a positive effect on reducing anxiety, stress, and agitation and improving sleep quality in people with cognitive disabilities. In the soundscape approach, people have agency in evaluating their sonic environment. This method is hardly possible when designing for people with dementia, as the severity of the disease makes communication incomprehensible in most cases. Therefore, caregivers and nurses are the best sources of evaluation; their familiarity with residents and their knowledge of residents’ behaviour and psychology are crucial in evaluating the soundscape.

This research uses feedback data from the caregivers and psychoacoustic parameters of sound to find ways to select suitable sounds that positively affect people with dementia. A logistic regression model with a single independent variable demonstrated the chance of a positive outcome (sound) versus a continuous indicator value (psychoacoustic parameter). The result shows that specific psychoacoustic indicators, such as sharpness, percentiles, and centre of gravity (COG), can result in a positive eval- uated response in sound augmentation.

1 Arezoo.talebzadeh@UGent.be

2 Timothy.VanRenterghem@UGent.be

3 Pieter.Thomas@UGent.be

4 P.Devos@UGent.be

5 Dick.Botteldooren@UGent.be

21-24 AUGUST SCOTTISH EVENT CAMPUS ? O? ? GLASGOW

1. INTRODUCTION

The effect of soundscape on people with dementia has been studied [1], and the relation between soundscape and BPSD (behavioural and psychological syndrome of dementia) is well known [2]. Sound is an essential sensory stimulus, especially for people with cognitive difficulties. Sound is significant in making people aware of their environment [3]; also, sound gives the “sense of place” [4]. However, when the sonic environment is unfamiliar, it adds to the anxiety of those who receive the sound, making the situation annoying and unpleasant. People with dementia feel isolated, lost, and lonely, and any unfamiliar sound or disturbing noise can be agitating and disturbing. Dementia is a broad name for symptoms caused by brain disorders. “Symptoms commonly include loss of short- term and long-term memory, judgment and reasoning, and changes in mood, behaviour and the ability to communicate” [5]. Symptoms of dementia affect a person’s ability to be socially active, work and perform daily tasks.

Dementia is known to be more common in older adults. As a result, people with dementia either live in long-term care (LTC) facilities or have to relocate to LTC to reduce care responsibilities from their families. LTC facilities feel unfamiliar and usually are not customized for individual needs, making them more agitating and disturbing, especially for people with mental illnesses. Residents typically have a private room (or a room to share with a roommate). Still, all other activities happen in shared spaces, from dining and social events to taking showers which usually take place in a shared (not private) facility. These spaces are designed commonly to be functional and are not intimate. Sensory perception in these spaces is unfamiliar for residents: light, sound, temperature, and smells may differ from the familiar setting of one’s home. Strange sensory stimuli add to the anxiety and annoyance of residents. Sound after smell is the most potent sensory stimulus [6] in changing mood, so it is essential to design a soundscape that promotes a positive attitude. Vulnerable populations are usually affected by their environment; the sense of place directly relates to auditory information and clarifies the location and situation [4].

The soundscape [7] is the acoustic environment perceived and experienced by a person in a spe- cific context [8]. This phenomenon depends on individuals’ listening habits and their relation to the environment; different people in the same environment may have a contrasting relationship to the soundscape [9] and, therefore, entirely different emotional responses to the same soundscape [10]. Research has shown the positive effect of natural and non-natural soundscape on people with severe or profound intellectual disabilities [11]. The same study also showed that natural sounds such as those found in forests and near beaches promote relaxation and interest in people with severe cogni- tive disabilities. Sound also generates a feeling of safety [11], influencing moods and triggering a specific action [12]. Augmenting soundscape for this purpose can improve the behaviour; adding (human-preferred) sound to the acoustic environment indicates “augmented soundscape” [13]. Soundscape can be seen as a positive environmental factor in improving health and well-being.

Research shows the effect of natural sound on attention restoration [14] and the importance of sound in making sense of place [15]. Memory plays an essential role in soundscape perception and reflects the interaction between a person and their environment [16]. Designing soundscapes for peo- ple with cognitive difficulties is challenging and requires an understanding of sound characteristics and human interaction with their sonic environment.

21-24 AUGUST SCOTTISH EVENT CAMPUS ? O? ? GLASGOW

People with dementia may not be able to communicate their feeling verbally; nurses and caregivers are the best sources as they have a good understanding of non-verbal reactions and the state of resi- dents. This paper looks at a sound selection method to augment a suitable soundscape for people with dementia, using the feedback data from the nursing home as the source of evaluation. In a study by Devos et al. [1] at Flanders nursing home, a designed soundscape intervention was delivered through speakers at a specific time during the day. The soundscape was a combination of different natural and human-made sounds chosen by soundscape researchers. Then the nursing staff evaluated the effect of the soundscape on residents through feedback buttons.

Although the feedback data is subjective, the nursing staff were encouraged to focus on residents’ reactions and not their own when evaluating the soundscape.

2. METHODOLOGY

2.1. Sound Selection

For making the sound database for this research, 218 sounds were collected either by on-location recording (27 sounds) or existing sound databases (191 sounds). The existing databases are FreeSounds (137 sounds), BBC sound effects (27 sounds), MusOpen (21 sounds), BenSound (3 sounds), YouTube (2 sounds) and ElectroBel (1 sound).

This collection of sounds includes natural sounds, anthropogenic sounds (from human activities) and music. In the preliminary stage, all sounds are labelled based on three categories of nature, man-made and music. This initial categorization is meant to be used as meta-information to validate the characterization of the sound database. Furthermore, all sounds are subcategorized into animals, birds, weather, water, wind, environment night for nature, clock, wind chime and transport for an- thropogenic sounds. Music stayed as one general category. All selected sounds had either non-com- pressed (wav, ac) or compressed formats (mp3) and were converted into two-channel MPEG-1 layer three files (mp3 “joint stereo”) at a sample rate of 4000 Hz with a constant bit rate (CBR) of 192 kbps using Adobe Audition software.

2.2. Sound Analysis

The sounds were analyzed using acoustic and psychoacoustic metrics. The sounds were character- ized in terms of level for Z, A, C-weighting, including continuous equivalent sound pressure (L eq , LA eq , LC) and percentiles L x with x= 5, 10, 25, 50, 75, 90, and 95, where L 5 and L 10 are the usual estimates of maximum level and L 90 and L 95 of minimum level.

All metrics were obtained from the full-duration sounds (not excluding the background noise). For psychoacoustic metrics, loudness, fluctuation strength, roughness and sharpness were analyzed. In addition, saliency, music likeness, the centre of gravity, number of events above LA 50 , danceability, beats per minute, and spectral complexity were analyzed.

Music Likeness (ML) is a metric that defines whether a sound is likely to be “musical” based on a low-frequency analysis. Saliency or sensory saliency is related to how much a sound stands out from a surrounding environment. The centre of gravity corresponds to the frequency that divides the spectrum into two halves such that the amount of energy in the top half (higher frequencies) is equal to that in the bottom half (lower frequencies). A sound with much high-frequency energy will have a significant value for the centre of gravity. The number of events above LA 50 is defined as the number

21-24 AUGUST SCOTTISH EVENT CAMPUS ? O? ? GLASGOW

of events 3-dB above the median A-weighted level for at least 3s. Danceability is a parameter esti- mated from the slope of the transients present in an audio signal.

2.3. Sound Evaluation

To choose suitable sounds for the personalized soundscape, six soundscape researchers (geriatric psychologist, occupational therapist, acoustic engineer, bioacoustics engineer and architect) reviewed a set of 218 sounds. The researchers evaluated the suitability of each sound for 17 different activities by rating them 0 (not suitable), 1 (“maybe”), or 2 (suitable). Also, they rated the degree of suitability of the sound for safety-enhancing, mood changing, or triggering behaviour on a five-point scale rang- ing from “not at all” to “very much.”

2.3.1 List of Activities

Tale 1 shows the list of activities used for the evaluation of sounds.

wake up wash & dress have breakfast

go to the toilet take medication eat lunch

drink coffee (coffee time) dinner fall sleep

sleeping rest or sleep take a bath or shower

expect social activities doing social activities * expect visitors in the room

having a visitor in the room* perform personal activities

Table 1: List of Activities

Two activities (*) were eliminated during the evaluation as there received almost no ratings. The average rating for sounds per activities then calculated. Sounds with an average of >1.2 were selected for level 2, and those with an average rating of >0.8 were chosen for level 1. Level 2 sounds would be a priority to play for a specific activity.

The team was aware of their biases during the rating process. None of the experts were diagnosed with cognitive disorders (such as dementia), and they listened to sounds in their comfort place, mainly through headsets. However, the diversity of their age, gender, ethnicity, professional background, knowledge of soundscape and dementia, and years of studying the effect of soundscape on people and perception of the sonic environment gave credit to their evaluation. In the end, 101 sounds were selected for this study.

2.4. Sound Player System

1. A dedicated sound playing device with a remote connection to the server to receive updated soundscape daily based on the feedback loop.

2. Feedback buttons with wireless connection for sound evaluation. A panel of 5 feedback but- tons for rating the participant’s behaviour by the staff and an additional snooze button to mute the sounds when necessary.

3. The web-interface gives access to the overall soundscape control. It allows for the initial com- position of the soundscape and the daily schedule of the different soundscape player systems. The interface connects to a cloud-based server and provides site-level control of the system, allowing the

21-24 AUGUST SCOTTISH EVENT CAMPUS ? O? ? GLASGOW

activation and deactivation of the players. This software program delivers the soundscape and uses the feedback from the button panel to personalize the soundscape.

The system is designed to obtain a personalized soundscape based on staff feedback through the button panel, which is transferred to the remote server. This server has the function to recommend improved sounds to be played the next day(s).

2.5. Feedback Evaluation by Caregivers

During the experiment in the nursing homes, caregivers evaluated the effect of specific sounds on the participants by pressing feedback buttons. Caregivers are very close to the residents and aware of their reactions and are thus well suited to assess residents’ agitation and stress. The feedback system uses a 5-point colour scale (green, yellow, orange, red, and black), where green should be used for strongly positively evaluated sounds and black for the non-desirable (disturbing) sounds. The algo- rithm adjusts based on the feedback system; if the feedback is negative, the system chooses another sound. The system continues playing the same sound when the input is positive or there is no feed- back. The data from the feedback buttons should mainly demonstrate the overall suitability of the sounds from the residents’ perspective. However, it cannot be excluded that the caregiver’s percep- tion and mood can affect the evaluation.

3. RESULT & DISCUSSION

Caregivers’ feedback was used to evaluate the sounds based on residents’ reactions, monitor the sounds that are the best received, and for any adverse effects of the sounds. The result of the five- button feedback system is demonstrated in Figure 1. Each column refers to a particular sound. Colours are based on a feedback system with off-white representing mute action.

100%

90%

80%

70%

60%

50%

40%

30%

20%

10%

0%

W0002

W0004

W0006

W0008

W0010

W0012

W0013

W0015

W0016

W0017

W0026

W0027

W0056

W0058

W0060

W0069

W0071

W0072

W0085

W0087

W0088

W0090

W0102

W0103

W0107

W0108

W0109

W0130

W0141

W0144

W0148

W0150

W0152

W0162

W0163

W0193

W0198

W0209

W0210

W0211

W0212

Series1 Series2 Series3 Series4 Series5 Series6

Figure 1: Caregivers’ Feedback

21-24 AUGUST SCOTTISH EVENT CAMPUS ? O? ? GLASGOW

The data from the feedback system was used to find a correlation between the characteristic of the sound and their effects on people. (Figure 2)

21-24 AUGUST SCOTTISH EVENT CAMPUS ? O? ? GLASGOW

For a start, a minimum number of buttons pressed per sound was set to 5; otherwise, the sound is disregarded.

A “positive” sound is defined as the “Green + Yellow” buttons pressed at least 60%. Some sound indicators were disregarded due to missing values.

Outliers in the indicators have been removed (based on a normal distribution with 𝜌 = .01, with a maximum of 5 removal points); this prevents a few extreme values from shaping the regression curve.

Odds ratios are calculated after pooling each indicator in two classes based on median (low value vs high value within the range of values present - shown by the vertical line in the graphs).

Logistic regression was used (outcome=positive sound or not) with a single independent variable (i.e. the psychoacoustic indicator, either continuous or dichotomous).

Median separation stats: p=0.028744 OR=3.6 Cl on OR=[1.1422-11.3467] n=52 ad re) T ° q 9° co) Chance of positive response ° ° Ss a T ° w ° 0 500 1000 1500 2000 Indicator value : beats_count 2500

The result shows the potential of using sharpness, COG, and percentile to choose a suitable sound.

Chance of positive response 2 w ° q ° co) ° S A 500 Median separation stats: p=0.022076 OR=3.8 Cl on OR=[1.2116-11.9185] n=53 WO. p- - e o 1000 1500 2000 2500 3000 3500 4000 Indicator value : COG 4500 5000

‘Median separation stats: p=0.022076 OR=3.8 Cl on OR=[1.2116-11.9185] n=53 ‘Chance of posi oa ° ° ° os 1 18 2 28 3 Indicator value : $10

‘Median separation stats: p=0.022076 OR=3.8 Cl on OR=[1.2116-11.9185] n=53 jSod yo eoueyD Indicator value : sharp75m

Figure 2: Chance that a specific psychoacoustic indicator results in a positively evaluated response in function of its value range. Only the indicators leading to a regression model statistically signifi-

cant at the 5 % level are shown. The dashed lines show the upper and lower 95% confidence inter- vals. The open circles are the actual data points. In addition, the odds ratios are shown in the case of

median separation (low vs high value) of the indicator. For example, a COG above about 1000 Hz

is 3.8 times more likely to end up with a positive response than a COG lower than 1000 Hz.

4. CONCLUSIONS

21-24 AUGUST SCOTTISH EVENT CAMPUS ? O? ? GLASGOW

People with dementia can benefit from augmented sonic environments since auditory stimulation can reduce agitation and anxiety and provide safety and familiarity. Using psychoacoustic parameters to select proper sounds for soundscape augmentation is a starting step in finding a suitable method for designing soundscapes in long-term care facilities and senior housing. This method eliminates the biases and assumptions and focuses on users’ perceptions and interests. Finding a correlation between caregivers’ feedback data and the characteristic of sounds helps augment a soundscape that fosters a healthier sonic environment for people who may feel confined inside their residential facilities. Alt- hough there is a need to study different contexts and diverse participants, the correlation result is promising. Psychoacoustic parameters might be the best characteristic of sound to design a suitable soundscape for people with cognitive difficulties.

5. REFERENCES

1. Devos, P., Aletta, F., Thomas, P., Filipan, K., Petrovic, M., Mynsbrugge, T. vander, van de

Velde, D., & de Vriendt, P. (2018). Soundscape design for management of behavioral disor- ders: a pilot study among nursing home residents with dementia . 2. Devos, P., Thomas, P., Aletta, F., Tara, ;, Mynsbrugge, V., de Vriendt, P., de Velde, D. van,

& Dick Botteldooren, ; (2019). Towards Understanding Healthy and Supportive Acoustic En- vironments: the case of a nursing home . 3. Brungart, D. S., Cohen, J., Cord, M., Zion, D., & Kalluri, S. (2014). Assessment of auditory

spatial awareness in complex listening environments. The Journal of the Acoustical Society of America , 136 (4), 1808–1820. https://doi.org/10.1121/1.4893932 4. van den Bosch, K. A., Andringa, T. C., Başkent, D., & Vlaskamp, C. (2016). The Role of

Sound in Residential Facilities for People With Profound Intellectual and Multiple Disabili- ties. Journal of Policy and Practice in Intellectual Disabilities , 13 (1), 61–68. https://doi.org/10.1111/jppi.12147

‘Median separation stats: p=0.022076 OR=3.8 Cl on OR=[1.2116-11.9185] n=53 ‘Chance of posi Indicator value : sharp50m

5. Dudgeon, Scott., RiskAnalytica., Alzheimer Society of Canada., & Gibson Library Connec-

tions. (2010). Rising tide : the impact of dementia on Canadian society : a study . Alzheimer Society of Canada. 6. Martin Lindstrom. (2005). brand sense, sensory secret behind the stuff we buy (Simon &

Schuster Bureau, Ed.). 7. Schafer R. M. (1977). The Tuning of the World . Alfred A. Knopf. 8. ISO. (2014). Acoustics-Soundscape-Part 1: Definition and conceptual framework Acous-

tique-Paysage sonore-Partie 1: Définition et cadre conceptuel . www.iso.org 9. Truax, Barry. (1984). Acoustic communication . Ablex Pub. Corp. 10. Cain, R., Jennings, P., & Poxon, J. (2013). The development and application of the emotional

dimensions of a soundscape. Applied Acoustics , 74 (2), 232–239. https://doi.org/10.1016/j.apacoust.2011.11.006 11. Andringa, T. C., & van den Bosch, K. A. (2013). Core affect and soundscape assessment:

fore-and background soundscape design for quality of life . 12. Devos, P., Aletta, F., Thomas, P., Petrovic, M., Mynsbrugge, T. vander, van de Velde, D., de

Vriendt, P., & Botteldooren, D. (2019). Designing supportive soundscapes for nursing home residents with dementia. International Journal of Environmental Research and Public Health , 16 (24). https://doi.org/10.3390/ijerph16244904 13. van Renterghem, T., Vanhecke, K., Filipan, K., Sun, K., de Pessemier, T., de Coensel, B.,

Joseph, W., & Botteldooren, D. (2020). Interactive soundscape augmentation by natural sounds in a noise polluted urban park. Landscape and Urban Planning , 194 . https://doi.org/10.1016/j.landurbplan.2019.103705 14. Ratcliffe, E. (2021). Toward a better understanding of pleasant sounds and soundscapes in

urban settings. Cities & Health , 5 (1–2), 82–85. https://doi.org/10.1080/23748834.2019.1693776 15. Adams, M., Cox, T., Moore, G., Croxford, B., Refaee, M., & Sharples, S. (2006). Sustainable

soundscapes: Noise policy and the urban experience. Urban Studies , 43 (13), 2385–2398. https://doi.org/10.1080/00420980600972504 16. Guastavino, C. (2006). The Ideal Urban Soundscape: Investigating the Sound Quality of

French Cities Perception of rhythmic similarity View project . https://www.re- searchgate.net/publication/233705298

21-24 AUGUST SCOTTISH EVENT CAMPUS ? O? ? GLASGOW