Testing Audio Performance of Hearables

Picture of AES paper on testing hearables

Testing Hearables AES Paper

Testing hearables, or smart headphones, is challenging. They have various interfaces ranging from hardwired to wireless and often contain signal processing on both the record and playback side. This means that their characteristics change according to ‘real world’ conditions such as their physical environment and background noises. Furthermore, their multifunctional nature means that there are many aspects of the device that may need to be tested, ranging from voice recognition to music playback or even operation as a telephone headset or hearing aid. In this AES paper, we discuss how to implement basic acoustic tests as well as the more complex real-world tests, techniques, standards, and equipment that are necessary.

Authors: Steve Temme, Listen, Inc.
Presented at AES Headphone Conference 2019, San Francisco, CA.

Full Paper
Poster Presentation

 

Abstract & introduction for “Testing Audio Performance of Hearables”

Abstract for “Testing Hearables”
Smart headphones or “hearables” are designed not only to playback music but to enhance communications in the presence of background noise and in some cases, even compensate for hearing loss. They may also provide voice recognition, medical monitoring, fitness tracking, real-time translation and even augmented reality (AR). They contain complex signal processing and their characteristics change according to their smartphone application and ‘real world’ conditions of their actual environment, including background noises and playback levels. This paper
focuses on how to measure their audio performance under the many various real-world conditions they are used in.

Introduction for “Testing Hearables”
Hearables are notoriously challenging to test. They have various interfaces ranging from hardwired to wireless (e.g. Bluetooth) and may contain much signal processing, both on the record side (e.g. beamforming, background noise filtering, voice activity detection, and on the playback side (e.g. loudness, compression, equalization, and active noise cancellation). This means that their characteristics change according to ‘real world’ conditions such as their physical environment and background noises. Some even have wake word detection, e.g. ‘Hey Siri’. Furthermore, their multifunctional nature means that there are many aspects of the device that may need to be tested, ranging from voice recognition to music playback or even operation as a telephone headset or hearing aid. Due to their complex non-linear use cases, these devices often need to be tested at different levels and in different environmental conditions, for example with background noise, different signals etc. Although, there are currently no standards for testing smart devices such as hearables, we can borrow principles and test configurations from many other audio devices and use existing standards such as; IEC for headphones [1], IEEE for headsets [2], ETSI for background noise [3], TIA/ITU for telephone test [4] and ANSI for hearing aids standards [5].

Flexibility of the test system and experience with testing a wide range of acoustic devices is critical to enable a device to be completely characterized. This paper discusses how to implement basic acoustic tests and some of the more complex real-world tests along with the techniques and standards that may be used. Test system requirements for measuring voice
enabled hearables will also be discussed.

Full Paper

More about Headphone & Hearables Testing

The Correlation Between Distortion Audibility and Listener Preference in Headphones

Picture of paper on listener preference & distortion audibility in headphones

Listener Preference & Distortion Audibility in Headphones

The correlation between listener preference and distortion audibility is investigated in this AES paper from Steve Temme, Dr. Sean Olive et al. Five popular headphones with varying degrees of distortion were selected and equalized to the same frequency response. Trained listeners compared them subjectively using music as the test signal, and the distortion of each headphone was measured objectively using SoundCheck. The correlation between subjective listener preference and objective distortion measurement is evaluated and discussed.

Authors: Steve Temme, Sean E. Olive*, Steve Tatarunis, Todd Welti*, and Elisabeth McMullin*            *Harman International
Presented at the 137th AES Conference, Los Angeles 2014

Full Paper

 

 

Listener Preference & Distortion Paper Abstract & Introduction

Abstract
It is well-known that the frequency response of loudspeakers and headphones has a dramatic impact on sound quality and listener preference, but what role does distortion have on perceived sound quality? To answer this question, five popular headphones with varying degrees of distortion were selected and equalized to the same frequency response. Trained listeners compared them subjectively using music as the test signal, and the distortion of each headphone was measured objectively using a well-known commercial audio test system. The correlation between subjective listener preference and objective distortion measurement is discussed.

Introduction
There has been much research published on how a loudspeaker’s linear performance, e.g. frequency, time and directional responses, affects perceived sound quality. However, there is little research published on how non-linear distortion affects perceived sound quality. In recent years, the increasing availability and affordability of high quality headphones and personal digital music
players e.g. MP3 players, has brought high quality music playback to the masses. The transducer performance is critical to listener enjoyment and Dr. Olive and others have presented research on what they believe the target frequency response of the headphone should be for optimum sound quality [1]. The attempt of this research is to determine what level and what kind of distortion is audible and how it affects the perceived sound quality.

Five different pairs of good quality over-the-ear headphones with varying levels of distortion were objectively measured and subjectively rated for their perceived sound quality. First, each headphone was equalized to the same target frequency response. Several different kinds of distortion metrics including harmonic, intermodulation, and non-coherent distortion, were measured for each headphone. A listening test was then conducted where the five headphones were rated by eight trained listeners based on preference and distortion using four short musical excerpts. The program material was selected for wide dynamic and frequency ranges to excite mechanisms in the headphone transducers that would cause distortion.

The different headphones were presented virtually to listeners via binaural recordings of the headphones reproduced through a calibrated low-distortion reference headphone, Stax SR-009. This virtual headphone test method minimized headphone leakage effects, and removed the influence of non-auditory biases (brand, price, visual appearance, comfort, etc.) from listeners’ judgment of sound quality. In this paper, correlations between subjective and objective ratings of distortion are examined (as was done previously [2]) in an attempt to develop an objective metric for measuring distortion audibility in headphones and other loudspeakers. This could possibly be extended to other types of audio devices such as amplifiers.

 

More about Headphone Testing using SoundCheck