Challenges of IoT Smart Speaker Testing

Quantitatively measuring the audio characteristics of IoT (Internet of Things) smart speakers presents several novel challenges. We discuss overcoming the practical challenges of testing such devices and demonstrate how to measure frequency response, distortion, and other common audio characteristics. In order to make these measurements, several measurement techniques and algorithms are presented that allow us to move past the practical difficulties presented by this class of emerging audio devices. We discuss test equipment requirements, selection of test signals and especially overcoming the challenges around injecting and extracting test signals from the device.

Authors: Glenn Hess (Indy Acoustic Research) and Daniel Knighten (Listen, Inc.)
Presented at the 143rd AES Conference, New York 2017

Full Paper

Evaluation of Audio Test Methods and Measurements for End-of-Line Automotive Loudspeaker Quality Control

In order to minimize costly warranty repairs, automotive manufacturers impose tight specifications and a “total quality” requirement on their part suppliers. At the same time, they also require low prices. This makes it important for automotive manufacturers to work with automotive loudspeaker suppliers to define reasonable specifications and tolerances, and to understand both how the loudspeaker manufacturers are testing and also how to implement their own measurements for incoming QC purposes.

Specifying and testing automotive loudspeakers can be tricky since loudspeakers are inherently nonlinear, time variant and affected by their working conditions & environment which can be change dramatically and rapidly in a vehicle. This paper examines the loudspeaker characteristics that can be measured, and discusses common pitfalls and how to avoid them on a loudspeaker production line. Several different audio test methods and measurements for end-of-the-line automotive speaker quality control are evaluated, and the most relevant ones identified. Speed, statistics, and full traceability are also discussed.

Authors: Steve Temme, Listen, Inc. and Viktor Dobos, Harman/Becker Automotive Systems Kft.
Presented at the 142nd AES Convention, Berlin, Germany

Full Paper

In-Vehicle Audio System Distortion Audibility versus Level and Its Impact on Perceived Sound Quality

As in-vehicle audio system output level increases, so too does audio distortion. At what level is distortion audible and how is sound quality perceived as level increases? Binaural recordings of musical excerpts played through the in-vehicle audio system at various volume levels were made in the driver’s position. These were adjusted to equal loudness and played through a low distortion reference headphone. Listeners ranked both distortion audibility and perceived sound quality. The distortion at each volume level was also measured objectively using a commercial audio test system. The correlation between perceived sound quality and objective distortion measurements is discussed.

Authors: Steve Temme, Listen, Inc. and Patrick Dennis, Nissan Technical Center North America, Inc.,
Presented at the 141st AES Convention, Los Angeles, CA 2015

Full Paper

The Correlation Between Distortion Audibility and Listener Preference in Headphones

It is well-known that the frequency response of loudspeakers and headphones has a dramatic impact on sound quality and listener preference, but what role does distortion have on perceived sound quality? To answer this question, five popular headphones with varying degrees of distortion were selected and equalized to the same frequency response. Trained listeners compared them subjectively using music as the test signal, and the distortion of each headphone was measured objectively using a well-known commercial audio test system. The correlation between subjective listener preference and objective distortion measurement is discussed.

Authors: Steve Temme, Sean E. Olive*, Steve Tatarunis, Todd Welti*, and Elisabeth McMullin*            *Harman International
Presented at the 137th AES Conference, Los Angeles 2014

Full Paper

Advances in Impedance Measurement of Loudspeakers and Headphones

Impedance measurement is often the sole electrical measurement in a battery of QC tests on loudspeakers and headphones. Two test methods are commonly used, single channel and dual channel. Dual Channel measurement offers greater accuracy as both the voltage across the speaker (or headphone) and the reference resistor are measured to calculate the impedance. Single Channel measurement methods are more commonly used on the production line because they only require one channel of a stereo soundcard, which leaves the other free for simultaneous acoustic
tests. They are less accurate, however, due to the test methods making assumptions of constant voltage or constant current. In this paper we discuss a novel electrical circuit that offers similar impedance measurement accuracy compared to complex dual channel measurement methods but using just one channel. This is expected to become popular for high throughput production line measurements where only one channel is available as the second channel of the typical soundcard is being used for simultaneous acoustic tests.

Authors: Steve Temme and Tony Scott
Presented at the 135th AES Conference, New York 2013

Full Paper

Measurement of Harmonic Distortion Audibility Using A Simplified Psychoacoustic Model – Updated

A perceptual method is proposed for measuring harmonic distortion audibility. This method is similar to the CLEAR (Cepstral Loudness Enhanced Algorithm for Rub & buzz) algorithm previously proposed by the authors as a means of detecting audible Rub & Buzz which is an extreme type of distortion[1,2]. Both methods are based on the Perceptual Evaluation of Audio Quality (PEAQ) standard[3]. In the present work, in order to estimate the audibility of regular harmonic distortion, additional psychoacoustic variables are added to the CLEAR algorithm. These variables are then combined using an artificial neural network approach to derive a metric that is indicative of the overall audible harmonic distortion. Experimental results on headphones are presented to justify the accuracy of the model.

Authors: Steve Temme, Pascal Brunet and Parastoo Qarabaqi
Presented at the 51st AES Conference, Helsinki, Finland, 2013

Full Paper

Measurement of Harmonic Distortion Audibility Using A Simplified Psychoacoustic Model

A perceptual method is proposed for measuring harmonic distortion audibility. This method is similar to the CLEAR (Cepstral Loudness Enhanced Algorithm for Rub & buzz) algorithm previously proposed by the authors as a means of detecting audible Rub & Buzz which is an extreme type of distortion[1,2]. Both methods are based on the Perceptual Evaluation of Audio Quality (PEAQ) standard[3]. In the present work, in order to estimate the audibility of regular harmonic distortion, additional psychoacoustic variables are added to the CLEAR algorithm. These variables are then combined using an artificial neural network approach to derive a metric that is indicative of the overall audible harmonic distortion. Experimental results on headphones are presented to justify the accuracy of the model.

Authors: Steve Temme, Pascal Brunet and Parastoo Qarabaqi
Presented at the 131th AES Convention, San Francisco, 2012

Full Paper

Practical Implementation of Perceptual Rub & Buzz Distortion and Experimental Results

In a previous paper [1], we demonstrated how an auditory perceptual model based on an ITU standard can be used to detect audible Rub & Buzz in loudspeakers using a single tone stimulus. In this paper, we discuss a practical implementation using a stepped sine sweep stimulus and present detailed experimental results on loudspeakers including comparison to human listeners and other perceptual methods.

Authors: Steve Temme, Pascal Brunet and Brian Fallon
Presented at the 129th AES Convention, San Francisco, 2010

Full Paper

Practical Measurement of Loudspeaker Distortion Using a Simplified Auditory Perceptual Model

Manufacturing defects in loudspeaker production can often be identified by an increase in Rub & Buzz distortion. This type of distortion is quite noticeable because it contributes an edgy sound to the reproduction and is annoying because it often sounds separate or disembodied from the fundamental signal. The annoyance of Rub & Buzz distortion is tied intimately to human perception of sound and psychoacoustics. To properly implement automated production-line testing of loudspeaker Rub & Buzz defects, one has to model or imitate the hearing process using a sufficiently accurate perceptual model. This paper describes the results of a Rub & Buzz detection system using a simplified perceptual model based on human masking thresholds that yields excellent results.

Authors: Steve Temme, Pascal Brunet and D.B. (Don) Keele
Presented at the 127th AES Convention, New York, 2009

Full Paper

Evolution of Time-Frequency Analysis Methods and their Practical Applications

Time-Frequency analysis has been in use for more than 20 years and many different Time-Frequency distributions have been developed. Four in particular, Short Time Fourier Transform, Cumulative Spectral Decay, Wavelet and Wigner-Ville have gained popularity and firmly established themselves as useful measurement tools. This paper compares these four popular transforms, explains their trade-offs and discusses how to apply them to analyzing audio devices. Practical examples of loudspeaker impulse responses, loose particles, and Rub & Buzz defects are given as well as a demonstration of their application to common problems with digital/analog audio devices such as Bluetooth headsets, MP3 players and VoIP telephones.

Authors: Pascal Brunet, Zachary Rimkunas and Steve Temme
Presented at the 123rd AES Convention, New York, 2007

Full Paper