A New Method for Transient Distortion Detection

/in /by

Transient distortion, or ‘loose particle’ measurement, is an important loudspeaker production line quality control metric that identifies and facilitates troubleshooting of manufacturing issues.

This paper introduces a new enhanced loose particle measurement technique that discriminates more accurately and reliably than current methods. This new method introduces ‘prominence’ after envelope detection, a new metric for audio measurements, that effectively isolates transient distortion in the presence of periodic distortion. This technique also offers the unique ability to listen to the isolated transient distortion waveform which makes it easier to set limits based on audibility and has widespread applications.

Authors: Steve Temme, Rahul Shakya and Jayant Datta, Listen, Inc.
Presented at 155th AES Conference (October 2023) New York, NY

Full Paper

 

Paper Introduction

Transient distortion, or ‘loose particle’ measurement, is a valuable quality control metric because it identifies non-periodic distortion, for example, rattling parts, separately from periodic distortion such as rubbing or buzzing parts. This facilitates troubleshooting of manufacturing issues. This paper introduces a new transient distortion measurement technique that is more accurate and reliable than current methods. In addition to improved performance, this new algorithm also aids understanding of the correlation between measurement results and audibility, since it is possible to isolate and listen to just the transient distortion artifacts. Although this analysis method was developed for measuring loose particles in loudspeaker drivers, it is also valuable for measuring rattling parts such as buttons, fasteners, and loose wires on various audio devices, and measuring impulsive distortion or Buzz, Squeak and Rattle (BSR) in automotive audio applications [1].

What is Transient Distortion? Why does it matter?

Transient distortion is caused by random clicking, popping, and other noises in the time domain. In a speaker or headphone driver, this might be caused by foreign particles such as glue or magnet fragments trapped in the gap behind the diaphragm or dust cap. In a device such as a smart speaker, transient distortion might come from a loose volume control button on the device that rattles when sound is played. In an automotive application, it could be characterized as buzz, squeak and rattle from loose wires, screws or fasteners in a car door that the loudspeaker is mounted in. In all cases, the sound is undesirable, so devices that exhibit such faults should be identified and rejected.

In the recorded time waveform, transient distortion faults appear as impulsive noises added on the stimulus wave. These impulses are not related to the frequency of the stimulus, but rather to the vibration caused by the displacement amplitude of the diaphragm. The transient distortion is more frequent and significant when the speaker is driven near or below its resonant frequency, where the displacement of the diaphragm is the greatest.

Although the sound – a random clicking, buzzing or popping noise – can sometimes sound similar to higher order harmonic distortion (Rub & Buzz), such defects are not clearly reflected in the frequency spectrum of the waveform. Figure 1 shows a waveform with transient distortion, and the corresponding frequency spectrum. The vertical black line represents the stimulus frequency and the orange broadband noise spectrum indicates the transient distortion. Transient distortion is best identified at the time the transients occur, unlike Rub & Buzz distortion which is best identified by the frequency at which it occurs [3].

The entire paper also covers:

Prior Measurement Methods

The new algorithm – comparison and results

Conclusions

Download paper

In addition to this paper, please also check out the Enhanced Loose Particles Webpage and our detailed video explanation of how this algorithm works.

Enhanced Perceptual Rub & Buzz Measurement for Testing Automotive Loudspeakers

/in /by

Loudspeaker Rub & Buzz faults are a problem for automotive manufacturers as they sound harsh and immediately give the perception of poor quality. There are two places such faults can occur – during speaker manufacturing and installation of the speaker in the car. A buzzing loudspeaker in a car is disappointing to a customer and is costly to replace. It is also challenging for a service center to determine exactly where the buzzing is coming from and whether it is caused by a faulty loudspeaker or bad installation. Perceptual distortion measurements are often considered the holy grail of end-of-line testing because rejecting speakers with only audible faults increases yield. Although such measurements have been around since 2011, production line adoption has been slow because until now, sensitivity to background noise has made limit-setting challenging. In this paper, a new algorithm is introduced that uses advanced technology to reduce the impact of background noise on the measurement and offer more repeatable results. This facilitates limit setting on the production line and makes it a truly viable production line metric for increasing yield. This same metric may also be used for end-of-line automotive quality control tests. Results from various algorithms will be shown, and their correlation to subjective and other non-perceptual distortion metrics explained.

Author: Steve Temme, Listen, Inc.
Presented at 2022 AES Automotive Conference, Dearborn, MI

Full Paper

 

Introduction

The automotive industry’s stringent quality expectations make end-of-line quality testing on automotive speakers and drivers absolutely critical. End-of-line tests typically measure a range of parameters including frequency response, THD, and polarity. Manufacturing-introduced defects such as Rub & Buzz and Loose Particles are also measured. Reliable, automated testing has been available for decades now, and most large manufacturers rely on these software-based systems for identification and rejection of defective products. While these tests do an excellent job of identifying defective units, there is always a certain level of false rejection where units with some distortion fail even though it is completely inaudible to the human ear. From a manufacturing perspective, higher yields and therefore greater profitability is always desirable.

Perceptual Distortion Measurements

This has driven the development of perceptual distortion measurements – automated measurements that replicate the human hearing to detect only audible distortion defects. Such metrics increase production line yield by passing products with inaudible distortion, as the product will still sound exactly as the manufacturer intended. Perceptual methods are very simple to configure for production line use. Since they return a result in Phons, an absolute measurement that can be easily correlated to the listener’s threshold of hearing, the operator can set a fixed limit across the board, regardless of product. Naturally, the price point and quality expectations for the product may influence the level of distortion that is deemed acceptable.

Perceptual Distortion Algorithms

Our algorithm, introduced in 2011, was the first commercial perceptual distortion metric, although in the past couple of years, other test system manufacturers have also started to offer perceptual distortion tests. It offers excellent correlation with human hearing and performs well in laboratory tests. However, like the human ear, repeatability decreases in the presence of background noise. This is not a failure of the algorithm as such, but an indication that the algorithm performs just like a human listener; when background noise is high, audible distortion is masked. This limitation restricts the value of such algorithms on the production line, as with today’s high-volume manufacturing, there is only time for one fast test sweep. If this sweep gets a different result under changing background noise conditions, limit setting becomes challenging, and repeatability and reliability is decreased. Similar algorithms from other test system manufacturers also suffer from the same problems.

New Perceptual Distortion Algorithm Development

This paper details efforts to create an algorithm that hears like a human in quiet conditions, e.g. in a living room or passenger automotive cabin, under the less-than-perfect conditions of a manufacturing environment where considerable and varying background noise may be present. In other words, a perceptual model that is more independent and reliable than the human ear when it comes to noisy environments. The resulting new algorithm overcomes these limitations to offer repeatable end-of-line test results, even in noisy environments. It incorporates noise reduction techniques and enhanced perceptual filters to overcome the reliability and high frequency masking issues of earlier versions. In short, the algorithm offers the performance of an ‘enhanced’ human ear – it detects distortion like an ear in a quiet environment, even when there is background noise. This makes it a viable solution for production line use.

In this paper we explain how the algorithm works, demonstrate how the results compare with earlier perceptual algorithms and show its correlation with human hearing and conventional distortion algorithms. We also compare its performance in the presence of background noise to other perceptual algorithms by adding recorded factory background noise to the signal before passing it through the algorithms.

Read More

More about Listen’s enhanced Perceptual Rub & Buzz algorithm

More about in-car measurement of  impulsive distortion / Buzz, Squeak and Rattle.

The Challenges of Testing Voice-Controlled Audio Systems

/in /by

Testing voice-controlled audio systems such as smart speakers, hearables, and vehicle infotainment systems is notoriously complex. They have numerous connections from wired to wireless and contain much signal processing, both on the record and the playback side. This means that their characteristics change according to ‘real world’ conditions of the environment that they are used in, such as background noise, playback levels, and room acoustics. Furthermore, their multifunctional nature means that there are many aspects of the device that may need to be tested, ranging from voice recognition to music playback, operation as a hands-free telephone, and in the case of hearables, hearing assistance. Due to their complex non-linear use cases, these devices often need to be tested at different levels and different environmental conditions. This paper focuses on tools and techniques to accurately measure the audio performance of such devices under the many various real-world conditions in which they are used.

 

语音控制的智能设备（例如智能扬声器、听觉设备和车辆信息娱乐系统）非常难以测试。它们具有从有线到无线的多样连接方式，并且在接收端和重放端使用了诸多信号处理技术。这意味着它们的特性会随着使用环境的“现实世界”条件（例如背景噪声、播放级别和室内声学条件）的不同而变化。 此外，它们的多功能特性意味着可能需要测试该设备的许多方面，包括语音识别、音乐播放、作为免提电话或听觉设备或助听器使用时的性能。由于其复杂的非线性使用情况，这些设备通常需要在不同级别和不同环境条件下进行测试。本文重点介绍在各种实际条件下准确测量此类设备的音频性能的工具和技术。

Author: Steve Temme, Listen, Inc.
Presented at ISEAT 2019, Shenzhen, China.

Full Paper – English Version
Full Paper – Chinese Version

 

Paper Preview: The Challenges of Testing Voice-Controlled Audio Systems

Abstract
Smart devices that are voice-controlled such as smart speakers, hearables, and vehicle infotainment systems are notoriously complex to test. They have numerous connections from wired to wireless and contain much signal processing, both on the record and the playback side. This means that their characteristics change according to ‘real world’ conditions of the environment that they are used in, such as background noise, playback levels, and room acoustics. Furthermore, their multifunctional nature means that there are many aspects of the device that may need to be tested, ranging from voice recognition to music playback, operation as a hands-free telephone, and in the case of hearables, hearing assistance. Due to their complex non-linear use cases, these devices often need to be tested at different levels and different environmental conditions. This paper focuses on tools and techniques to accurately measure the audio performance of such devices under the many various real-world conditions in which they are used.

Keywords: hearables, automotive infotainment, smart speakers, smartphones, test

Introduction
Smart Devices such as smart speakers, hearables and automotive infotainment systems have become increasingly challenging to test. They have many possible interfaces ranging from hardwired to wireless (Bluetooth, cloud-based), smartphone, voice (“Hey Siri”, “OK Google”, “Alexa”), and in the case of automotive, even USB memory stick and CarPlay/Android Auto. There is usually much signal processing, both on the record side (e.g. beamforming, background noise filtering, voice activity detection, and on the playback side (e.g. loudness, compression, equalization, and active noise cancellation).
This means that their characteristics change according to ‘real world’ conditions such as the physical environment and background noise. Furthermore, their multifunctional nature means that there are many aspects of the device that may need to be tested, ranging from voice recognition to music playback or even operation as a hands-free telephone, telephone headset or hearing aid. These devices often need to be tested at different levels and in different environmental conditions, for example different physical setups and with/without background noise, different signals etc.

Although, there are currently no standards for testing most smart devices, principles and test configurations are borrowed from many other audio devices and use existing standards such as IEC and BS EN for loudspeakers and headphones , IEEE for headsets, IEEE/TIA/ITU for telephone test, ANSI and IEC for hearing aid standards , and ETSI for background noise. Flexibility of the test system and experience with testing a wide range of acoustic devices is therefore critical to enable a device to be completely characterized.

This paper explains how to implement both basic acoustic tests and more complex real-world tests along with the techniques and standards that may be used. Most of the tests discussed are relevant to all smart devices including smart speakers, hearables and automotive infotainment, but some hearable-specific additional tests are also detailed. Finally, we present a check list of the test-system functionality you should look for when choosing a system to fully characterize a smart speaker or other smart device.

 

Full Paper – English Version
Full Paper – Chinese Version

 

More about testing infotainment systems.

Testing Audio Performance of Hearables

/in /by
Picture of AES paper on testing hearables

Testing Hearables AES Paper

Testing hearables, or smart headphones, is challenging. They have various interfaces ranging from hardwired to wireless and often contain signal processing on both the record and playback side. This means that their characteristics change according to ‘real world’ conditions such as their physical environment and background noises. Furthermore, their multifunctional nature means that there are many aspects of the device that may need to be tested, ranging from voice recognition to music playback or even operation as a telephone headset or hearing aid. In this AES paper, we discuss how to implement basic acoustic tests as well as the more complex real-world tests, techniques, standards, and equipment that are necessary.

Authors: Steve Temme, Listen, Inc.
Presented at AES Headphone Conference 2019, San Francisco, CA.

Full Paper
Poster Presentation

 

Abstract & introduction for “Testing Audio Performance of Hearables”

Abstract for “Testing Hearables”
Smart headphones or “hearables” are designed not only to playback music but to enhance communications in the presence of background noise and in some cases, even compensate for hearing loss. They may also provide voice recognition, medical monitoring, fitness tracking, real-time translation and even augmented reality (AR). They contain complex signal processing and their characteristics change according to their smartphone application and ‘real world’ conditions of their actual environment, including background noises and playback levels. This paper
focuses on how to measure their audio performance under the many various real-world conditions they are used in.

Introduction for “Testing Hearables”
Hearables are notoriously challenging to test. They have various interfaces ranging from hardwired to wireless (e.g. Bluetooth) and may contain much signal processing, both on the record side (e.g. beamforming, background noise filtering, voice activity detection, and on the playback side (e.g. loudness, compression, equalization, and active noise cancellation). This means that their characteristics change according to ‘real world’ conditions such as their physical environment and background noises. Some even have wake word detection, e.g. ‘Hey Siri’. Furthermore, their multifunctional nature means that there are many aspects of the device that may need to be tested, ranging from voice recognition to music playback or even operation as a telephone headset or hearing aid. Due to their complex non-linear use cases, these devices often need to be tested at different levels and in different environmental conditions, for example with background noise, different signals etc. Although, there are currently no standards for testing smart devices such as hearables, we can borrow principles and test configurations from many other audio devices and use existing standards such as; IEC for headphones [1], IEEE for headsets [2], ETSI for background noise [3], TIA/ITU for telephone test [4] and ANSI for hearing aids standards [5].

Flexibility of the test system and experience with testing a wide range of acoustic devices is critical to enable a device to be completely characterized. This paper discusses how to implement basic acoustic tests and some of the more complex real-world tests along with the techniques and standards that may be used. Test system requirements for measuring voice
enabled hearables will also be discussed.

Full Paper

More about Headphone & Hearables Testing

Testing Voice-Controlled & Smartphone Integrated Infotainment Systems

/in , /by

A tutorial and accompanying paper on testing infotainment systems that was presented at the AES Automotive Conference, Sept 11-13, 2019, Neuburg an der Donau, Germany.

Voice-controlled and smartphone integrated vehicle infotainment systems are notoriously complex to test. They have numerous connections from wired to wireless and contain much signal processing, both on the record and on the playback side. This means that their characteristics change according to ‘real world’ conditions of the vehicle’s environment, including cabin acoustics and background noises from road, wind and motors. Furthermore, their multifunctional nature means that there are many aspects of the device that may need to be tested, ranging from voice recognition to music playback and operation as a hands-free telephone. Due to their complex non-linear use cases, these devices often need to be tested at different levels and different environmental conditions.

This tutorial offers practical hands-on advice on how to test such devices, including test configurations, what to measure, the challenges of making open-loop measurements, and how to select a test system.

Download presentation (slides)

Download accompanying paper

 

Paper Introduction

Audio Tests for Infotainment Systems

Infotainment systems have become increasingly challenging to test. They have many possible interfaces; hard-wired or auxiliary input, radio, CD, memory card, hard drive, USB, Bluetooth, smartphone (including Apple CarPlay and Android Auto) and even voice. They contain much signal processing, both on the record side (e.g. beamforming, background noise filtering, voice activity detection, and on the playback side (e.g.loudness, compression, equalization, and active noise cancellation). Some even have wake word detection, e.g. “Hey Siri”, “OK Google”, and “Alexa”. Due to their complex non-linear use cases, these devices often need to be tested at different levels and in different environmental conditions, for example with different background noises and different test signals. To further complicate matters, the test signal may need to be in the cloud to enable playback for testing voice recognition systems. Each manufacturer’s ecosystem is different in how it plays and records.

Smartphone integrated infotainment systems usually require an internet connection with voice services in order to process commands. On the playback side, some enable you to upload your own recordings such as iTunes (although bear in mind that these will probably be compressed). Others require them to be on a media streaming platform such as Spotify. For microphone testing, some systems such as Alexa allow access to recordings made; others do not for security and privacy reasons, which makes microphone testing challenging. Although the actual physical testing setup is very similar from vehicle to vehicle, for each it is necessary to understand how to wirelessly route the signal. Furthermore, each device needs activating with a different wake word, needs different delay compensation, and records for a different amount of time after it hears the wake word. This needs figuring out (largely by trial and error) for each infotainment system that you need to test.

Infotainment System Testing Standards

Although, there are currently no standards for testing infotainment systems with smartphone integration, principles and test configurations can be borrowed from many other audio devices and use existing standards such as IEC for loudspeakers, IEEE/TIA/ITU for speakerphones, and ETSI for background noise. Flexibility of the test system and experience with testing a wide range of acoustic devices is critical to enable a device to be completely characterized. This paper focuses on how to implement basic acoustic tests and some of the more complex real-world tests along with the techniques and standards that may be used.

The rest of the paper covers:

Basic Acoustic Tests
Advanced or ‘Real World’ Testing of Infotainment Systems
Speech Recognition
Background Noise
Voice Quality
Measurement System Requirements

Full Paper

 

More about testing infotainment systems.

A New THD+N Algorithm for Measuring Today’s High Resolution Audio Systems

/in /by

In this paper, a mathematical definition of Total Harmonic Distortion + Noise suitable for testing high-resolution digital audio systems is presented. This formal definition of the “distortion analyzer” mentioned in AES17 defines THD+N as the RMS error of fitting a sinusoid to a noisy and distorted sequence of measurements. We present the key theoretical result that under realistic conditions a modern THD+N analyzer is well-described by a Normal probability distribution with a simple relationship between relative error and analysis dwell time. These findings are illustrated by comparing the output of a commercial distortion analyzer to our proposed method using Monte Carlo simulations of noisy signal channels. We will demonstrate that the bias of a well-designed distortion analyzer is negligible.

Authors: Alfred B. Roney The Mathworks, Inc. (formerly Listen, Inc.) Steve Temme, Listen, Inc.
Presented at AES 2018, New York, NY.

Full Paper

Evaluation of audio test methods and measurements for end-of-line loudspeaker quality control

/in /by

In order to minimize costly warranty repairs, loudspeaker OEMS impose tight specifications and a “total quality” requirement on their part suppliers. At the same time, they also require low prices. This makes it important for driver manufacturers and contract manufacturers to work with their OEM customers to define reasonable specifications and tolerances. They must understand both how the loudspeaker OEMS are testing as part of their incoming QC and also how to implement their own end-of-line measurements to ensure correlation between the two.

Authors: Steve Temme, Listen, Inc. and Viktor Dobos, Harman/Becker Automotive Systems Kft.
Presented at ISEAT 2017, Shenzhen, China

Full Paper

Challenges of IoT Smart Speaker Testing

/in /by

Quantitatively measuring the audio characteristics of IoT (Internet of Things) smart speakers presents several novel challenges. We discuss overcoming the practical challenges of testing such devices and demonstrate how to measure frequency response, distortion, and other common audio characteristics. In order to make these measurements, several measurement techniques and algorithms are presented that allow us to move past the practical difficulties presented by this class of emerging audio devices. We discuss test equipment requirements, selection of test signals and especially overcoming the challenges around injecting and extracting test signals from the device.

Authors: Glenn Hess (Indy Acoustic Research) and Daniel Knighten (Listen, Inc.)
Presented at the 143rd AES Conference, New York 2017

Full Paper

Evaluation of Audio Test Methods and Measurements for End-of-Line Automotive Loudspeaker Quality Control

/in /by

In order to minimize costly warranty repairs, automotive manufacturers impose tight specifications and a “total quality” requirement on their part suppliers. At the same time, they also require low prices. This makes it important for automotive manufacturers to work with automotive loudspeaker suppliers to define reasonable specifications and tolerances, and to understand both how the loudspeaker manufacturers are testing and also how to implement their own measurements for incoming QC purposes.

Specifying and testing automotive loudspeakers can be tricky since loudspeakers are inherently nonlinear, time variant and affected by their working conditions & environment which can be change dramatically and rapidly in a vehicle. This paper examines the loudspeaker characteristics that can be measured, and discusses common pitfalls and how to avoid them on a loudspeaker production line. Several different audio test methods and measurements for end-of-the-line automotive speaker quality control are evaluated, and the most relevant ones identified. Speed, statistics, and full traceability are also discussed.

Authors: Steve Temme, Listen, Inc. and Viktor Dobos, Harman/Becker Automotive Systems Kft.
Presented at the 142nd AES Convention, Berlin, Germany

Full Paper

In-Vehicle Audio System Distortion Audibility versus Level and Its Impact on Perceived Sound Quality

/in /by

As in-vehicle audio system output level increases, so too does audio distortion. At what level is distortion audible and how is sound quality perceived as level increases? Binaural recordings of musical excerpts played through the in-vehicle audio system at various volume levels were made in the driver’s position. These were adjusted to equal loudness and played through a low distortion reference headphone. Listeners ranked both distortion audibility and perceived sound quality. The distortion at each volume level was also measured objectively using a commercial audio test system. The correlation between perceived sound quality and objective distortion measurements is discussed.

Authors: Steve Temme, Listen, Inc. and Patrick Dennis, Nissan Technical Center North America, Inc.,
Presented at the 141st AES Convention, Los Angeles, CA 2015

Full Paper