Testing Voice-Controlled & Smartphone Integrated Infotainment Systems

/in , /by

A tutorial and accompanying paper on testing infotainment systems that was presented at the AES Automotive Conference, Sept 11-13, 2019, Neuburg an der Donau, Germany.

Voice-controlled and smartphone integrated vehicle infotainment systems are notoriously complex to test. They have numerous connections from wired to wireless and contain much signal processing, both on the record and on the playback side. This means that their characteristics change according to ‘real world’ conditions of the vehicle’s environment, including cabin acoustics and background noises from road, wind and motors. Furthermore, their multifunctional nature means that there are many aspects of the device that may need to be tested, ranging from voice recognition to music playback and operation as a hands-free telephone. Due to their complex non-linear use cases, these devices often need to be tested at different levels and different environmental conditions.

This tutorial offers practical hands-on advice on how to test such devices, including test configurations, what to measure, the challenges of making open-loop measurements, and how to select a test system.

Download presentation (slides)

Download accompanying paper

 

Paper Introduction

Audio Tests for Infotainment Systems

Infotainment systems have become increasingly challenging to test. They have many possible interfaces; hard-wired or auxiliary input, radio, CD, memory card, hard drive, USB, Bluetooth, smartphone (including Apple CarPlay and Android Auto) and even voice. They contain much signal processing, both on the record side (e.g. beamforming, background noise filtering, voice activity detection, and on the playback side (e.g.loudness, compression, equalization, and active noise cancellation). Some even have wake word detection, e.g. “Hey Siri”, “OK Google”, and “Alexa”. Due to their complex non-linear use cases, these devices often need to be tested at different levels and in different environmental conditions, for example with different background noises and different test signals. To further complicate matters, the test signal may need to be in the cloud to enable playback for testing voice recognition systems. Each manufacturer’s ecosystem is different in how it plays and records.

Smartphone integrated infotainment systems usually require an internet connection with voice services in order to process commands. On the playback side, some enable you to upload your own recordings such as iTunes (although bear in mind that these will probably be compressed). Others require them to be on a media streaming platform such as Spotify. For microphone testing, some systems such as Alexa allow access to recordings made; others do not for security and privacy reasons, which makes microphone testing challenging. Although the actual physical testing setup is very similar from vehicle to vehicle, for each it is necessary to understand how to wirelessly route the signal. Furthermore, each device needs activating with a different wake word, needs different delay compensation, and records for a different amount of time after it hears the wake word. This needs figuring out (largely by trial and error) for each infotainment system that you need to test.

Infotainment System Testing Standards

Although, there are currently no standards for testing infotainment systems with smartphone integration, principles and test configurations can be borrowed from many other audio devices and use existing standards such as IEC for loudspeakers, IEEE/TIA/ITU for speakerphones, and ETSI for background noise. Flexibility of the test system and experience with testing a wide range of acoustic devices is critical to enable a device to be completely characterized. This paper focuses on how to implement basic acoustic tests and some of the more complex real-world tests along with the techniques and standards that may be used.

The rest of the paper covers:

Basic Acoustic Tests
Advanced or ‘Real World’ Testing of Infotainment Systems
Speech Recognition
Background Noise
Voice Quality
Measurement System Requirements

Full Paper

 

More about testing infotainment systems.

Harmonic Distortion Measurement: The effects of sampling rate and stimulus frequency on the measured harmonic frequency (including THD and Rub & Buzz)

/in /by

Author: Steve Temme

Harmonic distortion measurements can be confusing to interpret, and it’s important to understand the data that you are looking at, particularly the relationship between the stimulus frequency and the measured harmonic frequency. This brief guide explains the relationships between sampling rate, stimulus frequency and measured harmonics for both normalized and conventional harmonic distortion measurements.

More

Practical Impedance Measurement

/in /by

Author: Steve Temme
Loudspeaker impedance measurements are made for many reasons. In the R&D lab, these range from the simple task of identifying a speaker’s resonant frequency to more complex functions such as calculating the speaker’s Thiele-Small parameters. On the production line, impedance measurement is a key quality control parameter that verifies that the speaker’s motor properties are correct, that the magnet is charged correctly, the voice coil number of turns is correct and that the moving mass (cone and voice coil) is within specification.
There are two basic methods of making impedance measurements on loudspeakers, micro-speakers and headphones using sound card and software based systems. These are basic single channel measurements, and more complex, but more accurate, dual channel methods. Both methods are implemented in SoundCheck, and with some additional hardware these tests are simple to carry out.

Full Article

Audio Test Stimuli – An Overview

/in /by
Stweep Stepped Sine Wave Stimulus

Stweep Stepped Sine Wave Stimulus

Author: Steve Temme

Most audio analysis systems offer a wide range of stimulus option, and selecting the right one can be daunting. This overview explains the different stimulus types, what the waveforms look and sound like (click the speaker icon to hear the sound), the advantages and disadvantages of each and when you would be likely to select each one.

More

Headphone Testing – a comprehensive overview

/in /by

Author: Steve Temme and Brian Fallon

With the headphone market growing towards $10 billion worldwide, and products across the price spectrum from under a dollar up to thousands, there are many and diverse quality expectations and test requirements. Many audio engineers have moved across from the shrinking loudspeaker industry to the burgeoning headphone marketplace. Although many of the characteristics that make for a good in-room listening experience with a loudspeaker – good frequency response, low distortion, no Rub & Buzz or loose particles, etc. – also apply to headphones, and many of the same test principles apply, there are some significant differences and additional issues associated with headphone measurement that need to be taken into account. These include couplers and associated correction curves, acoustic seal, fixturing and additional tests such as L/R tracking. In this paper we outline the issues that are common to testing all types of headphones as well as those specific to particular types of headphones such as Bluetooth and USB headphone testing, noise-cancelling headphones, and Max SPL measurements to prevent hearing loss.

Full Article

ESCORT: The Listen Guide to Audio Troubleshooting

/in /by

Author: Steve Temme

Audio test troubleshooting comes down to 5 things: Environment, Setup, Calibration, Organization, Range and Technique. In this article we discuss each of these areas of potential problems and provide suggestions for troubleshooting any audio testing setup, regardless of application. We also provide general hints and tips to improve your audio testing configuration.
Full Article

The Challenges of MP3 Player Testing

/in /by

Author: Steve Temme

MP3 players have been the ‘must-have’ electronic gadget for the past few years. Over 10 million players were sold in 2005, and this is predicted to rise to more than double this by 2010. But how can manufacturers carry out QA tests on the production line, ensure excellent sound quality and demonstrate their compliance with Sound Pressure Level regulations? MP3 player testing is challenging as it combines traditional audio analysis techniques with some characteristics unique to MP3 players. Here, we examine the equipment and techniques that MP3 player manufacturers can use to test the sound quality of their products, and advise how to overcome some of the challenges inherent to measuring MP3 players.
Full Article

HarmonicTrak Algorithm for Fast and Accurate Swept Sine Measurement

/in /by

Author: Steve Temme

The sine generator in SoundCheck®, operates on the same principles as a traditional swept sine measurement system consisting of a sine generator, voltmeter, tracking filter, and level recorder. The main difference is that it is implemented in the software as a virtual instrument. The SoundCheck sine generator sweeps in discrete steps; each frequency step contains numerous cycles that are synchronously averaged to minimize the effect of background noise. Transducer settling (transient ringing) is also minimized by discarding the first few cycles at each new frequency step and providing a phase continuous transition between frequencies. This provides higher frequency, amplitude, and phase accuracy, as well as excellent noise suppression.  Instead of using a tracking filter that can only measure one harmonic at a time, SoundCheck uses a proprietary FFT-based algorithm, called HarmonicTrakTM, which is equivalent to a parallel bank of individual tracking filters that measure all selected harmonics simultaneously. This parallel analysis technique saves considerable measurement time over the traditional serial analysis method.
Full Article

Are You Shipping Defective Loudspeakers to your Customers?

/in /by

Author: Steve Temme

Loudspeaker distortion is undesirable. The type and level of distortion, however, can greatly influence the perceived annoyance. In addition, identifying the type of distortion can also help pinpoint the mechanism or mechanisms in the loudspeaker that are causing the distortion. “Rub & Buzz” is a good example of a particularly annoying type of distortion that is very difficult to measure. Pinpointing the cause of the problem from the measurement is an even more difficult task. Understanding why this type of distortion is so annoying and how to measure it is critical in being able to properly test loudspeakers on the production line.
Full Article

Evaluation of Loudspeaker Performance at Low Frequencies

/in /by

Authors: Steve Temme and Christopher J Struck

Evaluation of loudspeaker performance at low frequencies is complicated by long wavelengths, room interaction and cabinet/baffle diffraction. Since low frequency measurements have traditionally required large, impractical testing environments, different techniques have been developed in an attempt to overcome this requirement. Anechoic chambers, outdoor measurements, half-space measurements, ground plane measurements, cepstral liftering, parametric modelling and near field techniques are compared with respect to accuracy, speed, bandwidth and practical implementation.
Full Article