Registration required

The content you have requested is available only to registered visitors. Please complete the short form below (all fields are required).

  • 简体中文 简体中文 Chinese (Simplified) zh-hans
  • English English English en
Listen, Inc.
  • Products
    • AUDIO MEASUREMENT
    • System Overview
      • What Soundcheck Measures
      • SoundCheck for R&D
      • SoundCheck for Production
      • Innovation
    • SoundCheck Software
      • Making Measurements
      • Algorithm Highlights
      • Integration
      • What’s New?
      • Packages, Modules & Pricing
      • Support Contracts
    • Audio Measurement Interfaces
      • AudioConnect 2 (2 channel))
      • AmpConnect 621 (6/2 channels)
      • AudioConnect 8×8 (8 channel)
      • AudioConnect 6×2 (6/2 channel)
      • AmpConnect ISC (2 channel)
      • 3rd Party Audio Interfaces
    • Audio Testing Hardware
      • Microphones & Amplifiers
        • Measurement Microphones
        • Microphone Power
        • Audio Test Amplifier
      • Specialist Test Interfaces
        • Bluetooth Audio Measurement Interfaces
        • MEMS microphone Interfaces
        • DC Connect
      • Audio Test Accessories
        • Turntables
        • Halo Automotive 6-Mic Array
        • Automotive Audio Test Kit
        • The SCC-1 Ear Coupler
        • Other Measurement Transducers
        • Impedance Box
        • Calibrators
    • NOISE & VIBRATION MEASUREMENT
    • System Overview
    • NoiseCheck Software
    • Noise & Vibration Interfaces
    • Noise and Vibration Testing Hardware
  • Applications
    • Headphones & Headsets
    • Loudspeakers & Microspeakers
    • Microphones
    • Automotive
    • Communications
    • Smart Devices
    • Hearing Aids
    • Audio Electronics
  • Support
    • Get Support
    • User Documentation
    • Knowledgebase
    • Training
    • Sequences & Resources
    • Drivers
    • Register Product
    • Report a Bug / Request a Feature
  • Resources
    • ‘Sound Matters’ Blog: Audio Measurement Expertise
    • Test Sequences
    • Article Reprints
    • Published Papers
    • Technical Notes
    • Measurement Services
    • Useful References
  • News & Events
    • News
    • Events
  • About Listen
    • Listen, Inc.
    • Contact
    • Distributors & Reps
    • Careers
    • Privacy Policy
  • Menu Menu

The Challenges of Testing Voice-Controlled Audio Systems

November 14, 2019/in Papers, Smart Device Papers/by Zarina

Testing voice-controlled audio systems such as smart speakers, hearables, and vehicle infotainment systems is notoriously complex. They have numerous connections from wired to wireless and contain much signal processing, both on the record and the playback side. This means that their characteristics change according to ‘real world’ conditions of the environment that they are used in, such as background noise, playback levels, and room acoustics. Furthermore, their multifunctional nature means that there are many aspects of the device that may need to be tested, ranging from voice recognition to music playback, operation as a hands-free telephone, and in the case of hearables, hearing assistance. Due to their complex non-linear use cases, these devices often need to be tested at different levels and different environmental conditions. This paper focuses on tools and techniques to accurately measure the audio performance of such devices under the many various real-world conditions in which they are used.

 

语音控制的智能设备(例如智能扬声器、听觉设备和车辆信息娱乐系统)非常难以测试。它们具有从有线到无线的多样连接方式,并且在接收端和重放端使用了诸多信号处理技术。这意味着它们的特性会随着使用环境的“现实世界”条件(例如背景噪声、播放级别和室内声学条件)的不同而变化。 此外,它们的多功能特性意味着可能需要测试该设备的许多方面,包括语音识别、音乐播放、作为免提电话或听觉设备或助听器使用时的性能。由于其复杂的非线性使用情况,这些设备通常需要在不同级别和不同环境条件下进行测试。本文重点介绍在各种实际条件下准确测量此类设备的音频性能的工具和技术。

Author: Steve Temme, Listen, Inc.
Presented at ISEAT 2019, Shenzhen, China.

Full Paper – English Version
Full Paper – Chinese Version

 

Paper Preview: The Challenges of Testing Voice-Controlled Audio Systems

Abstract
Smart devices that are voice-controlled such as smart speakers, hearables, and vehicle infotainment systems are notoriously complex to test. They have numerous connections from wired to wireless and contain much signal processing, both on the record and the playback side. This means that their characteristics change according to ‘real world’ conditions of the environment that they are used in, such as background noise, playback levels, and room acoustics. Furthermore, their multifunctional nature means that there are many aspects of the device that may need to be tested, ranging from voice recognition to music playback, operation as a hands-free telephone, and in the case of hearables, hearing assistance. Due to their complex non-linear use cases, these devices often need to be tested at different levels and different environmental conditions. This paper focuses on tools and techniques to accurately measure the audio performance of such devices under the many various real-world conditions in which they are used.

Keywords: hearables, automotive infotainment, smart speakers, smartphones, test

Introduction
Smart Devices such as smart speakers, hearables and automotive infotainment systems have become increasingly challenging to test. They have many possible interfaces ranging from hardwired to wireless (Bluetooth, cloud-based), smartphone, voice (“Hey Siri”, “OK Google”, “Alexa”), and in the case of automotive, even USB memory stick and CarPlay/Android Auto. There is usually much signal processing, both on the record side (e.g. beamforming, background noise filtering, voice activity detection, and on the playback side (e.g. loudness, compression, equalization, and active noise cancellation).
This means that their characteristics change according to ‘real world’ conditions such as the physical environment and background noise. Furthermore, their multifunctional nature means that there are many aspects of the device that may need to be tested, ranging from voice recognition to music playback or even operation as a hands-free telephone, telephone headset or hearing aid. These devices often need to be tested at different levels and in different environmental conditions, for example different physical setups and with/without background noise, different signals etc.

Although, there are currently no standards for testing most smart devices, principles and test configurations are borrowed from many other audio devices and use existing standards such as IEC and BS EN for loudspeakers and headphones , IEEE for headsets, IEEE/TIA/ITU for telephone test, ANSI and IEC for hearing aid standards , and ETSI for background noise. Flexibility of the test system and experience with testing a wide range of acoustic devices is therefore critical to enable a device to be completely characterized.

This paper explains how to implement both basic acoustic tests and more complex real-world tests along with the techniques and standards that may be used. Most of the tests discussed are relevant to all smart devices including smart speakers, hearables and automotive infotainment, but some hearable-specific additional tests are also detailed. Finally, we present a check list of the test-system functionality you should look for when choosing a system to fully characterize a smart speaker or other smart device.

 

Full Paper – English Version
Full Paper – Chinese Version

 

More about testing infotainment systems.

Share
  • Share on Facebook
  • Share on Twitter
  • Share on WhatsApp
  • Share on LinkedIn
  • Share by Mail
https://www.listeninc.com/wp/media/2019/11/ISEAT2019_voice-controlled-audio-test.png 417 300 Zarina https://www.listeninc.com/wp/media/logo.png Zarina2019-11-14 16:43:402025-08-28 12:55:11The Challenges of Testing Voice-Controlled Audio Systems

Audio Measurement Expertise


How Changing Industry Needs are Shaping Audio Test and Measurement

Smarter SoundCheck Sequences with Loops and Logic

Audio Basics: Polarity

How to Measure: Haptics

The Value of Big Data

Audio Basics: Phase Response

view all

Latest News


2026 End Of Year Promotions

Celebrating 30 years: Listening – The Secret Of Our Success

Automotive Test Sequence Including BSR, Max SPL and Frequency Response

Listen Introduces AudioConnect 2, a New Portable Dual-Channel Audio Interface

SoundCheck 21 Brings Enhanced Loose Particle Detection, Multichannel FFT, Metadata, Sequence Protection and more


Audio Measurement Expertise – Delivered Monthly

 

ABOUT LISTEN INC

Listen is a global leader in audio measurement and analysis, with over 30 years’ experience developing innovative test methods and algorithms. SoundCheck, our powerful software-based audio analyzer, pairs with our test interfaces, precision microphones, and accessories to offer a complete solution for testing loudspeakers, headphones, microphones, smart devices, communication systems, hearing aids, automotive audio, and more. Trusted by engineers worldwide, Listen delivers the accuracy and flexibility essential for modern audio testing, from the R&D lab to the production line.

PRODUCTS
APPLICATIONS
SUPPORT
RESOURCES
NEWS & EVENTS
ABOUT

CONNECT

YouTube
LinkedIn
LinkedIn
Share
Instagram

Copyright © Listen, Inc.   |   Privacy Policy   |   AI Disclosure
  • Instagram
  • LinkedIn
  • Youtube
  • Mail
Testing Audio Performance of HearablesPractical Testing of Voice Controlled Devices
Scroll to top