Posts

Practical Testing of Voice Controlled Devices

/in /by

Author: Steve Temme.  Reprinted from the Jan 2020 issue of AudioXpress.

This article discusses tools and techniques that are available to accurately measure the audio performance of voice-controlled and connected devices under the many various real-world conditions they may be used. It covers basic acoustic measurements such as frequency and distortion response, which have always been carried out on conventional wired systems, and the more complex real-world tests that apply specifically to voice-activated devices, along withthe techniques and standards that may be used.
Full Article

 

 

 

 

 

The Challenges of Testing Voice-Controlled Audio Systems

/in /by

Smart devices that are voice-controlled such as smart speakers, hearables, and vehicle infotainment systems are notoriously complex to test. They have numerous connections from wired to wireless and contain much signal processing, both on the record and the playback side. This means that their characteristics change according to ‘real world’ conditions of the environment that they are used in, such as background noise, playback levels, and room acoustics. Furthermore, their multifunctional nature means that there are many aspects of the device that may need to be tested, ranging from voice recognition to music playback, operation as a hands-free telephone, and in the case of hearables, hearing assistance. Due to their complex non-linear use cases, these devices often need to be tested at different levels and different environmental conditions. This paper focuses on tools and techniques to accurately measure the audio performance of such devices under the many various real-world conditions in which they are used.

 

语音控制的智能设备（例如智能扬声器、听觉设备和车辆信息娱乐系统）非常难以测试。它们具有从有线到无线的多样连接方式，并且在接收端和重放端使用了诸多信号处理技术。这意味着它们的特性会随着使用环境的“现实世界”条件（例如背景噪声、播放级别和室内声学条件）的不同而变化。 此外，它们的多功能特性意味着可能需要测试该设备的许多方面，包括语音识别、音乐播放、作为免提电话或听觉设备或助听器使用时的性能。由于其复杂的非线性使用情况，这些设备通常需要在不同级别和不同环境条件下进行测试。本文重点介绍在各种实际条件下准确测量此类设备的音频性能的工具和技术。

Author: Steve Temme, Listen, Inc.
Presented at ISEAT 2019, Shenzhen, China.

Full Paper – English Version
Full Paper – Chinese Version

Testing Audio Performance of Hearables

/in /by

Smart headphones or “hearables” are designed not only to playback music but to enhance communications in the
presence of background noise and in some cases, even compensate for hearing loss. They may also provide voice
recognition, medical monitoring, fitness tracking, real-time translation and even augmented reality (AR). They
contain complex signal processing and their characteristics change according to their smartphone application and
‘real world’ conditions of their actual environment, including background noises and playback levels. This paper
focuses on how to measure their audio performance under the many various real-world conditions they are used
in.

Authors: Steve Temme, Listen, Inc.
Presented at AES Headphone Conference 2019, San Francisco, CA.

Full Paper
Poster Presentation

Seminars in China: Smart Speakers and Automotive Audio

/in , /by

The Challenges of Testing Speech Controlled Audio Systems

Your devices got smarter. Did your test system?

Learn how to implement open loop tests for both playback and recording in a range of devices including smart speakers, automotive audio, robots, IoT devices and more in this practical seminar.

Open loop testing (testing devices where inputs and outputs are independent) enables many types of smart devices and their components to be tested in various formats and situations including:

  • Smart speakers, smart watches and other smart devices
  • Microphone arrays
  • Speech recognition systems with microphones
  • In-vehicle audio systems
  • Audio devices/systems with no physical inputs or outputs
  • Testing in noisy environments

We explain how to measure the same parameters as traditional loudspeakers, discussing such challenges as injecting and extracting response signals, time delays, and triggering the system using voice commands. We also demonstrate how to measure microphone array performance, speakerphone performance and more. Course content includes:

  • Stimulating and capturing responses from a device where you don’t have direct access to the microphone or speaker (open loop testing).
  • Delays, asynchronous stimulus / acquisition, and working with the ‘cloud’
  • Testing with real world signals such as speech and music, and how to analyze results from these test stimuli
  • Voice Recognition – key word spotting, wake word testing, directionality, and the effect of background noise on voice recognition performance
  • Telephony – testing smart speakers for hands-free calling

Speakers /主讲人: Steve Temme, Listen, Inc. and Peter Wulf Andersen, GRAS.

Date & Locations /时间与地点:

2019年1月15日(Tuesday) – Taipei台北
2019年1月16日(Wednesday) – Shenzhen深圳
2019年1月18日(Friday) – Suzhou苏州

More information and registration

Seminars in UK and Germany: The Challenges of Testing Speech Controlled Audio Systems

/in , /by

smart_speakersYour devices got smarter. Did your test system?

Learn how to implement open loop tests for both playback and recording in a range of devices including smart speakers, automotive audio, robots, IoT devices and more in this practical half-day seminar.

Open loop testing (testing devices where inputs and outputs are independent) enables many types of smart devices and their components to be tested in various formats and situations including:

  • Smart speakers, smart watches and other smart devices
  • Microphone arrays
  • Speech recognition systems with microphones
  • In-vehicle audio systems
  • Audio devices/systems with no physical inputs or outputs
  • Testing in noisy environments

We explain how to measure the same parameters as traditional loudspeakers, discussing such challenges as injecting and extracting response signals, time delays, and triggering the system using voice commands. We also demonstrate how to measure microphone array performance, speakerphone performance and more. Course content includes:

  • Stimulating and capturing responses from a device where you don’t have direct access to the microphone or speaker (open loop testing).
  • Delays, asynchronous stimulus / acquisition, and working with the ‘cloud’
  • Testing with real world signals such as speech and music, and how to analyze results from these test stimuli
  • Voice Recognition – key word spotting, wake word testing, directionality, and the effect of background noise on voice recognition performance
  • Telephony – testing smart speakers for hands-free calling

There is no charge for this one-day seminar and lunch will be provided. Space is limited, so please RSVP today.

 

Locations and Dates:

PrintUK: Tuesday Sept 25th, Sharnbrook Hotel, Bedford, England. The Sharnbrook Hotel, Park Lane (off A6), Sharnbrook Bedfordshire MK44 1LX. Note: If you need to stay here overnight, tell them you are with the ACSoft group to take advantage of a discounted rate.

 

PrintGermany: Thursday Sept 27th, NH München Ost Conference Center, Einsteinring 20, 85609 Aschheim, Germany.

 

 

Full Agenda (9am-4.00pm)

  • Introduction to Open Loop Testing
    • What is it?
    • What Kinds of Analysis Can Be Done?
      • Classic Acoustic Testing: Frequency Response and Distortion
        • Stimulus Signals
        • Distortion Methods: THD, THD+N, Rub & Buzz (Squeak & Rattle), Perceptual Rub & Buzz, Non-Coherent Distortion
    • Mixing Digital and Analog Units
    • Capturing Signals Asynchronous to the Stimulus
    • Dealing with Triggering, playback and recording Delays, digital clock Resampling, and corresponding Frequency Shift
  • Open Loop Applications for Playback
    • Smart Devices
      • Frequency Response and Distortion
      • Directionality
    • Automotive
      • 6 Mic Tree for Audio Tuning
      • Distortion methods
      • HATS for Impulse Response Testing @ seated position
  • Networking Lunch
  • Open Loop Applications for Recording
    • Smart Devices
      • Frequency Response and Distortion
      • Voice Recognition
      • Directionality
    • Automotive
      • Frequency Response and Distortion (general and to ITU P.11xx)
      • SNR placement study
      • Impulse Response function from HATS mouth -> microphone
      • Directionality
  • Handsfree Communications
    • Terminology for Handsfree Communications
    • Introduction to Telephony Metrics
  • GRAS Product Overview
    • Choosing the right microphone for your application (far field vs near field vs unconventional environments)
    • When to use a Mouth Simulator vs Head and Torso Simulator for testing

Presenters:

Steve Temme – Listen, Inc.

 

 

Seminar: Smarter Measurements for Smarter Speakers – Chicago and Boston

/in , /by

Smarter speakers require smarter test methods!

smart_speakersLearn how to test smart speakers, robots, voice-controlled automotive audio and other IoT devices in our practical half-day seminar taught by industry experts Steve Temme, and Marc Marroquin of Listen, Inc.

Although acoustic testing of smart speakers and other voice-activated devices presents challenges such as injecting and extracting response signals, time delays, and triggering the system using voice commands, it is still possible to measure the same parameters as traditional loudspeakers, In addition we can also measure microphone array performance, speakerphone performance and more.

We demonstrate how to do this, focusing on:

  • Stimulating and capturing responses from a device where you don’t have direct access to the microphone or speaker (open loop testing).
  • Delays, asynchronous stimulus / acquisition, and working with the ‘cloud’
  • Testing with real world signals such as speech and music, and how to analyze results from these test stimuli
  • Voice Recognition – key word spotting, wake word testing, directionality, and the effect of background noise on voice recognition performance
  • Telephony – testing smart speakers for hands-free calling

There is no charge for this one-day seminar and lunch will be provided. Space is limited, so please RSVP today.

Print

Locations and Dates:

Addison (Chicago area): Wednesday July 18th, 2018 – 9.00am – 1.30pm. Hilton Garden Inn, 551 N Swift Rd, Addison, IL 60101

Waltham (Boston area): Friday July 20th, 2018 – 9.00am – 1.30pm. Hilton Garden Inn, 450 Totten Pond Rd, Waltham, MA 02451

 

Agenda (9am-1.30pm)

  • Introduction
  • Open Loop Testing
    • What is ‘open loop’ testing?
    • Can you really measure frequency response and harmonic distortion in an open loop?
    • Using Frequency Shifting to time align asynchronous stimuli and response
    • Working with cloud based services
  • Testing with Real World Signals
    • How to use real voice and / or non test tones to evaluate acoustic performance
    • Analysis techniques used for evaluating non test tone stimuli
  • Voice Recognition
    • Key Word spotting / Wake Up word testing
    • Room effects, environmental distractors, and SNR…how they affect voice recognition performance
    • Measuring directionality
  • Telephony
    • What voice quality metrics are important for speakerphones?
    • Challenges of testing speakerphones in different environments
  • Networking lunch

Reserve your space