100 Things #83: Open Loop Measurements Of Smart Devices

Open loop testing enables measurement of smart devices, phones, and any device that cannot play a stimulus and record at exactly the same time. Both speakers and embedded microphones on these devices can be measured with SoundCheck, thanks to it’s module test architecture. With SoundCheck, the stimulus and acquisition are separated, so a stimulus file can easily be loaded onto a smart device and triggered either manually or via network commands. For internet connect devices like smart homes, stimulus files can be uploaded and played directly from the cloud.

Open Loop Measurements Of Smart Devices

Learn more about testing smart devices

Our full Smart Device Testing seminar features test demonstrations of smart homes, Bluetooth headphones, and active noise cancellation. Device design considerations and individual component tests are shown, and how SoundCheck can measure audio quality from R&D to production.

Video Script:

Open loop testing lets us measure a device when it’s not possible to play the stimulus and record the response from SoundCheck at exactly the same time, for example smart devices, phones and more. This is simple in SoundCheck, since its architecture has always allowed analysis to be performed independently of acquisition. In fact, we pioneered open loop testing, originally for testing MP3 players,  back in 2006 – a good 10 to 15 years before others started implementing this capability.

Let’s start by explaining how open loop testing is different from conventional closed loop tests. In conventional audio tests, such as speaker and microphone tests, the stimulus playback and response recording occur simultaneously. The two signals are correlated, and ready to analyze.

Devices such as mobile phones and smart speakers do not have a simultaneous input/output audio path, but we can test both the speakers and the microphone using open loop testing. In open loop testing, the stimulus includes a trigger to tell SoundCheck when to start recording. 

Let’s look at a mobile phone test. For the speaker, a stimulus file is loaded onto the phone and played back either manually or via a command. SoundCheck records the response using a trigger record acquisition. To test the microphone, a stimulus is played through a mouth simulator, and manually or via a command recorded to the phone. The recording is then transferred to SoundCheck for analysis. 

I’ve already uploaded the stimulus file onto my phone, and I’ll manually playback the stimulus file and trigger the record in SoundCheck.  This short “sine chirp” in the stimulus file serves as the trigger and is not analyzed. This is  a sophisticated level and cross-correlation trigger that is more robust and less susceptible to false triggers than simple level and frequency options. The test sequence then uses resampling and  frequency shift post processing steps in the sequence to correct for sample rate mismatches and slight sample clock differences between the device under test and your test interface. This enables perfect alignment of stimulus and response for accurate analysis. I’ll run the sequence…

There you go. The top graph displays the frequency response of the phone’s speaker, to the right is the recorded response superimposed over the stimulus, the bottom graphs include THD, Rub and Buzz and Enhanced Rub and Buzz.

Let’s look at some other examples. These Apple “lightning” wired  earbuds can’t be tested using a standard headphone test interface as these do not support the proprietary connector. However,  the lightning earbuds can be connected to an iPhone, and the phone can playback the test stimulus.

Open loop testing even allows testing via the cloud. With a smart speaker, the stimulus file can be uploaded to the cloud, and accessed and played back from the cloud via voice command through the speaker, using the same triggered record acquisition that I demonstrated earlier.

These are just a few common examples, but with these methods, you can measure just about any device or system that does not have a direct wired connection between signal input and output. Check out our seminar on smart device testing for more open loop demos.

Smart Device Testing Seminar

Testing smart devices such as smart speakers, hearables, watches and more involves a combination of tests from many different application areas, ranging from simple transducer tests to open loop tests, Bluetooth measurements, voice activation tests, telecoms standards, hearing aid measurements and more. In this demo-focused seminar, we demonstrate how SoundCheck makes all the measurements you need throughout the entire smart device design process from component selection to pre-qualification testing.

Smart device testing demonstrations include:

  • Component tests: MEMS microphone measurement, directional microphone measurement, speaker evaluation
  • Prototype tests – open loop speaker test on a smart speaker, open loop microphone test on a smart speaker, Bluetooth hearables test, headphone Active Noise Cancellation (ANC) Measurement
  • Background Noise Tests: calibration, using pub noise to evaluate a microphone’s ability to reject background noise

Presenters: Steve Temme, Mark Latshaw, Steve Tatarunis
Duration: 48 Mins

Smart Device Measurement Resources

This seminar was originally broadcast on June 6th 2023. The recording below does not include the live Q&A at the end for attendee confidentiality reasons. However, several links to additional resources were provided during the Q&A session, and these are provided below.

  1. Internally routing audio: Open Loop microphone testing requires audio to be routed into SoundCheck. We recommend:
    1. Virtual Audio Cable for Windows: https://vac.muzychenko.net/en/index.htm
    2. Soundflower for Mac: https://github.com/mattingalls/Soundflower
  2. In response to a question about whether haptic feedback could be measured in a smartwatch, we shared a SoundCheck test sequence for measuring haptic feedback: https://www.listeninc.com/products/test-sequences/free/linear-motor-test-sequence/
  3. We touched briefly on communications audio tests when smart speakers are used as hands-free devices. To learn more about Communications Testing techniques and standards, please see our Communications Testing Seminar.
  4. Similarly, in answer to a question about testing automotive infotainment systems, we shared our Automotive audio testing seminar

More about how to Measure Smart Devices

Check out our main page on Smart Device Testing, which includes links to test sequences, relevant products and more.

 

100 Things #68: Using Transfer Function to Measure Smart Devices

Modern devices are becoming better and better at filtering out unwanted background noise from calls. This includes noises like sine sweeps, typically used to test these devices. Instead testing these devices with real world signals like speech, music, and noise can bypass noise suppression. The ability to use transfer function in SoundCheck to test these devices with non-linear signals was added in 2005. Transfer function can do more than just measure smart speakers, with the ability to also test loudspeaker impedance or compare measurements of a reference mic to a DUT mic, all with the same SoundCheck module.

Using Transfer Function to Measure Smart Devices

Learn more about transfer function, and other SoundCheck features

Read on about SoundCheck features and functionality, discussing algorithms, automation features, and more. If you’re ready to start testing on your own SoundCheck system, see our full catalog of free Loudspeaker test sequences.

Video Script:

Modern devices such as mobile phones, smart speakers, TWS earbuds and audio infotainment systems use sophisticated DSP algorithms to improve the voice quality and intelligibility of these devices. While these algorithms greatly improve the user experience, they create challenges for the audio test engineer as they often filter out common test signals such as sinewaves.

So, if we can’t use sinewaves, then what are our alternatives? SoundCheck’s transfer function algorithm, which was first introduced in Soundcheck back in 2005, can use broadband signals such as speech, music and noise as a stimulus to measure the frequency response of a device. 

The transfer function algorithm works in the frequency domain to perform a complex FFT (including magnitude and phase) on the stimulus and response waveforms.  From these two spectra, a variety of results can be calculated including frequency response, non-coherent distortion, coherence, non-coherence, signal to noise ratio, cross spectrum, coherent power and non-coherent power. Time domain analysis outputs are also available and include impulse response, auto-correlation of the stimulus and response as well as cross-correlation. A power averaging option can be used when the response waveform contains jitter or other artifacts that would result in phase calculation errors when complex averaging is used.  When power averaging is selected, it limits the algorithm’s output to frequency response, auto spectrum of the stimulus and response waveforms and auto-correlation.

Let’s say we want to measure a microphone embedded in a smart speaker which we know uses DSP to filter out sinewaves.  We construct a compound stimulus containing the smart speaker’s wake word, Alexa for example, followed by a pink noise stimulus having a minimum bandwidth equal to that of our device under test.  When this stimulus is played from SoundCheck, the wake word triggers the smart speaker and its recording of the stimulus playback can be retrieved from the cloud as a WAV file and recalled into SoundCheck for analysis.

Transfer function analysis isn’t limited to just open loop measurements using non-sinusiodal stimuli.  We can use transfer function to make high accuracy electrical impedance measurements by measuring the speakers terminal voltage and current or even use the recorded time waveforms of a reference mic and DUT mic to analyze the response of the DUT mic.  

In this video, we’ve only scratched the surface of the capabilities of this powerful algorithm.  If you’d like to learn more about it, contact one of our sales engineers to arrange a demo. Thanks for watching!

100 Things #6: Open Loop Testing of Smart Devices Using Triggered Record

Triggered record is an essential feature for open loop audio tests – any test where the stimulus and response are asynchronous. This includes, for example measuring smart devices, automotive systems, or anywhere that signals travel wirelessly via Bluetooth or the cloud. In this short video, we review the three different triggered record options in SoundCheck.

Open Loop Testing of Smart Devices Using Triggered Record

Learn more about testing smart devices

Our full Smart Device Testing seminar features test demonstrations of smart homes, Bluetooth headphones, and active noise cancellation. Device design considerations and individual component tests are shown, and how SoundCheck can measure audio quality from R&D to production.

Video Script:

Triggered recording is an essential feature for open loop audio tests – any test where the stimulus and response are asynchronous. This includes, for example measuring smart devices, automotive systems, or anywhere that signals travel wirelessly via Bluetooth or the cloud.

Triggered record was first introduced into SoundCheck in 2007 but did you know that there are now three triggered record options available in SoundCheck? 

Lets take a quick look…

The level trigger is the simplest implementation of this feature.  The user selects the signal path, trigger level and record duration and once the level is exceeded, the step will record for the prescribed duration.

Level triggers can be error prone and for this reason the level and frequency trigger was developed and introduced in 2017.  In order to begin recording, not only does the level need to be exceeded but a sinewave must also be detected. In this case, the stimulus should be prepended with a pilot tone.

The third option, a level and cross-correlation trigger is the most accurate.  This method uses  a chirp-based conditioning tone and searches for the exact sweep range, making it more robust and less susceptible to false triggers than the simpler level and frequency options.