Digital Voice Assistant Latency Test Sequence
Introduction
Digital voice assistants (DVA) will often respond to a wake word or other voice commands with a chime, confirming that the command has been properly received. This sequence measures the latency or delay between a voice command and chime to confirm that the DVA’s response time is within allowable limits.
The sequence uses a compound stimulus comprised of two wave files and two segments of silence as follows:
- Silence 1: Leading silence before wake word
- WAV 1: Wake word
- Silence 2: Silence between wake word and command
- WAV 2: Command
The compound stimulus is played from an artificial mouth (preferred) or calibrated source speaker and a calibrated reference microphone is used to record the stimulus playback and the DVA’s response chimes. A series of post processing steps are then applied to the recorded time waveform to calculate the latency between the wake word, command and their corresponding chimes. The final display shows the stimulus waveform, recorded time waveform, the two latency values and the duration of the compound stimulus segments.
Sequence Configuration
Before running the sequence, some of the sequence steps must be configured:
Steps 3-6 are numeric message steps which are hidden by default. If the desired values won’t change from run to run, you may configure them with appropriate values for your test conditions and leave them hidden. If the values will change from run to run, you can display the numeric message step every time the sequence is run by right clicking on the step of interest, select “Configure step…” and then “Display step when run for”.
Step 3: (Silence 1): Enter the desired value for stimulus leading silence. Default=1s
Step 4: (Silence 2): Enter the desired value for stimulus silence between wake word and command. It should be of sufficient length to account for the duration of Chime 1 and estimated DVA latency. Default=1s
Step 5: (Wake word duration): Enter the duration of the wake word WAV file. Default=1s.
Step 6: (Set intersection value FS): Enter the value for step 12 (command WAV duration detection) Default=10m FS. If you are getting inaccurate results, it may be due to background noise in your test environment. Increasing this value may help.
Step 7: (Set intersection value Pa): Enter the value for steps 19 & 20 (chime start detection) Default=150m Pa. If you are getting inaccurate results, it may be due to background noise in your test environment. Increasing this value may help.
Additional step configuration:
Step 10 (Recall): Configure the desired recall path to the folder containing the command recordings. Default = SoundCheck 22/wav files
Step 12 (Stimulus): Enter the desired stimulus level. Default=80 dB SPL
Step 14 (Acquisition step). You will need to adjust the Record Padding value to ensure that the Command chime is completely captured during acquisition. Record Padding default=6s
Software Requirements
SoundCheck Plus version 22 or later
Hardware Requirements
- Audio interface: Listen AudioConnect 2 or similar (part number 4047)
- Reference microphone: Listen SCM-4 or similar (part number 4012)
- Mouth simulator or source speaker
- Power amplifier: Listen SCAmp (part number 4060). Not needed if using a powered mouth or speaker
Hardware Setup & Calibration
- Calibrate the reference microphone per the instructions in the SoundCheck user manual
- Calibrate the mouth simulator/source speaker per the instructions in the SoundCheck user manual
- Position the mouth simulator at an appropriate distance from the DVA
- Position the reference microphone between the DVA and mouth simulator
You are ready to start the sequence.
Note: Run the sequence directly from the distribution folder as it contains relative paths for autosave and recall
System Diagram
This sequence has been designed for simplicity and has been written to be accessible to 100% of SoundCheck customers. Ways in which you could modify or further develop the sequence include:
- Apply limits to the latency values
- Automate the Command WAV selection through the use of a Python script or the Copy From Config File custom vi
- Add a background noise speaker array to the hardware setup
Get sequence
Download Digital Voice Assistant Latency Test (zip file)
Download Digital Voice Assistant Latency Test (PDF)
