Introduction

This document describes how Audio Weaver has been integrated on the Qualcomm Snapdragon SOC using low-level APIs. It bypasses Audio Reach and provides significant performance improvements. This document should be used in conjunction with the generic description of the Audio Weaver architecture described in Audio-Weaver--Architecture. The overall design is flexible enough to handle all automotive use cases and configurations. You should be able to fully engineer your automotive audio system using Audio Weaver without having to write custom Hexagon code¹.

The platform specific code needed to wrap Audio Weaver is referred to as the Board Support Package (BSP). We will use the term BSP throughout this document to describe this code. The BSP code is based on Qualcomm low-level AudioLite APIs. This is a low-level software layer which provides an RTOS, TDM port I/O, and basic system features.

The document covers integrations on the Gen4 SA8255 and the Gen5 SA8397 / SA8797 “Nordy” chipsets. The integrations are very similar and when they diverge, we will separately document each integration.

Platform Features

• Graphical development

• Multicore support - distribute audio processing across all Hexagon DSPs and the Arm.

• Unified signal flow showing an overall view of all cores and threads

• Integrated profiling

• Highly optimized including HVX support on Gen5 chipsets

• Access to IP

  • Over 500 Audio Weaver modules

  • Qualcomm voice IP

  • Custom module API

  • 3rd party ecosystem

• Real-time audio features

  • TDM serial port configuration via Audio Weaver modules

  • Independent TDM ports automatically synchronize to within 1 sample

  • ALSA I/O configuration via Audio Weaver modules

  • Low latency - as low as 0.5 msec digital-in to digital-out using a 0.25 msec block size

  • Early audio within 1.5 seconds

  • Resynchronizes automatically after CPU overrun

• Software integration

  • BSP configurable via a text initialization file

  • Run-time control via TinyMix APIs

  • Text based control API with integrated data type, range, and array bounds checking

  • Subsystem Restart (SSR) feature reboots DSPs and restarts the Audio Service if there is a critical run-time failure

  • Asynchronous event handling

• Integrated full-featured Matlab API for scripting and regression testing

• Supports all automotive use cases with concurrent operation

• TFLM and ONNX support

• Fully documented

• Online training available

Comparison with Audio Reach

Audio Weaver Real-Time Architecture

This section gives an overview of Audio Weaver’s real-time architecture. Before diving into this section, you should read the Audio Weaver Architecture document, especially Chapter 3 which focuses on real-time audio.

Traditionally, Audio Weaver relied on a single hardware input pin and single hardware output pin for all I/O in the system. All audio channels needed to be sourced by the BSP code and interleaved on a single input. All I/O had to be at the same block size and sample rate. On the Snapdragon, this has been changed to doing I/O with TDM and ALSA modules instead. This gives greater flexibility and works around many of the earlier limitations. We are in the process of updating Audio Weaver Designer to operate without the traditional “HW Input” and “HW Output” pins. For now, you need to include these and they provide real-time I/O on the PC. Future releases will remove them completely.

Audio Weaver’s real-time architecture is based on fully synchronous operation. Audio processing occurs in fixed block sizes, and block sizes through the system are multiples of the fundamental block size of the system. This provides consistent CPU loading (no spikes), fixed scheduling, and efficient use of computational resources.

Audio Weaver utilizes double buffering at the hardware inputs and outputs. In a properly designed system that operates at a fixed block size, Audio Weaver is able to achieve the theoretical lowest latency of 2 x block size. There is no extraneous buffering in Audio Weaver which negatively affects latency.

Basic Block Size

Audio Weaver real-time processing requires one of the TDM audio devices to generate an interrupt at the Basic Block Size of the system. The Basic Block Size is the smallest interval of audio processing and all other threads (layouts) execute at multiples of the Basic Block Size. For systems using Road Noise Cancellation (RNC), we recommend setting the Basic Block Size to 0.25 or 0.5 msecs (12 or 24 samples at 48 kHz). For systems without RNC, we recommend setting the Basic Block Size to 1 msec (48 samples at 48 kHz). ALSA devices cannot be used to generate this interrupt, only TDM devices can.

One of the serial ports in the system operates at the fundamental block size and is used as the underlying “master clock” for the system. This serial port is called the Synchronous Master and is located on the Primary Audio Controller (PAC). The Synchronous Master generates real-time interrupts to the PAC and the PAC signals the other audio processing cores via IPCC when they have work to do as shown below.

The PAC has the highest interrupt rate and the other audio processing cores can operate at multiples of the fundamental block size. For example, the PAC may operate at 0.5 msec while the GPDSP or Arm could operate at 10 msec.

Synchronous Multicore Operation

This synchronous operation extends to all audio processing threads distributed across the various cores. Consider a 3 core design that operates at the following block sizes:

• Core 0: 0.5 msec, 0.5 msec, 4 msec, 20 msec [PAC]

• Core 1: 5 msec, 5 msec, 5 msec, 5 msec [Wakes up every 5 msec based on IPCC]

• Core 2: 10 msec, 10 msec, 20 msec, 20 msec [Wakes up every 10 msec based on IPCC]

Processing is fully synchronous and triggered by the 0.5 msec interrupt controller. The system wide pattern of audio processing (“pumping”) is shown below:

This architecture imposes some restrictions as to how block sizes can be used. Here are some examples:

1. An algorithm cannot change its block size on the fly at run-time. (To realize this, you’ll need to operate the two algorithms in parallel and Activate / Inactivate the processing at run-time. You’ll need twice the memory, but the CPU load is constrained.)

2. Algorithms with block sizes of 240 and 256 samples cannot be directly cascaded. (To accomplish this, you would need a basic block size of 16 samples. You would have to first buffer down from 240 to 16 samples and then buffer up from 16 to 256 samples. This places the modules into layouts with clock dividers to 15 and 16.)

3. Asynchronous sample rate conversion cannot be supported inside the signal flow. This must be done at the edges in specialized modules (like the TDM and ALSA modules).

Resynchronization

Audio Weaver operation is based on meeting real-time deadlines. In a complicated SOC that is executing multiple applications simultaneously, it is possible that an audio processing thread may not finish in time. Before Audio Weaver “pumps” a thread, it checks if the last pump has already finished. If not, Audio Weaver declares that an overrun has occurred and starts a resynchronization process. This is processed as detailed in Section 3.2.2 of the Audio Weaver Architecture document.

During resynchronization, Audio Weaver stops pumping audio. It resets ChangeThread and BufferUp/BufferDown modules so that the ping-pong buffers are in phase and aligned. Then it restarts pumping. During resynchronization there will be a momentary dropout in audio.

Audio Weaver starts resynchronization if there is an overrun in any thread. And if an overrun occurs, it stops and restarts all audio threads. It is not possible to resynchronize only the thread in which the overrun occurred.

Tracking Overruns

When debugging overruns, the SystemVariable module is very helpful. The module can be configured to output the “ResetCounter”. This is a global variable, and the SystemVariable module can be placed on any core and in any thread. The ResetCounter increments whenever an overrun occurs. Often, you may not hear a pop or click, but you can see the moment that the overrun occurred using this module. The module can also count resets during long unsupervised tests.

You can go a step further and report overruns to the HLOS using the SystemVariable module together with an Event module. A reference implementation is shown below. The Event module must have an objectID assigned to it so that it is visible to TinyMix.

This processing can run on any audio processing core and detects overruns throughout the system (even on another core). It can run in a lower priority thread to save MHz. We recommend that it runs every 100 msec.

The SystemVariable module outputs the number of times that there was an overrun in any real-time Audio Weaver thread (in a layout). The Delay and Compare modules are additional logic that cause the Event module to trigger when the reset count changes.

After an overrun has been reported, an application can separately query per-thread profiling information. This is available through the AWECore tuning command “PFID_GetAllProfiling” which returns detailed per-thread profiling information. This call can be used to identify exactly which real-time thread is overflowing. The command is issued to one of the audio processing cores (e.g., ADSP or GPDSP), and it can be used to read information for one thread (layout) or from multiple threads (layouts). For each layout, it returns:

timePerProcessMeasured - number of clock ticks between calls to the layout’s pump function. This is measured in real-time.

timePerProcessCalculated - number of clock ticks between calls to the layout’s pump function. This is computed theoretically based on the profiling clock speed and the pump rate of the thread. For example, if the thread is executing every 1 msec, then the ADSP is using a profiling clock of 19.2 MHz, this will be 19200.

averageCycles - average number of clock ticks needed to complete execution. This is smoothed with a first order smoother.

instantaneousCycles - instantaneous clock ticks needed to complete the last pump of the layout. This changes continuously.

peakCycles - the peak number of clock ticks that have been measured since system startup.

overflowCount - number of times that this layout has not completed in real-time and caused a systemwide reset event.

Profiling

Audio Weaver uses a time base for computing real-time loads. On the Hexagon DSP, we utilize the 19.2 MHz hardware “tick timer”. On Linux, we use the High Precision Event Timer which runs at 10 MHz. Suppose you are running audio processing on the Hexagon ADSP with a block size of 48 samples at a 48 kHz sample rate. The block time is 1 msec and Audio Weaver computes the number of hardware ticks available as 19.2 MHz x 1 msec = 19,200. This is computed based on known quantities and is not measured at run-time. Next, Audio Weaver measures how long it takes for the 1 msec processing thread to compute. Suppose it takes 6,000 hardware ticks. Audio Weaver will then report the load as:

6,000/19,200 = 31.3%

If the ADSP is running at 1.344 GHz, then the reported load will be 421 MHz.

Low Power Sleep and Wake Up

In some systems, there is a requirement to stop audio processing and then restart at a later time. When you do this, it is important to clear out internal state variables. Otherwise, when you wake up the system, a burst of audio will be heard due to state variables holding audio samples when the system entered the low power sleep state.

TinyMix allows you to send a “Stop Audio” command to the Audio Weaver processing cores. This command halts all audio DMA and also resets all internal state variables and clears out wire buffers. Issue this command prior to entering the low power state. Later, to “wake up” the audio processing, just issue the “Start Audio” command. DMA will resume and no pops or clicks will be heard⁴.

TDM Serial Port Modules [R4.0]

Serial port I/O is accomplished by two modules: SnapdragonTDMSource and SnapdragonTDMSink. These modules appear in the module browser as shown below.

You can add multiple TDM Serial Port modules to your signal flow and interface to multiple hardware devices. Release 4.0 has the following restrictions:

• All TDM ports must be on the ADSP.

• All TDM ports must be at 48 kHz.

• The block size is variable but all TDM ports must be at the same block size.

• All TDM port interrupts start at the same time in the code, but there could be a small latency shift between them.

• All TDM ports must be in clockDivider 1 and operate at the Basic Block Size.

• All TDM port clocks must be derived from the same master clock. That is, they must be at the same sample rate and synchronous.

Module Arguments [R4.0]

In this release, there are separate modules for interfacing to input and output serial ports. The modules are configured via their module arguments. The Source and Sink modules have the same arguments as shown below. The settings map directly to the standard Qualcomm registers exposed in Audio Reach.

The last two items (lpaif_idx and lpaif_type) specify the serial port to use. You can only use a serial port once in your design, and it can be used for both input and output.

TDM Serial Port Modules [R4.1]

Release 4.1 is adding

• Can be on different Hexagon DSPs

• Can be on different clockDividers

• Can utilize different block sizes and sample rates

• TDM initialization follows a prescribed sequence including initializing bitclk first followed by the frame sync.

• Synchronous TDM ports time align themselves automatically to within 1 sample.

• Asynchronous serial ports including an ASRC and output data at a fixed sample rate and block size.

Synchronization Modes

The serial port modules support synchronous and asynchronous operation. In synchronous operation, the serial ports are all clocked off the same master clock. In asynchronous operation, the serial port has a different clock that is not equal to the master clock. Operation can be further classified as

Synchronous Master - Operates at the smallest block size and generates the primary interrupt. This port must be on the Primary Audio Core (PAC) which is specified in the system configuration file. Only one serial port in the system can be defined as the Synchronous Master.

Synchronous Aligned – the serial ports start simultaneously and remain in lock step throughout. This requires external hardware connections to share serial port clocks and framesyncs. On the SA8255, this is achieved by tying the serial ports to the master clock, starting DMA, and then starting the frame sync.

This is achieved on SA8797 by configuring the serial ports to use the master clock together with the Group Bonding mechanism to start all DMA simultaneously. All synchronous serial ports must be on the same audio processor (TDM bonding only works on a single core.

Synchronous Aligned serial ports are able to achieve the lowest possible input to output latency (equal to 2 times the block size). This is the ideal solution for low latency applications like RNC.

Synchronous Unaligned – the serial ports are driven off the master clock but do not start synchronously. The data stream is driven by the same clock source/root as the master clock, but is misaligned in time. The vast majority of custom designs will use this approach, and the Audio Weaver TDM modules are able to time align these ports to within 1 sample of the Synchronous Master.

Asynchronous – the serial port uses its own clock source / domain. It has a double buffer and hooks its own DMA completion interrupt. In its interrupt handler, audio data is copied from the DMA double buffer to the input buffer of an Asynchronous Sample Rate Converter. Then in the module’s processing function, sample rate conversion occurs and data is copied from (to) the temporary buffer to the Audio Weaver wire buffer.

We will refer to these different synchronization schemes throughout this document and will use the abbreviations SM, SA, SU, and AS for Synchronous Master, Synchronous Aligned, Synchronous Unaligned, and Asynchronous operation.

There will be two different Audio Weaver modules for interfacing to the TDM ports, The SnapdragonSyncTDMPort will handle all of the synchronous use cases and will support input and output ports. The SnapdragonAsyncTDMPort will handle the asynchronous use cases.

The serial ports on the Snapdragon record a 64-bit time stamp upon buffer completion (the ping/pong switch). This time stamp has a resolution of 19.2 MHz and is written by the hardware peripherals. It is very accurate and does not rely on a software interrupt which may have jitter. This time stamp information will be used to align unaligned transfers and for the ASRC control loop needed for AS ports.

There are separate synchronous and asynchronous port interface modules as shown below. Each module can be used for input or output.

We describe the Synchronous and Asynchronous ports separately.

Hexagon TDM Port - Synchronous [R4.1]

This module supports all of the Synchronous clocking schemes described above.

Module Arguments [R4.1]

direction - specifies whether this is a sink device (=0) or a source device (=1):

When configured as a TDM_SOURCE, the module has an audio output port (on the right), and a Timing Information port (TI) located on the top:

When configured as a TDM_SINK, the module has an input port on the left side and a Timing Information port (TI) located on the top:

numChannels, blockSize, and sampleRate - specify the properties of the TDM port. If this is an input module, then these values will also specify the properties of the output wire. If this is an output port, then the wire properties at the input must match these values. The sampleRate value is taken from a droplist:

clockingMode - specifies the clocking properties that was described above:

dataFormat - taken from AudioLite. You should only use “DATAFORMAT_FIXED_POINT”.

All other properties are taken directly from the AudioLite APIs and are used when opening the port.

Timing Information Output Pin

This pin has a size of 6x1 samples and contains integer data. (This is a control signal with 6 channels and a block size of 1). It has a sample rate of sampleRate/blockSize. It contains these values:

isStarted - Boolean value indicating whether the device has been enabled

timeStampL - the low 32-bits of the time stamp when the ping/pong buffer of the module last flipped.

timeOffsetFromMaster - time skew between this port and the SM. In microseconds. This is measured based on the DMA time stamps.

rwOffset - computed offset used to index early or late in the DMA buffer

sampleLatencyFromMaster - the final latency in samples relative to the SM port.

Module Inspector [R4.1]

Start - User Parameter. Allows you to turn the port on or off.

DMA Safety Delay - User Parameter. Additional use specified time delay that will be used in the offset calculation. This will be used for testing and for situations when we want to give more time for the DMA to complete. The actual math used is:

DMA Offset = floor((TS_SM - TS_SU + dmaSafeDelay) / (Sample Period))

Startup Settling Count - User Parameter. This specifies how many blocks pump cycles the module will do and continue to output zero data. This gives the module time to calculate its read/write offset so that it doesn’t introduce a pop or click when it changes.

Startup Latency - User Parameter. The BSP will wait the specified number of microseconds before enabling the serial port clocks. This value is used for testing and will allow us to play with the alignment between ports. During normal usage, it should be set to 0.

Error Status - Read Back State. Reports errors in startup or operation. 0 indicates no error.

Pump Counter - Read Back State. Increments every time the module’s processing function is called.

Time Offset to Master - Read Back State. This is the computed time skew between this port and the SM This is a floating-point number and represents microseconds.

DMA Ptr RW Offset - Read Back State. This is the computed sample offset when accessing the DMA buffer. This is calculated per block. Instead of accessing the buffer at the expected start of the ping or pong buffer, this offset is added.

Offset to Master - Read Back State. This is the true offset in samples relative to the SM device. It is used to add compensating delay in the system. If a SU device has a latency of +1 samples, then the SM should be delayed by 1 sample.

Time Alignment Between Synchronous Devices [R4.1]

The SM and all SA devices operate off the same clocks and will be completely synchronized; 0 sample latency.

When operating as a SU port, there is internal logic which uses the time stamp of each port’s DMA interrupt to align the data with the SM port. If you have dmaSafetyDelay set to 0 then the SU port will be aligned to within 1 sample for input and output ports. This could be a fractional sample delay since the clocks start independently.

If dmaSafetyDelay is nonzero, then this will add additional latency. dmaSafetyDelay is specified in microseconds and you end up with different latency from boot-to-boot. Fortunately, the timing output pin reports the actual latency (in samples) in sampleLatencyFromMaster and you can use this to time align signals in your layout. For example, if the SU port reports a sampleLatencyFromMaster of 1 sample, then it means that the SU port is 1 sample delayed relative to the SM. To align the ports, you need to add 1 sample of delay to the output of the SM port.

Hexagon TDM Port - Asynchronous [R4.1]

The Asynchronous input port operates as shown in the figure below. There is a device side, which interfaces to the serial port hardware and manages the DMA, and an Audio Weaver side which operates within the context of the Audio Weaver pump function.

Module arguments specify the number of channels and the output block size and sample rate in Audio Weaver. These can be changed at design time only. The device uses the same number of channels, but has separate parameters (run-time changeable) for the device block size and device sample rate. This allows you, for example, to initially operate the device at 32 kHz with a block size of 256 samples, while the audio received by Audio Weaver is 48 kHz with a block size of 48 samples. At run-time, you can change the device to 16 kHz with a block size of 128 samples, while the audio received by Audio Weaver continues to be 48 kHz with a block size of 48 samples.

Asynchronous Sample Rate Converter (ASRC) Algorithm

The first release of this module is based on DSP Concepts’ ASRC library. We expect future versions of this module to leverage a different ASRC algorithm from Qualcomm. This section briefly describes DSP Concepts’ library and it aids in understanding the overall implementation.

A conceptual view of the library and how it interfaces between the DMA buffers and Audio Weaver wire buffer is shown below.

This library has an input FIFO called the “jitter buffer” which stores samples at the input rate. The jitter buffer provides margin allowing the algorithm to adapt to changing sample rates. The jitter buffer is filled by calls to the function dspc_asrc_write(), and this occurs in the DMA interrupt for the serial port. The call dspc_asrc_write() executes quickly and copies a complete deviceBlockSize of data into the jitter buffer.

Then in the module’s processing function, which executes at Audio Weaver’s pump rate, the library calls the dspc_asrc_read() function. This performs fractional sample interpolation and writes the output Q31 data into the module’s output wire buffer. Every time the module pumps, it outputs one (Audio Weaver) block of data. The bulk of the processing occurs in this function and will be included when the module is profiled.

Input and output data is in Q31 format. A temporary buffer is used for format conversion prior to the call to dspc_asrc_write(). The ASRC library is designed to be thread safe with the dspc_asrc_write() and dspc_asrc_read() functions being called from different threads and even executing at the same time.

The implementation also contains a control loop for computing the input and output sample rates. This calculation uses the time stamps of the DMA buffers to adjust the fractional interpolation process.

The architecture for output devices is slightly more complicated and requires an additional FIFO in the output path as shown below.

Several things happen in the module’s processing function. First, it writes one (Audio Weaver) block of data into the jitter buffer using the call dspc_asrc_write(). The input data is at Audio Weaver’s sample rate. Next, it converts all data in the jitter buffer to output samples. This Q31 data is written into the temp buffer. Then it is format converted to the bitwidth of the TDM port and written into an output FIFO. All this processing happens in the Audio Weaver pump function. Then, in the TDM port’s interrupt handler, data is read from the FIFO and written into the DMA buffer. With this approach, everything that occurs in the Audio Weaver pump function is captured in profiling.

Module Arguments [R4.1]

The Asynchronous TDM port adds 2 more arguments as shown below.

maxDeviceBlockSize - specifies the maximum block size that the device will ever be configured for. This is used to size internal buffers.

asrcQuality - a droplist which configures the quality of the asynchronous sample rate converter. The choices are shown below and “Music” is the default.

The quality metric maps approximately to the distortion level in the converted signal. The distortion will be at least this many dB below the signal level:

Voice - 80 dB

Music - 100 dB

HiFi - 120 dB

This feature has not yet been implemented. In R4.1, all ASRCs are at the “Voice” performance level.

Module Inspector for Input Devices [R4.1]

The inspector for the Asynchronous TDM port when configured as an input is shown below.

Start - User settable parameter. Enables DMA and starts streaming audio data.

Set Device Block Size - user settable parameter. Specifies the deviceBlockSize and allows you to configure the DMA buffer size and interrupt rate of the device.

Set Device Sample Rate - User settable parameter. Specifies the sample rate of the device.

Port ISR Count - Read back state. Increments every time the device generates an interrupt at the ping / pong buffer DMA switch. This will update deviceSampleRate/deviceBlockSize times per second.

Pump Counter - Read back state. Increments every time that the module pumps in Audio Weaver. This will update sampleRate/blockSize times per second (these are the settings on the “Audio Weaver side”.)

Error Status - Read back state. Indicates if everything is functioning correctly. 0 indicates no error. See the table below.

Reset Counters - User settable parameters. Clicking “reset” will clear all counters and error status on the inspector.

ASRC Prefill Level - User settable parameter. Number of samples at the deviceSampleRate that will be filled in the ASRC’s internal “jitter buffer” before conversion starts. This helps to prevent ASRC underruns. We recommend setting this to 2 x deviceBlockSize.

ASRC Input Avail - Read back state. Free space in the ASRC’s jitter buffer. This is how many device samples can be written into the buffer before an ASRC Overrun occurs.

ASRC Output Avail - Read back state. Number of samples at Audio Weaver’s sample rate that can be pulled from the ASRC. If this goes to zero, then an ASRC Underrun occurs.

ASRC Overruns - Read back state. Counts the number of times that the DMA completes and there is insufficient room in the ASRC jitter buffer to receive the data.

ASRC Underruns - Read back state. Counts the number of times that Audio Weaver pumps and wants to pull data out of the ASRC, but there is insufficient data in the jitter buffer.

Module Inspector for Output Devices [R4.1]

The inspector for the Asynchronous output device adds another row of controls for configuring and monitoring the FIFO.

FIFO Prefill Level - User settable parameter. Specifies the number of samples that must be in the FIFO before data is streamed via DMA. This prevents FIFO underruns from occurring.

FIFO Input Avail - Read back variable. Number of samples that can be written by the module’s pump function into the FIFO. If this goes to zero, then a FIFO Overruns occurs.

FIFO Output Avail - Read back variable. Number of samples that can be read by the DMA handler. If this goes to zero, then a FIFO Underrun occurs.

FIFO Overruns - Read back variable. Number of times that the FIFO has overrun. This indicates that Audio Weaver wants to write ASRC output data, but the FIFO is full.

FIFO Underruns - Read back variable. Number of times that the DMA interrupt handler wants to read data from the FIFO, but there is insufficient data.

Table of Error Codes for AsyncTDMPort

Run-time Reconfiguration of Asynchronous TDM Ports

The asynchronous ports allow the device properties to be changed at run-time. Use the following sequence of operations to reconfigure the port:

1. Disable the device.

   1. On the inspector, uncheck “Start”.

   2. Alternatively, via the HLOS control interface, set the variable “isStarted” to zero. This halts DMA and frees DMA memory.

2. Reconfigure the device.

   1. On the inspector, set the “Device Block Size” or the “Device Sample Rate”.

   2. Alternatively, via the HLOS control interface, set “deviceSampleRate” and “deviceBlockSize”.

3. Enable the device

   1. On the inspector, check “Start”.

   2. Alternatively, via the HLOS control interface, set the variable “isStarted” to one.

   3. This allocates DMA memory (based on deviceSampleRate) and opens the TDM using deviceSampleRate. DMA then resumes.

Audio Startup Sequence [R4.1]

When a system is built using Audio Weaver Designer, or when an AWB file is loaded, the TDM modules are individually constructed based on their location in the signal flow. The TDM ports, however, are not started until the “audio_start” command is received. (This is usually the last command when building or loading a system.)

The Snapdragon BSP starts SM and SA TDM ports using the following sequence:

1. Enable the bit clock for the SM port

2. Enable DMA for the SA ports. This enables the DMA but does not start interrupts.

3. Enable SM port framesync. At this point, the SM and SA ports will be operational and generating interrupts synchronously.

After the SM and SA ports are running, then the SU and AS ports are started. There are no special startup requirements for the SU and AS ports.

On the SA8255, the TDM ports can be placed on different Hexagon cores. Each DSP boots in order and the serial ports on each core follow the sequence shown above. On the SA8255, all SM and SA ports must be on the PAC; you cannot place them on different DSPs.

During early audio, the TDM ports are started as follows:

1. The Primary Audio Core (PAC) initializes and loads its AWB first.

   1. All TDM ports on the PAC are initialized using the sequence above.

   2. Audio start command starts DMA on the PAC. Audio is active.

2. PAC signals GPDSP0 to start initializing

   1. GPDSP0 loads its own AWB

   2. GPDSP0 initializes the TDM modules in its AWB

   3. TDM audio interrupts start on GPDSP0

   4. GPDSP0 signals the PAC that it is done initializing

   5. The PAC signals GPDSP0 to pump audio

3. PAC signals GPDSP1 to start initializing

   1. GPDSP1 loads its own AWB

   2. GPDSP1 initializes the TDM modules in its AWB

   3. Audio interrupts start on GPDSP1

   4. GPDSP1 signals the PAC that it is done initializing

   5. The PAC signals GPDSP1 to pump audio

4. Early audio is fully initialized

TDM Port Latency

Usage Guidelines

1. Do not make the block sizes of your TDM ports too large. There is a limited amount of DMA memory on the Snapdragon and you may exceed this. (Audio Weaver testing has been with block sizes up to 5.0 msec.)

2. You are only allowed to define a single TDM port as being the Synchronous Master. If you define multiple ports as Synchronous Master, then you will get a build error.

3. You are allowed to place the TDM Port modules into different threads. For example, one port operates at a block size of 1 msec and is placed in thread 1A. A second port operates at a block size of 5 msec and is placed in thread 5A. A third port operates at a block size of 1 msec and is placed in thread 1B.

4. Large block sizes lead to more efficient operation. You should restrict the small block sizes (0.25 and 0.5 msec) for use cases that require low latency. Other ports doing non-latency sensitive operations (like radio or Bluetooth), should use larger block sizes.

5. If you stop a port, it will stop the audio DMA and interrupts. The DMA memory is freed up. When you start the port, DMA memory is reallocated and DMA interrupts start again. SA ports will restart and be time aligned to the SM port. SU ports will restart and resynchronize themselves to the SM port.

6. The SM, SA, and SU ports have a single block size argument. This block size is used to set the block size of the DMA interrupt and also the block size at the output (or input) audio pins in Audio Weaver.

7. The AS port has separate sample rates and block sizes for the device configuration and the Audio Weaver configuration. After stopping an AS device, you can change the device block size and sample rate, and these new values will be used when the port is restarted. This feature is specifically for interfacing with Bluetooth devices.

8. TDM ports can only be accessed from the Hexagon DSPs. They are not available on the Arm.

9. On the SA8255, the DSPs are all part of the early audio boot sequence and thus all TDM ports must be initialized during early audio. On Nordy, only DSP0 is part of the early boot sequence. TDM ports can be on DSP0 and on DSP1 or DSP2 which are part of the late boot sequence.

Troubleshooting

Asynchronous TDM Input Port

1. ASRC Overruns occur. Should not happen in normal operation, but can happen if:

   1. The device being interfaced to is the clock master, and the sample rate of the device does not match deviceSampleRate.

   2. The module is no longer “Active” in Audio Weaver. Pumping does not occur and the ASRC jitter buffer is not emptying.

2. ASRC Underruns occur.

   1. The device being interfaced to is the clock master, and the sample rate of the device does not match deviceSampleRate.

   2. This can also happen if Audio Weaver Weaver is pumping and there is not enough data in the ASRC’s jitter buffer. You should increase the ASRC Prefill Level on the inspector. Set the level so that there is enough data to generate 2 output blocks of data at the output Audio Weaver block size and sample rate.

Asynchronous TDM Output Port

1. ASRC Overruns. Should never happen.

2. ASRC Underruns. Should never happen.

3. FIFO Overruns

   1. The device being interfaced to is the clock master, and the sample rate of the device does not match deviceSampleRate.

4. FIFO Underruns

   1. The device being interfaced to is the clock master, and the sample rate of the device does not match deviceSampleRate.

   2. Increase the FIFO prefill level so that it equals at least 2 deviceBlockSize samples.

ALSA I/O

This section describes the modules for streaming audio between the High-Level OS (HLOS) and Audio Weaver. There are 4 modules provided:

The “ALSA Sink Endpoint” and “ALSA Source Endpoint” modules look and act like hardware pins, but they are implemented as Audio Weaver modules. As modules, they have all of the functionality of a module (module arguments, parameters, inspectors, thread placement, etc.). You should add one ALSA Source module for each ALSA playback device, and one ALSA Sink device for each ALSA capture device. The ALSA I/O modules can be on any of the audio processing cores including Hexagon DSPs and Arm cores.

By default, the ALSA Sink and Source modules exchange data at a fixed sample rate. If your HLOS application needs to stream audio and multiple sample rates to the same playback or capture device, then you need to include the Sample Rate Converter modules.

TinyALSA Integration

Behind the scenes, these modules are implemented as TinyALSA plugins. Data is streamed via shared memory and care was taken to minimize the number of buffer copies. A figure showing the HLOS and Audio Weaver portions of an ALSA Source device are shown below.

In Audio Weaver, you specify the output wire properties (numChannels, blockSize, and sampleRate) as module arguments. These are fixed values and do not change at run-time. You also specify the deviceName (up to 31 characters) and a unique integer deviceID. The HLOS application opens the ALSA playback device based on its deviceName or deviceID. When the device is opened, it also sets these properties:

numChannels - number of channels that the HLOS will stream.

blockSize - number of samples that the HLOS will write per transfer.

sampleRate - sample rate of the data.

bitWidth - number of bits per sample (i.e., 8, 16, 24, or 32). All data is fixed-point with a fractional representation.

bufferSize - size of the circular transfer buffer, in samples. We recommend that the bufferSize is at least 4 x blockSize. Larger buffers reduce underruns but increase latency.

startLevel - used during device startup. Number of samples that must be in the circular buffer before Audio Weaver starts reading data.

stopLevel - TBD.

These properties are independent of the Audio Weaver arguments. For example, the HLOS can use a blockSize of 480 samples (10 msec) while Audio Weaver uses a blockSize of 48 samples (1 msec). Other mismatches are handled as follows:

If there is a sampleRate mismatch, then if the module is configured to use the SRC, then there is no error. Otherwise, there will be an error when the device is opened.

bitWidth mismatch. Never happens. The HLOS can stream at any bitWidth and Audio Weaver always converts this to fract32.

numChannels mismatch. Audio Weaver employs its “channel matching” algorithm. For example, suppose that Audio Weaver is configured to output 4 channels. If the HLOS opens the device for 2 channels, then the HLOS data will be on channels 1 and 2 while channels 3 and 4 will be set to zero. If the HLOS opens the device for 6 channels, then channels 1 to 4 will be output on the Audio Weaver pin while channels 5 and 6 will be skipped. Do we do this or do we require that the number of channels match perfectly between the HLOS and Audio Weaver?

ALSA Source Module

The module appears as follows on the canvas. It appears as a “pin”, but is actually a module.

The module has two output pins. The (unlabeled) pin on the right hand side is the audio data. The “BOS” (“Beginning of Stream”) pin at the bottom of the module contains stream status information. It outputs an integer valued control signal on a block-by-block basis and indicates:

0 = Stream is idle. No audio is being streamed.

1 = First block of the stream.

2 = Middle of the stream

3 = Last block of the stream.

Module Arguments

deviceName - string giving a unique identifier to the device. The HLOS can search for devices based on this name. The name is limited to 31 or fewer characters.

{numChannels, blockSize, sampleRate} - set the output wire properties of the device. As shown above, the device outputs stereo data, every 20 msec, at a sample rate of 48 kHz.

deviceID - integer ID which is exposed to the HLOS. This is the deviceID passed to the pcm_open() command and is used to find the device. Note, each ALSA device needs to have a unique deviceID and this is checked at build time.

vmMask - bit mask which specifies which OS’es the audio device is available in. For example, if you have a “safety chimes” device, it should only be available in the PVM, instead of the GVM. More details are in Section TBD.

isASRC - Boolean which indicates whether a sample rate converter is available to convert to the output rate of 48 kHz. This is currently not implemented when AWE core on ARM.

The circular buffer in shared memory is not allocated by Audio Weaver, but by the TinyALSA plug-in when the audio device is opened by the HLOS. This memory is allocated outside of the Audio Weaver heaps and you will not see this memory in the Audio Weaver profiling.

Module Inspector

The inspector is used for viewing the state of the module and debugging. There are no user settable parameters here.

{Sample Format, Buffer Size, Start Threshold, Stop Threshold} - Parameters set when the playback device is opened by the HLOS.

Device State - Can be one of 3 values:

DSP Init - Audio Weaver has finished initializing the device, but it has not yet been opened by the HLOS. Integer value of 1.

Stopped - The HLOS has opened the device but is not streaming data. Integer value of 2.

Started - The HLOS has opened the device and is streaming data. Integer value of 3.

Is Started - Boolean which indicates whether the HLOS is streaming audio data.

FIFO Level - Number of samples in the circular transfer buffer that can be read by Audio Weaver. If this goes to zero, then an underrun occurs.

FIFO Free - Number of samples that can be written into the transfer buffer by the HLOS. If this goes to zero, then an overrun occurs.

{Write Index, Read Index} - pointer offsets when accessing the circular buffers.

XRUN Count - Number of overruns or underruns that have occurred. The ALSA Source device can only have underruns.

Error Status - Internal API error status. Should be 0 during normal operation.

ALSA Source Sample Rate Conversion Module

If you need Audio Weaver to do sample rate conversion of ALSA streams, set the “isSRC” module argument to true. This will add two more pins to the ALSA Source module which are used to exchange timing information. These pins are used in conjunction with the ALSA Source Sample Rate Converter module as shown below:

In this example, the ALSA Source SRC module is configured to output a block size of 48 samples at 48 kHz. The ALSA Source SRC module contains the DSP Concepts’ ASRC library which was described in Section TBD. The ALSA Source SRC module “pulls” data from the ALSA device and writes it into the ASRC’s jitter buffer. It pulls enough data to generate 48 samples during the next pump cycle. The ALSA Source SRC module communicates how many samples are required during the next pump cycle by the integer value in its TI output pin. For example, if the sample rate of the ALSA stream is 44.1 kHz, then the ALSA Source SRC would request 44 or 45 samples in order to generate 48 output samples at 48 kHz.

The ALSA Source module shown above has an output block size of 960 samples. 960 samples are not transferred every time, only as many samples requested by the ALSA Source SRC⁵. You should configure this to be large enough to support any data request by the ALSA Source SRC. Suppose that the HLOS needs to stream data at 384 kHz. 1 msec of data now equals 384 samples and the wire size should be oversized to allow the ALSA Source SRC to prefill data. We recommend sizing the output block size of the ALSASource module at 2 times the largest block size that is expected. In this case, you set it at 768 samples. This sizing must take into account the output block size of the ALSA Source SRC module. Suppose it is outputting 10 msec of data instead of 1 msec, then the input buffer should be sized at 7680 samples. There is no harm (except in wasted memory) in oversizing ALSA Source module’s output wire.

The wire that connects the output of the ALSA Source SRC’s TI pin with the TI input of the ALSA Source module is a feedback wire. You’ll see that this wire is drawn in blue in the figure above. You need to manually configure this wire in Audio Weaver. Right click on the wire, and select “Edit Feedback Properties”. Configure the wire as shown below:

The Timing Information wire between the ALSA Source module and the ALSA Source SRC module contains 3 integer values. You typically don’t have to look at these, but the curious person may want to know:

Wire[0] - isStarted. Boolean

Wire[1] - ALSA sample rate.

Wire[2] - Number of samples written to the output wire. This is usually the number of samples requested by the ALSA Source SRC, but can be smaller if there is insufficient data in the ALSA device’s circular buffer.

If the ALSA device has not started streaming, then all 3 values will be zero.

ALSA Source SRC Module Arguments

The module arguments configure the output wire properties in Audio Weaver.

The number of channels is not specified but is instead inherited from the ALSA Source module.

ALSA Source SRC Inspector

lastDeviceState - Read back variable. Indicates whether the device is active, or not.

lastALSASampleRate - Read back variable. Sample rate of the data supplied by the ALSA device.

InterpolationMethod - User settable parameter. Specifies whether the device implements Linear or Cubic interpolation. Currently, this does nothing and only linear interpolation is used.

outBlockSize - Read back variable. Output block size of the Audio Weaver. This should equal the constructor argument outBlockSize.

inputAvail - Read back variable. Number of samples at the ALSA sample rate that can be written into the ASRC’s jitter buffer.

outputAvail - Read back variable. Number of samples at the Audio Weaver sample rate that can be read from the ASRC based on the level of the jitter buffer.

numSamplesRequested - Read back variable. Number of samples requested from the ALSA Source device. These are to be sent during the next pump cycle.

numSamplesReceived - Read back variable. Number of samples that were actually received from the ALSA Source device.

errorStatus - Read back variable. Returns API error status. Should be 0 during normal operation. See Section TBD for how to interpret this value.

underRun - Read back variable. Number of underruns when attempting to read data out of the ASRC. If you get underruns, then the HLOS is not writing data fast enough.

overRun - Read back variable. Number of overruns. An overrun indicates that the jitter buffer in the ASRC is full. This should never happen for ALSA Source devices.

reset - User settable parameter. Click “reset” to set the errorStatus and all counters back to zero.

ALSA Sink Module

This module streams audio from Audio Weaver to the HLOS. It is similar to the ALSA Source Module. It appears as a “pin”, but is actually a module.

Module Arguments

These are shown below but are exactly the same as the ALSA Source module

Module Inspector

The inspector is shown below but is exactly the same as the one for the ALSA Sink module.

ALSA Sink Sample Rate Conversion Module

If you need Audio Weaver to do sample rate conversion of ALSA streams, set the “isSRC” module argument to true. This will add two more pins to the ALSA Sink module which are used to exchange timing information. These pins are used in conjunction with the ALSA Sink Sample Rate Converter module as shown below:

In this configuration, the feedback wire from the ALSA Sink module to the ALSA Sink SRC now contains 2 elements:

Wire[0] - isStarted. Boolean

Wire[1] - ALSA sample rate.

If the ALSA device has not started streaming, then both values will be zero.

ALSA Sink SRC Module Arguments

This argument specifies the block size at the output of the ALSA Sink SRC module and should be sized based on the highest sample rate that the ALSA Sink module will output.

ALSA Sink SRC Inspector

ALSA Device State - Read back variable. Indicates whether the device is active, or not.

ALSA Sample Rate - Read back variable. This is the sample rate of the ALSA stream and configures the output sample rate of the ASRC algorithm. This information is communicated via the 2 element timing information (TI) pin.

InterpolationMethod - User settable parameter. Specifies whether the device implements Linear or Cubic interpolation. Currently, this does nothing and only linear interpolation is used.

inputAvail - Read back variable. Number of samples at the ALSA sample rate that can be written into the ASRC’s jitter buffer.

outputAvail - Read back variable. Number of samples at the Audio Weaver sample rate that can be read from the ASRC based on the level of the jitter buffer.

underRun - Read back variable. Number of underruns when attempting to read data out of the ASRC. If you get underruns, then the HLOS is not writing data fast enough.

overRun - Read back variable. Number of overruns. An overrun indicates that the jitter buffer in the ASRC is full. This should never happen for ALSA Source devices.

reset - User settable parameter. Click “reset” to set the errorStatus and all counters back to zero.

ALSA Tips and Tricks

Using Internal Variables

The ALSA modules have internal variables which are useful for controlling audio processing in your signal flow. You can use ParamGet modules to read these variables and put them onto control wires. We list the useful variables:

deviceState - int32_t. This is the “ALSA Device State” variable shown on the inspector. Possible values are: 1 = The DSP has initialized the device but it has not yet been opened by the HLOS. 2 = The HLOS has opened the device but it has not been started. 3 = The device is started and data is streaming.

isStreaming - int32_t. The is the “Is Started” variable on the inspector. This indicates if the device has been started via the inspector.

ALSA Troubleshooting

This section describes how to properly configure the ALSA devices to prevent audio dropouts. If you get dropouts, then this is because the client HLOS applications are not writing or reading data fast enough. Different techniques are needed for ALSA Source and ALSA Sink modules.

ALSA Source Underruns

Suppose that the HLOS opens the device with a 5 msec block size and a buffer size of 10 msec, and that Audio Weaver is configured to read data every 1 msec⁶. (We are ignoring asynchronous devices here, but the same explanation holds.) The ALSA Source module will output zeros after the device has been opened and the HLOS has not yet written data. This does not count as an underrun. The HLOS writes the first 5 msec block of data. The ALSA Source module sees the new data and then starts streaming data. It reads data out of the circular transfer buffer 1 msec at a time. The HLOS needs to write the next block of data within 5 msec or an underflow will occur.

Sometimes the HLOS needs more time to write data. This is where the startLevel of the ALSA device comes into play. Suppose that the HLOS application opens the ALSA device with a buffer size of 40 msec and a startThreshold of 20 msec. The HLOS application then writes 4 blocks of data. The circular transfer buffer now holds 20 msec of data and the startLevel has been crossed. At this point, the ALSA Source module stops writing zeros and starts streaming data. Since there is 20 msec of data (the startLevel) in the buffer, the HLOS has more time to refill and write the next block.

Of course, the startLevel increases the latency, but often this is needed to eliminate underruns. These are recommended minimum settings for ALSA Source devices used with Linux and Android operating systems:

blockSize 10 msec

bufferSize 40 msec (4 x blockSize)

startThreshold 20 msec (2 blockSize)

stopThreshold TBD msec

QNX is more deterministic, which allows you to reduce the block size and reduce latency:

blockSize 5 msec

bufferSize 20 msec (4 x blockSize)

startThreshold 10 msec (2 blockSize)

stopThreshold TBD msec

ALSA Sink Overruns

In a similar fashion, it is possible to get overruns if the HLOS does not read data quickly enough from the ALSA Sink device. To solve this problem, you have to increase the ALSA buffer size, and thereby increase the size of the circular transfer buffer. Adjusting the startThreshold and stopThreshold does not help.

Sample Rate Conversion Overhead

The ALSA Source and Sink SRC modules only do sample rate conversion if there is a mismatch between the ALSA sample rate and the Audio Weaver sample rate. In Release R4.0, the cost of 44.1 to 48 kHz sample rate conversion is about 30 MHz per channel. The cost increases linearly with the output sample rate. For example, converting 44.1 to 96 kHz costs about 60 MHz per channel.

vmMask

This is a module argument to the ALSA Source and Sink devices. It is a bit mask, and allows you to restrict which operating systems have access to specific ALSA source and sink devices. For example, you may have an ALSA device dedicated to safety chimes, and only want applications running on the PVM to stream data to this device.

The vmMask bitfield maps to specific operating system as follows.

ALSA Error Status Values

USB Audio [R5.x]

Ethernet AVB [R5.x]

Snapdragon Specific Modules

Audio Weaver includes a set of Snapdragon specific audio modules. These modules provide additional control and capabilities. These modules are found in Audio Weaver in the “Snapdragon” folder of the Module Browser.

We also describe how the Audio Weaver Event module operates in the context of a Snapdragon system. Each module is described in turn.

QXDM Logging

This module works in conjunction with the Qualcomm diagnostics and logging APIs. It allows you to stream real-time data from the Audio Weaver signal flow to the Qualcomm QXDM logging APIs. Once the data has been captured by QXDM, it can be parsed by the Qualcomm “QCAT” tool to generate WAV files.

The module has a single input pin and accepts fract32 data. It supports any number of channels and any block size. The module works on any Hexagon DSP and you can have multiple instances of the module throughout your design.

Module Arguments:

logCode. This is a UINT16 that is entered as a hexadecimal string (e.g., 0x1586). It is used together with logTapID to uniquely identify this module. There are a fixed number of logCode’s available and you select from a droplist:

If you are logging PCM data, then it doesn’t matter which logCode you use; it only matters that you pick the matching one in the QXDM application.

logTapID. This is a UINT16 that is entered as a hexadecimal string (e.g., 0x1234). It is used together with logCode to uniquely identify this module.

format. An integer which specifies the format of the data. Allowable values are: 0=PCM (the default), 1=Bitstream, and 2=Raw. The format flag is used by the Qualcomm tools to interpret the data. In Audio Weaver, you should stay with PCM (the default).

logBehavior. An integer which specifies in which thread the logging occurs. 0=deferred. 1=immediate (the default). If immediate, then the QXDM calls happen from the module’s real-time thread. If deferred, then several blocks are buffered up and then the calls to QXDM happen from a non-real-time thread.

deferredBufferSize. An integer which specifies the size of the circular buffer used for deferred logging. This specifies the number of blocks of data to store in the circular buffer.

transpose. Specifies if the inspector is drawn horizontally (the default) or vertically

logCode and logTapID are used to generate the file names by the QCAT utility.

The module can operate in “Immediate” mode which causes the QXDM functions to be called directly from the module’s real-time thread. Data is logged one block at a time. In “Deferred” mode, the data is added to a circular buffer. When the circular buffer is half full, then the module’s Set() function is called using Audio Weaver’s deferred functionality. The Set() function repeatedly calls the QXDM function to log the data. Note that if the number of blocks in the deferred buffer is large, then you may overwhelm the QXDM subsystem. We recommend keeping the deferredBufferSize between 6 and 12 blocks.

Inspector

The module’s inspector is shown below when there are 6 channels being logged. This is the default horizontal orientation.

The controls are as follows:

Master Enable - A single Boolean which enables or disables the logging. By default, this is TRUE.

Chan N Enable - This is an array of Boolean values, one per channel. This allows you to log a subset of the channels. By default, channelEnable is all ones and all channels will be logged.

Blocks Logged - this is a counter which increments every time a block of data is logged.

Overrun Count - This increments anytime that Audio Weaver is unable to log a block of data. This may be because the QXDM API returns an error, or the buffer of deferred data is full.

Reset Counts - This is a momentary switch which resets Blocks Logged and Overrun Count.

Example

This example streams 2 channels of audio data to QXDM. The first channel is a sine wave and the second is pink noise.

The QXDM module has the default settings and streams to logCode 0x1586 and logTapID 0x0000.

Next, start the Qualcomm “QUTS Status App” and connect to your target device

Then launch QXDM. Go to the Tools menu and select “CFG File Generator”. On the right side of the dialog, search for the logID that you selected. If you do not select this, then the QXDM module will not log packets and you will get overruns.

Click “OK” to dismiss the window. At this point, data will be logged by QXDM. Build and run the system in Designer. The inspector will display the number of Blocks Logged:

The number of blocks logged should match the pump rate of your module. In this case, it updates 1000 times per second.

Return to QXDM to export your data. Right click and select “Copy All Items to File…” This saves an HDF file to disk.

Next, using the QCAT utility to convert the HDF file to WAV data. Start by opening the HDF file (File→Open).

Then on the View menu select “Vocoder Playback”. Select “Replace Dropped Frames”. This option will use the time stamps of the data packets to reconstruct the WAV data. If there are missing frames (because of dropped QXDM packets, or because you disabled recording or certain channels) then zeros will be inserted.

Click “Process” in the upper left corner to generate the WAV files. The QCAT utility generates 3 files per recorded channel: labels, raw data, and WAV data. In this example, it generates the files:

Interpreting the last row:

0x1586 = logCode

0x0 = session ID

0x1 = tapID

0x2 = channel number

Tx = direction. All from modem perspective. Tx = upstream. Rx = to loudspeaker

48 kHz = sample rate

Q31 = numeric format

0 = ???

The file names have the logCode, logTapID, and channel name embedded. Opening the WAV files in Audacity, we see:

The gap around 11 second was because the “masterEnable” inspector button was toggled. The gap in channel 2 at 15 seconds was because the channelEnable” was toggled. If you do not select “Replace Dropped Frames” in QCAT, then the files will be shorter (no zero periods) and discontinuous.

Troubleshooting

Overruns are occurring

Did you remember to enable your logID in QXDM?

You may be overwhelming the QXDM subsystem and need to reduce the amount of data being logged. Reduce the number of channels or the sample rate of the data collected.

The WAV data is discontinuous

It is possible that no overruns are reported and yet the collected WAV data is discontinuous. When data is passed from Audio Weaver to QXDM, there is an indication that the data was successfully received by QXDM (if not, then Overrun Count increments). There is a second step where QXDM sends the data to the PC. If this fails, there is no indication back to Audio Weaver. Audio Weaver delivered the data to QXDM but QXDM had trouble logging it to the PC. As above, you’ll need to reduce the data rate.

Clock Settings

This module is part of R4.0 and will be replaced by the Clock Voting module in R4.1

This module allows you to change the Hexagon DSP’s clock speed and the speed of its DDR memory access. The module has no input or output pins, and can be placed in any Hexagon thread. The module does not do audio processing per se, but its Set function sets the internal Hexagon state. The module can be placed into any Hexagon thread (it doesn’t matter what thread it runs in).

When the Hexagon DSP first boots, it uses the clock speed settings from the awe_config.xml file. Next, when the Audio Weaver signal flow is loaded (the AWB), if it contains a Clock Settings module, then this module will override the initial settings. Finally, this module can be used to adjust the Hexagon clock during run-time. Adjustments can be made using the inspector or with a ParamSet module.

If your design needs a fixed clock speed, then just update the awe_config.xml file. Only use this module if you need to dynamically adjust the clock speed.

Clock Voting

This module allows you to change the Hexagon DSP’s clock speed and the speed of its DDR memory access. The module has no input or output pins, and can be placed in any Hexagon thread. The module does not do audio processing per se, but its Set function sets the internal Hexagon state. The module’s Get function reads back the actual clock settings. This allows you to verify the actual clock settings without using SysMon. The module can be placed into any Hexagon thread (it doesn’t matter what thread it runs in).

When the Hexagon DSP first boots, it uses the clock speed settings from the awe_config.xml file. Next, when the Audio Weaver signal flow is loaded (the AWB), if it contains a Clock Settings module, then this module will override the initial settings. Finally, this module can be used to adjust the Hexagon clock during run-time. Adjustments can be made using the inspector or with a ParamSet module.

If your design needs a fixed clock speed, then just update the awe_config.xml file. Only use this module if you need to dynamically adjust the clock speed.

Shared Memory Mapper

This module is used to map shared memory between a client application running in the HLOS and an audio module running in the Audio Weaver signal flow. This module is not used by any of the supplied Audio Weaver modules, but is provided to allow customers to develop their own custom modules that leverage shared memory. The module appears in Designer as shown below.

The module has a single module argument which allows you to set the sample rate of the module’s output wire.

The module has an internal 3 element array “regionInfo” which is used to exchange memory region information between the HLOS and the audio processing cores. This array is initially all zeros when the system is built. During operation, the client application requests memory from the Qualcomm Audio Service (QAS). The application specifies:

• Size in bytes of the allocation

• Name of the Shared Memory Mapper module in their signal flow to share the memory width

The QAS performs the following operations:

1. The QAS allocates the memory with the correct sharing attributes

2. The 64-bit physical address and size of the region is written to the module’s regionInfo array via TinyMix commands. This is a 3 element array consisting of 32-bit values:

regionInfo[0] → Low 32-bits of the 64-bit physical address

regionInfo[1] → High 32-bits of the 64-bit physical address

regionInfo[2] → Size of the memory region, in bytes

1. The module’s set function converts the physical address to a virtual address usable by the audio processing core (Hexagon or Arm).

2. The module then outputs the following 3 values on its output wire:

Wire[0] = Low 32-bits of the virtual address

Wire[1] = High 32-bits of the virtual address

Wire[2] = size of the shared region, in bytes

1. The Set function waits pumpDelay pump cycles before returning to the QAS.

2. The QAS then returns the SMMU mapped address to the client and also a handle for deallocating the region later.

After regionInfo is written and successfully converted, the inspector will update:

• “Is Initialized” becomes 1

• Addr Low shows the low 32-bits of the virtual address

• Addr High shows the high 32-bits of the virtual address (On the Hexagon the high 32-bits will always be zero)

• Region Size - size of the shared memory region, in bytes.

We expect that custom modules will use this information to read and write the shared memory. If the size of the region is zero, then this indicates that the memory has not been allocated, or that it has subsequently been deallocated.

When the client application is finished with the shared memory, it should stop accessing the memory and call the TBD QAS deallocation function. This function:

1. Writes zeros to the named Shared Memory Mapper module’s regionInfo array.

2. The module’s Set function sets the output wire information to 0 and then waits pumpDelay pump cycles.

3. Next it unmaps the memory region

4. Then returns to the QAS, and the QAS returns to the client application.

The pump delay is provided to allow any downstream modules to have sufficient time to stop accessing the memory region.

TBD. Instead of basing the delay on the number of pumps, should we have just used a sleep() command? As designed, the mapping and unmapping only works when the real-time audio is pumping.

Remove the pumpDelay. Just let the client application signal all of the modules that are using the mapped memory to stop using it.

Event Module

This module is part of the standard Audio Weaver distribution and is useful for SOC integration like the Snapdragon. The module generates asynchronous events that originate in the Audio Weaver signal flow and need to be routed to a client application running on the HLOS. Client applications subscribe to the events through the TinyMix API. In addition to originating in a module, events can also originate in the BSP itself and be routed to the HLOS.

Typical uses are:

• Sending a trigger word notification to the HLOS

• Have some data reported recurrently, e.g. VAD or RMS measurements.

The BSP will use the same event propagation mechanism to convey internal information, like:

• AWB has been successfully loaded on an instance

• AWE core is pumping now (i.e. an SSR is ready)

• An audio overrun has happened in the system (real-time performance impacted)

• When peak CPU load of DSP exceeds 90%

This module was recently added to Audio Weaver in release D-8.D.2.6. An example of how the Event Module would be used to send a wake word event notification is shown below. The first input pin “Trigger” contains Boolean data and is used to trigger the event. The second input pin specifies the data payload to send back to the HLOS. The payload can contain up to 64 words of data, or 256 bytes. In the example below, the payload for the Sensory wake word is which command was detected.

The system designer can place an arbitrary number of Event modules in the signal flow and they can be placed on any audio processing core.

Module Arguments

The Event module has a configurable eventType argument which is sent at design time. This eventType is a signed 32-bit integer which can be arbitrarily set by the system designer. The eventType is returned to the HLOS as part of the event notification.

There are no predefined event types and it is up to the system designer to define them. For example, you may have 4 wake word engines in the signal flow, one per seat location. Each wake word engine would have a corresponding Event module. The same eventType would be used for all of these wake word events. The payload would differentiate the seat location.

Module Inspector

The module inspector is shown below.

This provides further control of the module

Trigger Behavior - specifies when the event notification occurs. The default is “Deferred” which means that the notification occurs in Audio Weaver’s deferred thread. This causes the notification to occur in Audio Weaver’s non-real-time thread. The other option is “Instance” which causes the notification to occur in the real-time thread.

Trigger Type - specifies how the data on the trigger pin input is used:

Rising Edge - trigger when the input goes from 0 to 1 (or larger)

Falling Edge - trigger when the input goes from 1 (or larger) to 0

High - trigger any time the input is 1 or larger

Low - trigger any time the input equals 0

None - never trigger. This can be used to disable the event generation.

Callback Registered - Boolean readback variable that shows 1 if the Event module properly registered itself with the BSP. If this is 0, then the BSP hasn’t registered the event callbacks.

Successful Trigger Count - counter that increments whenever the BSP’s trigger callback successfully executes. On the Snapdragon, this means that the event structure was successfully added to the event queue in shared memory.

Failed Trigger Count - counter that increments whenever the BSP’s trigger callback returns an error. On the Snapdragon, this means that the event structure could not be added to the event queue in shared memory. It means your event queue is too small and you should adjust its size in your BSP configuration file.

Reset Counts - this is a push button which sets the two trigger counters to zero.

Receiving Events by Client Applications

In order for events to be visible by TinyMix, you have to assign an objectID to each event module. Then you use TinyMix as described in TBD to register and receive event callbacks.

System Variable

The SystemVariable module is a standard part of Audio Weaver. We mention it here because it is very useful for system bring up and debugging. It can also be used for detecting CPU overruns.

The module has an output pin with 1 channel and a block size of 1 (it is a control signal). The output wire will hold a single floating-point value. The module’s inspector allows you to choose which of 12 system variables to output:

This module returns internal information from the AWECore library and the layout (thread) that the module is running in. The following items can be selected:

SampleRate - this is the “Sample rate” field taken from the target information. It corresponds to the sample rate of the hardware input and output pins. In units of Hz.

ProfileClockSpeed - this is the “Profile clock rate” field taken from the target information. It corresponds to the speed of the profiling clock used for module profiling. In units of Hz. On the Hexagon, this is 19.2 MHz; on Linux, this is 10 MHz.

BlockSize - this is the “Basic block size” field taken from the target information. It corresponds to the fundamental block size of the BSP code and usually corresponds to the DMA buffer size. In units of samples.

CoreClockSpeed- this is the “CPU clock rate” field taken from the target information. It corresponds to the overall processor speed. In units of Hz.

AverageTime - this is the number of seconds required to complete the processing of the layout containing this module. This information is averaged at run-time with a first order smoother. In a single threaded system, this information corresponds to the “Average ticks per block” shown on the block-by-block profiling window.

PeakTime - this is the maximum number of seconds that was ever required to complete the processing of the layout containing this module. This information is tracked during real-time processing. In a single threaded system, this information corresponds to the “Peak ticks per block” shown on the block-by-block profiling window.

NumPumps - this is the number of times that this layout was pumped. It should increase linearly and the rate depends upon the clock divider of the hosting layout (thread).

CycleTime - this is the number of seconds between calls to the layout’s pump function. This information is averaged at run-time with a first order smoother. In a single threaded system, this information corresponds to the “Total ticks per block process available” shown on the block-by-block profiling window. For example, if you are processing audio at a 1 msec block size, then this value should be 0.001.

ElapsedTime - this is the total elapsed time in seconds for the layout containing this module. It equals (NumPumps x CycleTime).

PercentCPU - this is the percentage of CPU consumed by this layout. It is in the range of [0 100]. This is calculated per-frame. AverageTime/CycleTime approximates PercentCPU.

InstCycleTime - this is the same as CycleTime but is unsmoothed. It is the number of seconds between calls to this module’s process function. It is the instantaneous information for the previous pump cycle. No averaging occurs.

ResetCount - this is the number of times that the containing layout was reset. Reset occurs when there is an overrun in any audio processing thread throughout the system. This variable will be the same across all layouts. When debugging a system, you only need to watch for resets on a single thread and the ResetCount will reflect system-wide behavior.

You can combine the System Variable and Event modules as shown below to generate event notifications whenever the ResetCount variable changes.

Use Case Examples

We should include a full-featured block diagram which shows how to implement a complete automotive system in Audio Weaver. We could have placeholders for some functions, like RNC and Telephony, but it should be multicore and highlight best practices for designing systems.

Appendix A. System Configuration File

The R4.0 system uses an XML file to configure the Audio Weaver and the underlying BSP. This is converted to a binary file and loaded at system boot. An annotated version of the XML file is shown below.

<config platform="8755">

<awe_process>

<!--Unused. Will be removed in the future-->

<enable type="BOOL">true</enable>

</awe_process>

<params>

<log_level type="UINT32" max_level="2">

<!-- Specifies the log levels for each core. -->

<adsp type="UINT32">0</adsp>

<gpdsp0 type="UINT32">0</gpdsp0>

<gpdsp1 type="UINT32">0</gpdsp1>

<arm type="UINT32">0</arm>

</log_level>

<thread_num>

<!-- Number of child audio processing threads per core-->

<adsp type="UINT8">4</adsp>

<gpdsp0 type="UINT8">10</gpdsp0>

<gpdsp1 type="UINT8">10</gpdsp1>

<arm type="UINT8">4</arm>

</thread_num>

<adsp_thread_priority>

<!-- thread_parent is the priority of the main audio processing interrupt.

This is the highest priority audio thread. -->

<thread_parent type="UINT8">50</thread_parent>

<thread_child>

<!-- Next separate priorities for each child thread.

The number of child threads is specified above.

The child threads should be in decreasing priority from the parent_thread. -->

<thread_0 type="UINT8">51</thread_0>

<thread_1 type="UINT8">52</thread_1>

<thread_2 type="UINT8">53</thread_2>

<thread_3 type="UINT8">54</thread_3>

</thread_child>

</adsp_thread_priority>

<gpdsp0_thread_priority>

<thread_parent type="UINT8">50</thread_parent>

<thread_child>

<thread_0 type="UINT8">51</thread_0>

<thread_1 type="UINT8">52</thread_1>

<thread_2 type="UINT8">53</thread_2>

<thread_3 type="UINT8">54</thread_3>

<thread_4 type="UINT8">55</thread_4>

<thread_5 type="UINT8">56</thread_5>

<thread_6 type="UINT8">57</thread_6>

<thread_7 type="UINT8">58</thread_7>

<thread_8 type="UINT8">59</thread_8>

<thread_9 type="UINT8">60</thread_9>

</thread_child>

</gpdsp0_thread_priority>

<gpdsp1_thread_priority>

<thread_parent type="UINT8">50</thread_parent>

<thread_child>

<thread_0 type="UINT8">51</thread_0>

<thread_1 type="UINT8">52</thread_1>

<thread_2 type="UINT8">53</thread_2>

<thread_3 type="UINT8">54</thread_3>

<thread_4 type="UINT8">55</thread_4>

<thread_5 type="UINT8">56</thread_5>

<thread_6 type="UINT8">57</thread_6>

<thread_7 type="UINT8">58</thread_7>

<thread_8 type="UINT8">59</thread_8>

<thread_9 type="UINT8">60</thread_9>

</thread_child>

</gpdsp1_thread_priority>

<arm_thread_priority>

<thread_parent type="UINT8">63</thread_parent>

<thread_child>

<thread_0 type="UINT8">62</thread_0>

<thread_1 type="UINT8">61</thread_1>

<thread_2 type="UINT8">60</thread_2>

<thread_3 type="UINT8">59</thread_3>

</thread_child>

</arm_thread_priority>

<!-- Number of audio processing cores -->

<num_of_instances type="UINT32">4</num_of_instances>

<adsp>

<!-- Core ID (or instance ID) of the core. -->

<coreID type="UINT8">0</coreID>

<!-- Sizes of the Audio Weaver heaps, in units of 32-bit words. -->

<fastheapA type="UINT32">250000</fastheapA>

<fastheapB type="UINT32">250000</fastheapB>

<slowheap type="UINT32">250000</slowheap>

<cpuclock type="UINT32" unit="KHz">1344000</cpuclock>

</adsp>

<gpdsp0>

<coreID type="UINT8">1</coreID>

<fastheapA type="UINT32">250000</fastheapA>

<fastheapB type="UINT32">250000</fastheapB>

<slowheap type="UINT32">250000</slowheap>

<cpuclock type="UINT32" unit="KHz">1708800</cpuclock>

</gpdsp0>

<gpdsp1>

<coreID type="UINT8">2</coreID>

<fastheapA type="UINT32">250000</fastheapA>

<fastheapB type="UINT32">250000</fastheapB>

<slowheap type="UINT32">250000</slowheap>

<cpuclock type="UINT32" unit="KHz">1708800</cpuclock>

</gpdsp1>

<arm>

<coreID type="UINT8">3</coreID>

<fastheapA type="UINT32">7000000</fastheapA>

<fastheapB type="UINT32">70000</fastheapB>

<slowheap type="UINT32">70000</slowheap>

<cpuclock type="UINT32" unit="KHz">2100000</cpuclock>

</arm>

<!-- Size of the shared memory heap. This is visible to all cores.

In units of 32-bit words -->

<shared_heap_size type="UINT32" unit="UINT32">262000</shared_heap_size>

<!-- These values specify the "targetInfo" which is read back by the Audio Weaver

Server when it connects to the target. This will be removed in the future. -->

<block_size type="UINT16">48</block_size>

<sample_rate type="UINT16">48000</sample_rate>

<in_channel type="UINT16">16</in_channel>

<out_channel type="UINT16">16</out_channel>

</params>

</config>

Appendix B: Performance Benchmarks

This section contains some performance benchmarks for this release. During the measurements, processor clock and DDR speeds were set to maximum.

Interprocessor Communication

This is the measured MHz to transfer 16 channels of 32-bit data at 48 kHz between cores. Transfers were done with 1 msec blocks. This test uses the ChangeThread module and part of the reported MHz is on the sending core and part is on the receiving core.

When sending data between different cores, it is going through the carveout shared memory. When sending audio between the same core, this means that data is going to another hardware thread on the same core and non-shared is used.

TDM I/O

The limiting factor for serial port I/O is the speed of the LP DMA memory. This is 85 MB/second for reading and 170 MB/second for writing. These tests were with 48 sample block sizes.

TDM Source

16 channels @ 48 kHz. 16-bit samples: 28.3 MHz (theoretical limit of 24 MHz)

16 channels @ 48 kHz. 32-bit samples: 49 MHz

TDM Sink

16 channels @ 48 kHz. 16-bit samples: 14.8 MHz (theoretical limit of 12 MHz)

16 channels @ 48 kHz. 32-bit samples: 25.6 MHz

Input to Output Latency

This was measured by connecting the TDMSource directly to the TDMSink as shown below.

We measured the delay with an oscilloscope and the path included:

A/D → A2B → TDMSource → Copy → TDMSink → A2B → D/A

The latency varied based on the block size as shown below:

In R4.1 provide measures with Synchronous Unaligned Ports

ALSA I/O

This test measures the overhead of streaming audio data between the HLOS and Audio Weaver. 16 channels of data streamed at 48 kHz. 1 msec block size in Audio Weaver and a 10 msec block size at the HLOS.

ALSA Source SRC [R4.1]

HLOS sends data at 44.1 kHz and Audio Weaver converts to 48 kHz.

ALSA Sink SRC (R4.1)

Audio Weaver converts 48 kHz to 44.1 kHz for the HLOS.

TDM + ALSA Latency

In this test, we measure the latency from analog in to analog out including a round trip through the HLOS using ALSA modules. The path is:

A/D → A2B → TDMSource → ALSA Sink → HLOS → ALSA Source → TDMSink → A2B → D/A

An application was running on the HLOS which would read the ALSA Source and send it to the ALSA Sink. A 48 kHz sample rate was used throughout and Audio Weaver was processing at a 48 sample block size. The HLOS application was using a 240 sample block size. The ALSA settings used in the test were:

ALSA Sink

• Buffer size: 960 samples

• Block size: 240 samples

• startThreshold: 0 samples

• stopThreshold: 0 samples

ALSA Source

• Buffer size: 960 samples

• Block size: 240 samples

• startThreshold: 480 samples (prefill to avoid underruns)

• stopThreshold: 0 samples

The measured latency was 13.2 msec. The breakdown is:

1 msec analog latency

2 msec TDM digital latency (using 1 msec block size)

10 msec ALSA latency (using 5 msec block size)

We then modified the test to include sample rate conversion between Audio Weaver and the HLOS. The measured latency was now:

Maximum CPU Loading

In this test, we used the BiquadLoading module to load each thread in the system. We measured how many Biquad stages we could run before we started having CPU overruns. We used a 1 msec block size on the Hexagon DSPs and a 10 msec block size on the Arm. This test measures code and framework efficiency.

ADSP

GPDSP0

GPDSP1

Arm (only loaded a single thread)

Early Audio KPIs

In release R4.0, we are not able to fully measure this KPI. However, we were able to measure the Audio Weaver contribution to the boot time. We instrumented the code and measured the time that the main() function was reached on the ADSP until the time that real-time audio interrupts started.

For the measurement, we used the file “SA8255_Early_Audio_Parallel_with_Load.awd”. This contains 1320 modules spread across all four cores. It also had a TDM input and TDM output port, and would generate audio on the A2B output. There was a signal generator on each core (sine wave or noise). The signal flow was designed so that you could distinguish the sound of each core and verify that each core was properly running just by listening. The top-level is simple and there were additional subsystems that would load up each of the cores.

All Hexagon DSPs are part of the early audio group while the Arm booted later. The combined AWB file was 335,760 bytes long and this was split into 4 separate AWBs, one per core:

SA8255_Early_Audio_Parallel_with_Load_0.awb [ADSP. 84,644 bytes]

SA8255_Early_Audio_Parallel_with_Load_1.awb [GPDSP0. 83,656 bytes]

SA8255_Early_Audio_Parallel_with_Load_2.awb [GPDSP1. 83,824 bytes]

SA8255_Early_Audio_Parallel_with_Load_3.awb [Arm. 83,680 bytes]

The DSPs log information when they boot. We observed:

00:00:43.617500 [awe_bsp.c 1140] AWE ADSP:awe cfg is ready!

00:00:43.635000 [awe_bsp.c 1417] AWE ADSP:The first time to pump audio

00:00:43.617500 [awe_bsp.c 1140] AWE GPDSP0:awe cfg is ready!

00:00:43.638750 [awe_bsp.c 1417] AWE GPDSP0:The first time to pump audio

00:00:43.617500 [awe_bsp.c 1140] AWE GPDSP1:awe cfg is ready!

00:00:43.645000 [awe_bsp.c 1417] AWE GPDSP1:The first time to pump audio

The time from the ADSP booting until it is ready to generate audio is 17.5 msec

The time from the ADSP booting until GPDSP0 and GPDSP1 are ready to generate audio is 27.5 msec.

A2B Board Setup

The board shown below is the “Rev D” version. There are 4 smaller jumper switches and they need to be set as shown:

¹ Unless, of course, you want to write your own custom modules that execute on the Hexagon.

⁵ This is how varying sample rates are handled in Audio Weaver. The buffers are oversized for the worst case transfer size and side information - in the timing information pins - is used to regulate the flow.

⁶ The discussion here is based on block times which are easier to follow. The actual configuration is based on block sizes.

^{4 This is a new feature which was recently implemented in Audio Weaver. If your design has custom modules, then you may need to add a custom resetState function.}

² The objectID of this SourceInt module is specified in the system configuration file.

^{3 You will also need a Matlab license. We recommend Matlab 2022b.}