Android

In the modern world, vehicles are no longer just modes of transportation; they have transformed into mobile entertainment hubs and communication centers. The integration of advanced audio systems in vehicles has revolutionized the driving experience, providing drivers and passengers with a seamless blend of music, navigation guidance, voice commands, and much more. However, what makes the audio in vehicles truly special goes beyond just the melodies and beats. In this blog, we delve into the intricacies of automotive audio systems, exploring the unique features that make them stand out.

What is special about audio in vehicles?

Automotive Audio is a feature of Android Automotive OS (AAOS) that allows vehicles to play infotainment sounds, such as media, navigation, and communications. AAOS is not responsible for chimes and warnings that have strict availability and timing requirements, as these sounds are typically handled by the vehicle’s hardware.

Here are some of the things that are special about audio in vehicles:

Many audio channels with special behaviors

In a vehicle, there can be many different audio channels, each with its own unique purpose. For example, there may be a channel for music, a channel for navigation instructions, a channel for phone calls, and a channel for warning sounds. Each of these channels needs to behave in a specific way in order to be effective. For example, the music channel should not be interrupted by the navigation instructions, and the warning sounds should be audible over all other channels.

Critical chimes and warning sounds

In a vehicle, it is important to be able to hear critical chimes and warning sounds clearly, even over loud music or other noise. This is why these sounds are often played through a separate set of speakers, or through the speakers at a higher volume.

Interactions between audio channels

The audio channels in a vehicle can interact with each other in a variety of ways. For example, the music channel may be muted when the navigation instructions are spoken, or the warning sounds may override all other channels. These interactions need to be carefully designed in order to ensure that the audio system is safe and effective.

Lots of speakers

In order to provide good sound quality in a vehicle, there are often many speakers installed. This is because the sound waves need to be able to reach all parts of the vehicle, even if the driver and passengers are not sitting directly in front of the speakers.

In addition to these special features, audio in vehicles is also subject to a number of challenges, such as:

Noise

There is often a lot of noise in a vehicle, from the engine, the road, and the wind. This noise can make it difficult to hear the audio system, especially the critical chimes and warning sounds.

Vibration

The vehicle can vibrate, which can also make it difficult to hear the audio system.

Temperature

The temperature in a vehicle can vary greatly, from very hot to very cold. This can also affect the performance of the audio system.

Despite these challenges, audio in vehicles is an important safety feature and can also be a great way to enjoy music and entertainment while driving.

Automotive Sounds and Streams

The world of automotive sounds and streams is a testament to the intersection of technology, design, and human experience. The symphony of sounds within a vehicle, coupled with the seamless integration of streaming services, creates a holistic journey that engages our senses and transforms the act of driving into an unforgettable adventure

In car audio systems using Android, different sounds and streams are managed:

Logical Streams

Logical streams are the streams of audio data that are generated by Android apps. These streams are tagged with AudioAttributes, which provide details like where they come from, and information about the type of audio, such as its importance, latency requirements, and desired output devices.

Physical Streams

Physical streams are the streams of audio data that are output by the vehicle’s audio hardware. These are the actual sounds that come out of the speakers. These streams are not tagged with AudioAttributes, as they are not controlled by Android. They are made by mixing logical streams together. Some sounds, like important warnings, are managed separately from Android.

The main difference between logical streams and physical streams is that logical streams are controlled by Android, while physical streams are not. This means that Android can control the volume, routing, and focus of logical streams, but it cannot control the volume, routing, or focus of physical streams.

Android App Sounds

Apps make sounds, like music or navigation. These sounds are sent to a mixer and then to the speakers. The mixer combines different sounds and makes them into one.

External Sounds

External sounds are sounds that are generated by sources other than Android apps, such as seatbelt warning chimes. These sounds are managed outside of Android and are not subject to the same audio policies as Android sounds. Some sounds shouldn’t go through Android, so they go directly to the mixer. The mixer can ask Android to pause other sounds when these important sounds play.

External sounds are typically managed outside of Android because they have strict timing requirements or because they are safety-critical. For example, a seatbelt warning chime must be played immediately when the seatbelt is not buckled, and it must be audible over any other sounds that are playing. This is why external sounds are typically handled by the vehicle’s hardware, rather than by Android software.

Contexts

Contexts are used to identify the purpose of the audio data. This information is used by the system to determine how to present the audio, such as the volume level, the priority, and whether or not it should be interrupted by other sounds.

Buses

Buses are logical groups of physical streams that are routed to the same output device. This allows the system to mix multiple audio streams together before sending them to the speakers.

Audio Flinger

AudioFlinger is the system service that manages the audio output. It uses the context to mix logical streams down to physical streams called buses. This allows multiple logical streams to be mixed together, even if they are in different formats or have different priorities.

The IAudioControl::getBusForContext method maps from context to bus. This method is used by applications to get the bus that is associated with a particular context. This information can be used to route the audio output to the desired speakers.

For example, the NAVIGATION context could be routed to the driver’s side speakers. This would ensure that the navigation instructions are always audible, even if the music is playing.

The physical streams, contexts, and buses are an important part of the Android audio system. They allow the system to intelligently manage the audio output and ensure that the most important sounds are always audible.

Output Devices

Audio Flinger is like the conductor of an orchestra. It takes the different streams from each context and mixes them together into something called a “bus.” Think of a bus as a big container for mixed sounds.

In the Audio HAL (the part of the system that handles audio), there’s something called “AUDIO_DEVICE_OUT_BUS.” It’s like a general way to send sounds to the speakers in a car. The AUDIO_DEVICE_OUT_BUS device type is the only supported output device type in Android Automotive OS. This is because it allows for the most flexibility in terms of routing and mixing audio streams.

A system implementation can choose to use one bus port for all Android sounds, or it can use one bus port for each CarAudioContext. A CarAudioContext is a set of audio attributes that define the type of audio, such as its importance, latency requirements, and desired output devices.

If a system implementation uses one bus port for all Android sounds, then Android will mix everything together and deliver it as one stream. This is the simplest approach, but it may not be ideal for all use cases. For example, if you want to be able to play different sounds from different apps at the same time, then you will need to use one bus port for each CarAudioContext.

The assignment of audio contexts to output devices is done through the car_audio_configuration.xml file. This file is used to define the audio routing and mixing policies for the vehicle.

Microphone Input

When we want to record audio (like using a microphone), the Audio HAL gets a request called “openInputStream.” This request includes a way to process the microphone sound.

There’s a special type called “VOICE_RECOGNITION.” This is used for things like the Google Assistant. It needs sound from two microphones (stereo) and can cancel echoes. Other processing is done by the Assistant.

If there are more than two microphones, we use a special setting called “channel index mask.” This setting helps handle multiple microphones properly.

Here’s a simple example of how to set this up in code:

// Setting up the microphone format
AudioFormat audioFormat = new AudioFormat.Builder()
    .setEncoding(AudioFormat.ENCODING_PCM_16BIT)
    .setSampleRate(44100)
    .setChannelIndexMask(0xf /* 4 channels, 0..3 */)
    .build();

// Creating an AudioRecord object with the format
AudioRecord audioRecord = new AudioRecord.Builder()
    .setAudioFormat(audioFormat)
    .build();

// Choosing a specific microphone device (optional)
audioRecord.setPreferredDevice(someAudioDeviceInfo);

If both “setChannelMask” and “setChannelIndexMask” are used, then “setChannelMask” (maximum of two channels) wins.

Starting from Android 10, the Android system can record from different sources at the same time, but there are rules to protect privacy. Some sources, like FM radio, can be recorded along with regular sources like the microphone. Apps using specific devices like bus microphones need to tell the system which one to use explicitly.

Audio Context

Audio contexts are groups of audio usages that are used to simplify the configuration of audio in Android Automotive OS. Let’s first discuss audio usage

Audio Usage

In Android Automotive OS (AAOS), AudioAttributes.AttributeUsages are like labels for sounds. They help control where the sound goes, how loud it is, and who has control over it. Each sound or request for focus needs to have a specific usage defined. If no usage is set, it’s treated as a general media sound.

Android 11 introduced system usages, which are special labels that require specific permissions to use. These are:

1. USAGE_EMERGENCY
2. USAGE_SAFETY
3. USAGE_VEHICLE_STATUS
4. USAGE_ANNOUNCEMENT

To set a system usage, you use AudioAttributes.Builder#setSystemUsage. If you try to mix regular usage with system usage, it won’t work.

package com.softaai.automotive.audio


import android.media.AudioAttributes;

/**
 * Created by amoljp19 on 8/12/2023.
 * softAai Apps.
 */
public class AudioAttributesExample {

    public static void main(String[] args) {
        // Constructing AudioAttributes with system usage
        AudioAttributes.Builder attributesBuilder = new AudioAttributes.Builder()
                .setSystemUsage(AudioAttributes.USAGE_ALARM); // Set a system usage (alarm)

        // You can also set a general usage, but not both a system usage and a general usage
        // attributesBuilder.setUsage(AudioAttributes.USAGE_MEDIA); // Uncommenting this line would cause an error

        // Building the AudioAttributes instance
        AudioAttributes audioAttributes = attributesBuilder.build();

        // Checking the associated system usage or usage
        int systemUsage = audioAttributes.getSystemUsage();
        System.out.println("Associated System Usage: " + systemUsage);

    }
}

In this example:

1. We use AudioAttributes.Builder to create an instance of audio attributes.
2. We use setSystemUsage to specify a system context for the audio, in this case, an alarm usage.
3. Attempting to set both a system usage and a general usage using setUsage would result in an error, so that line is commented out.
4. We then build the AudioAttributes instance using attributesBuilder.build().
5. Finally, we use audioAttributes.getSystemUsage() to retrieve the associated system usage and print it.

Audio Context

Audio contexts are used in Android to identify the purpose of a sound. This information is used by the system to determine how to present the sound, such as the volume level, the priority, and whether or not it should be interrupted by other sounds.

The following are the audio contexts that are currently defined in Android:

1. MUSIC: This is for playing music in the vehicle, like your favorite songs.
2. NAVIGATION: These are the directions your vehicle’s navigation system gives you to help you find your way.
3. VOICE_COMMAND: When you talk to the vehicle, like telling it to change settings or do something for you.
4. CALL_RING: When someone is calling you, this is the ringing sound you hear.
5. CALL: This is for when you’re having a conversation with someone on the phone while in the vehicle.
6. ALARM: A loud sound that might go off if something needs your immediate attention.
7. NOTIFICATION: These are little messages or reminders from the vehicle’s systems.
8. SYSTEM_SOUND: The sounds you hear when you press buttons or interact with the vehicle’s controls.

The following table summarizes the mapping between audio contexts and usages in Android Automotive OS:

The audio context for a sound can be specified by the application that is playing the sound. This is done by setting the context property of the AudioAttributes object that is used to create the sound.

The system uses the audio context to determine how to present the sound. For example, the volume level of a sound may be higher for the MUSIC context than for the NOTIFICATION context. The system may also choose to interrupt a sound of a lower priority with a sound of a higher priority.

Audio contexts are an important part of the Android audio system. They allow the system to intelligently manage the audio output and ensure that the most important sounds are always audible.

Multi-zone Audio

In cars, multiple people might want to listen to different things at the same time. Multi-zone audio makes this possible. For instance, the driver could be playing music in the front while passengers watch a video in the back.

Starting from Android 10, car makers (OEMs) can set up separate audio zones in the vehicle. Each zone is like a specific area with its own volume control, sound settings, and ways to switch between different things playing.

Imagine the main cabin is one zone, and the screens and headphone jacks in the back are another zone.

This setup is done using a special file called “car_audio_configuration.xml.” A part of the car’s system reads this and decides how sounds should move between zones. When you start a music or video player, the system knows where to send the sound based on the zone and what you’re doing.

Each zone can focus on its own sounds, so even if two people are listening to different things, their sounds won’t interfere with each other. This makes sure everyone gets their own audio experience.

• Zones are collections of devices within the vehicle that are grouped together for audio routing and focus management. Each zone has its own volume groups, routing configuration for contexts, and focus management.
• The zones are defined in car_audio_configuration.xml. This file is used to define the audio routing and focus policies for the vehicle.
• When a player is created, CarAudioService determines for which zone the player is associated with. This is done based on the player’s uid and the audio context of the stream that it is playing.
• Focus is also maintained independently for each audio zone. This means that applications in different zones can independently produce audio without interfering with each other.
• CarZonesAudioFocus within CarAudioService is responsible for managing focus for each zone. This ensures that only one application can have an audio focus in a given zone at a time.

In a simpler way, multi-zone audio lets different parts of the car play different sounds at the same time, so everyone can enjoy what they want to hear.

Audio HAL

In automotive audio, the Android system uses something called the Audio HAL to manage audio devices. This helps control how sounds are sent to speakers and received from microphones.

Audio HAL Components:

1. IDevice.hal: Handles creating sound streams, controlling volume, and muting. It uses “createAudioPatch” to connect different devices for sound.
2. IStream.hal: Manages the actual streaming of audio to and from the hardware, both for input and output.

Automotive Device Types:

Here are some device types that matter for cars:

• AUDIO_DEVICE_OUT_BUS: Main output for all Android sounds in the car.
• AUDIO_DEVICE_OUT_TELEPHONY_TX: For sending audio to the phone for calls. Here “TX” stands for “transmit”. In general, TX refers to the device that is sending data.
• AUDIO_DEVICE_IN_BUS: Used for inputs that don’t fit other categories.
• AUDIO_DEVICE_IN_FM_TUNER: Only for radio input.
• AUDIO_DEVICE_IN_LINE: For things like AUX input.
• AUDIO_DEVICE_IN_BLUETOOTH_A2DP: For music from Bluetooth.
• AUDIO_DEVICE_IN_TELEPHONY_RX: For audio from phone calls. Here “RX” stands for “receive.” In general, RX refers to the device that is receiving data.

Configuring Audio Devices:

To make audio devices work with Android, they must be defined in a file called “audio_policy_configuration.xml”.

<audioPolicyConfiguration version="1.0" xmlns:xi="http://www.w3.org/2001/XInclude">
    <modules>
        <module name="primary" halVersion="3.0">
            <attachedDevices>
                <item>bus0_phone_out</item>
<defaultOutputDevice>bus0_phone_out</defaultOutputDevice>
            <mixPorts>
                <mixPort name="mixport_bus0_phone_out"
                         role="source"
                         flags="AUDIO_OUTPUT_FLAG_PRIMARY">
                    <profile name="" format="AUDIO_FORMAT_PCM_16_BIT"
                            samplingRates="48000"
                            channelMasks="AUDIO_CHANNEL_OUT_STEREO"/>
                </mixPort>
            </mixPorts>
            <devicePorts>
                <devicePort tagName="bus0_phone_out"
                            role="sink"
                            type="AUDIO_DEVICE_OUT_BUS"
                            address="BUS00_PHONE">
                    <profile name="" format="AUDIO_FORMAT_PCM_16_BIT"
                            samplingRates="48000"
                            channelMasks="AUDIO_CHANNEL_OUT_STEREO"/>
                    <gains>
                        <gain name="" mode="AUDIO_GAIN_MODE_JOINT"
                                minValueMB="-8400"
                                maxValueMB="4000"
                                defaultValueMB="0"
                                stepValueMB="100"/>
                    </gains>
                </devicePort>
            </devicePorts>
            <routes>
                <route type="mix" sink="bus0_phone_out"
                       sources="mixport_bus0_phone_out"/>
            </routes>
        </module>
    </modules>
</audioPolicyConfiguration>

This file has parts like:

• module name: It specifies the type of device, like “primary” for automotive.
• devicePorts: This is where you define different input and output devices with their settings.
• mixPorts: It lists the different streams for audio, like what’s coming from apps.
• routes: These are connections between devices and streams.

For example, you can define an output device called “bus0_phone_out” that mixes all Android sounds. You can also set the volume levels for it.

In simpler words, the Audio HAL helps manage how sounds come out of speakers and go into microphones in cars. Devices and settings are defined in a special file to make everything work correctly.

Chimes and warnings

Chimes and warnings within vehicles serve as auditory cues that communicate vital information to the driver and occupants. From seatbelt reminders to collision warnings, these sounds are designed to promptly draw attention to situations that require immediate action. These auditory cues enhance situational awareness and contribute to the overall safety of the driving experience.

Android’s Role in Automotive Audio

While Android has become a ubiquitous operating system for various devices, it presents certain considerations when it comes to automotive safety. Android, in its standard form, is not classified as a safety-critical operating system. Unlike dedicated safety-critical systems found in vehicles, Android’s primary focus is on delivering a versatile and user-friendly platform.

The Absence of an Early Audio Path

In the context of chimes and warnings, Android lacks an early audio path that is essential for producing regulatory and safety-related sounds. An early audio path would involve direct access to the audio hardware, ensuring that these crucial sounds are played promptly and without interruption. Android, being a multifunctional operating system, may not possess the mechanisms required for such instantaneous audio playback.

Regulatory Sounds Beyond Android

Given the critical nature of regulatory chimes and warnings, generating and delivering these sounds falls outside the Android operating system. To ensure that these sounds are reliable and timely, they are often generated and mixed independently from Android, later integrating into the vehicle’s overall audio output chain. This approach guarantees that regulatory sounds maintain their integrity, even in scenarios where Android might face limitations due to its primary focus on versatility.

Safety-Critical Considerations

The absence of an early audio path within Android highlights a broader concern related to the safety-critical nature of automotive audio. As vehicles continue to integrate advanced technologies, including infotainment systems and connectivity features, the challenge lies in finding the balance between innovation and safety. Regulatory bodies and automotive manufacturers collaborate to ensure that safety-critical elements, such as chimes and warnings, are given the utmost attention and reliability.

The Road Ahead: Safety and Technology Integration

The integration of technology, including operating systems like Android, into vehicles is a testament to the dynamic evolution of the automotive landscape. As the industry continues to innovate, addressing safety concerns remains paramount. The future promises advancements that bridge the gap between safety-critical needs and technological capabilities. This may involve further synchronization between Android and the vehicle’s safety systems, ensuring that critical alerts and warnings are delivered seamlessly and without compromise.

In short, the realm of chimes and warnings in automotive audio underscores the delicate balance between safety and technology. While Android contributes significantly to the modern driving experience, there are specific safety-critical aspects, such as regulatory sounds, that demand specialized attention. The collaborative efforts of regulatory bodies, automotive manufacturers, and technology providers will continue to shape a safer and more immersive driving journey for all.

Conclusion

The audio systems in modern vehicles have evolved far beyond their humble beginnings as simple radios. They have become intricate orchestras, harmonizing various audio contexts to provide an engaging and safe driving experience. The integration of multiple audio channels, critical warning sounds, seamless context interactions, and an abundance of speakers all contribute to the unique symphony that accompanies us on our journeys. As technology continues to advance, we can only anticipate further innovations that will elevate the in-car audio experience to new heights.