First drop of audio framework modifications for audio effects support.
- AudioTrack/AudioRecord:
Added support for auxiliary effects in AudioTrack
Added support for audio sessions
Fixed left right channel inversion in setVolume()
- IAudioFlinger:
Added interface methods for effect enumeraiton and instantiation
Added support for audio sessions.
- IAudioTrack:
Added method to attach auxiliary effect.
- AudioFlinger
Created new classes to control effect engines in effect library and manage effect connections to tracks or
output mix:
EffectModule: wrapper object controlling the effect engine implementation in the effect library. There
is one EffectModule per instance of an effect in a given audio session
EffectChain: group of effects associated to one audio session. There is one EffectChain per audio session.
EffectChain for session 0 is for output mix effects, other chains are attached to audio tracks
with same session ID. Each chain contains a variable number of EffectModules
EffectHandle: implements the IEffect interface. There is one EffectHandle object for each application
controlling (or using) an effect module. THe EffectModule maintians a list of EffectHandles.
Added support for effect modules and effect chains creation in PlaybackThread.
modified mixer thread loop to allow track volume control by effect modules and call effect processing.
-AudioMixer
Each track now specifies its output buffer used by mixer for accumulation
Modified mixer process functions to process tracks by groups of tracks with same buffer
Modified track process functions to support accumulation to auxiliary channel
Change-Id: I26d5f7c9e070a89bdd383e1a659f8b7ca150379c
The problem is due to a too big difference between the buffer size used at the hardware interface and at the A2DP interface.
When no resampling occurs we don't notice problems but the timing is very tight. As soon as resampling is activated, the AudioTrack underruns.
This is because the AudioTrack buffers are not resized when moving the AudioTrack from hardware to A2DP output.
The AudioTrack buffers are calculated based on a hardware output buffer size of 3072 bytes. Which is much less than the A2DP output buffer size (10240).
The solution consists in creating new tracks with new buffers in AudioFlinger when the A2DP output is opened
instead of just transfering active tracks from hardware output mixer thread to the new A2DP output mixer thread.
To avoid synchronization issues between mixer threads and client processes, this is done by invalidating tracks
by setting a flag in their control block and having AudioTrack release the handle on this track (IAudioTrack)
and create a new IAudioTrack when this flag is detected next time obtainBuffer() or start() is executed.
AudioFlinger modifications:
- invalidate the tracks when setStreamOutput() is called
- make sure that notifications of output opening/closing and change of stream type to output mapping are sent synchronously to client process.
This is necessary so that AudioSystem has the new stream to output mapping when the AudioTrack detects the invalidate flag in the client process.
Previously their were sent when the corresponding thread loop was executed.
AudioTrack modifications:
- move frame count calculation and verification from set() to createTrack() so that is is updated every time a new IAudioTrack is created.
- detect track invalidate flag in obtainBuffer() and start() and create a new IAudioTrack.
AudioTrackShared modifications
- group all flags (out, flowControlFlag, forceReady...) into a single bit filed to save space.
Change-Id: I9ac26b6192230627d35084e1449640caaf7d56ee
Some variables and structure members should be renamed to reflect the fact that they contain the
number of channels in a track (channel count) or the actual channels used by a track (channel mask).
Especially member "channels" of track control block (struct audio_track_cblk_t) is actually the
number of channels (channels count).
Change-Id: I220c8dede9fc00c8a5693389e790073b6ed307b8
There is a bug in the way notification client list is managed when the client binder
interface dies that makes that the dead client is not removed from the list: the week
reference passed by binderDied() cannot be promoted and compared to the strong
references in the list.
The fix consists in creating a new NotificationClient class that implements the
binder DeathRecipient and holds a strong reference to the IAudioFlingerClient interface.
A new instance of this class is created for each cient and a strong reference to this
object is added to the notification client list maintained by AudioFlinger.
When binderDied() is called on this object, it is removed from the list preventing
AudioFlinger to notify this client for further io changes.
Also added code to disable LifeVibes effects when the client that has enabled the
enhancements dies.
Change-Id: Icedc4af171759e9ae9a575d82d44784b4e8267e8
This changes fixes the issue for the direct output thread that was not
addressed by commit 71f37cd8a175ee00635cb91506d6810fd02b5b51.
Change-Id: I1bbe26be5f444415dd97270e49257650f5d2858f
The problem is a bug in AudioFlinger::MixerThread::prepareTracks_l() that makes that even if the TrackHandle
is destroyed, the corresponding Track will remain active as long as frames are ready for mixing.
If the track uses shared memory (static mode) and the sound is looped, this track will play for ever.
The fix consists in removing the track from active list immediately if the track is terminated.
Change-Id: I4582aa1d981079ab79be442fb6185f5afaed5cf3
- AudioPolicyManager: allow platform specific choice for opening a direct output.
Also fixed problems in direct output management.
- AudioFliinger: use shorter standby delay and track inactivity grace period for direct output
thread to free hardware resources as soon as possible.
- AudioSystem: do not use cached output selection in getOutput() when a direct output
can be selected.
Change-Id: If44b50d29237b8402ffd7a5ba1dc43c56f903e9b
The problem is that AudioRecord never exits read() when a timeout occurs while trying
to get new PCM data from audio hardware input buffer: it just keeps waiting and retrying until stop() is called.
In the same time, opencore AndroidAudioInput::audin_thread_func() loop cannot be exited when stuck
in AudioRecord::read() because the iExitAudioThread flag can only be sampled when AudioRecord::read()
returns. We remain stuck with the audio input thread running.
The fix consists in modifying AudioRecord behavior in case of timeout when getting new PCM samples.
We now wait only one timeout period and try to restart audio record, in case the problem is due to a media_server
process crash. If this fails, we exit read() with a number of bytes read equals to 0 so that
AndroidAudioInput::audin_thread_func() loop can exit.
Also modified Audioflinger::RecordThread() loop so that we attempt to recover from HAL read errors.
In case of read error, the input stream is forced to standby so that next read attempt does a
reconfiguration and restart of the audio input device.
At some point the implementation became complicated because of
SurfaceFlinger's special needs, since we are now relying on gralloc
we can go back to much simpler MemoryDealer.
Removed HeapInterface and AllocatorInterface, since those don't need
to be paramterized anymore. Merged SimpleMemory and Allocation.
Made SimplisticAllocator non virtual.
Removed MemoryDealer flags (READ_ONLY, PAGE_ALIGNED)
Removed a lot of unneeded code.
Added getRenderPosition() API to IAudioFlinger to retreive number of audio frames
written by AudioFlinger to audio HAL and by DSP to DAC.
Added getRenderPosition() API to AudioHardwareInterface to retreive number of audio frames
written by DSP to DAC.
Exposed AudioTrack::getPosition() to AudioSink() to make it available to media player.
Removed excessive log in AudioHardwareGeneric.
Modified AudioService.getActiveStreamType() so that STREAM_VOICE_CALL is selected when a track using this stream
type is playing.
Chanded isMusicActive() for a more generic isStreamActive(stream) method in AudioSystem, IAudioFlinger and AudioFlinger.
Modified AudioFlinger duplicating output thread so that audio tracks are not mixed until both outputs (A2DP and hardware) have exited standby mode. This avoids to have one output far ahead of the other and audio frames dropped because the compensation mechanism cannot keep up.
Also calculate the maximum wait time in OutputTrack::write() based the on smallest frame count of all output threads instead of the frame count of the thread the OutputTrack is connected to. This avoids starving the thread with the smallest frame count by waiting too long on the other thread.
Since the frame count was reduced on hardware output to reduce latency the difference between A2DP and hardware outputs frame counts had become problematic.
Also increased the number of overflow buffers to cope with bigger timing differences among outputs.
The ToneGenerator failed to initialize because no more tracks were available in AudioFlinger mixer.
All tracks were used because the duplicating output was failing to free the tracks on audio hardware output mixer when exiting due to a misplaced test on output activity: output tracks where only freed if the duplicating output was active when exiting.
The fix consists in freeing the output tracks when the duplicating thread is destroyed without condition.
Fixed AudioFlinger::openInput() broken in change ddb78e7753be03937ad57ce7c3c842c52bdad65e
so that an invalid IO handle (0) is returned in case of failure.
Applied the same correction to openOutput().
Modified RecordThread start procedure so that a failure occuring during the first read from audio input stream is detected and causes
the record start to fail.
Modified RecordThread stop procedure to make sure that audio input stream fd is closed before we exit the stop function.
Fixed AudioRecord JAVA and JNI implementation to take status of native AudioRecord::start() into account
and not change mRecordingState to RECORDSTATE_RECORDING if start fails.
The problem comes from a deadlock with AudioPolicyService mutex: When the second ringtone starts,
this mutex is locked by AudioPolicyService::startOutput() which in turn calls setParameters() to change the output device.
Audioflinger::ThreadBase::setParameters() signals the parameter change to the AudioFlinger mixer thread and waits for a condition
indicating that the parameter change has been processed.
At the same time, the mixer thread detects that the audio track corresponding to the first ring tone has been killed and calls its destructor.
This calls AudioPolicyService::releaseOutput() which tries to lock the AudioPolicyService mutex.
If this happens before the mixer thread can process the setParameters() command we are deadlocked.
The deadlock ends because setParameters() uses a timeout when waiting for the condition.
This regression was introduced by change 33736 fixing issue 2265163.
The fix consists in calling AudioPolicyService::releaseOutput() from Track::destroy() instead of from Track destructor: as detroy() is never called from the mixer thread loop (as opposed to the destructor) the deadlock described above cannot occur.
This is a second attempt to fix the audio routed to earpiece syndrom.
The root cause identified this time is the crash of an application having an active AudioTrack playing on the VOICE_CALL stream type.
When this happens, the AudioTrack destructor is not called and the audio policy manager is not notified of the track stop.
Results a situation where the VOICE_CALL stream is considered as always in use by audio policy manager which makes that audio is routed to earpiece.
The fix consists in moving the track start/stop/close notification to audio policiy manager from AudioTrack to AudioFlinger Track objet.
The net result is that in the case of a client application crash, the AudioFlinger TrackHandle object (which implements the remote side of the IAudioTrack binder interface) destructor is called which in turn destroys the Track object and we can notify the audio policy manager of the track stop and removal.
The same modification is made for AudioRecord although no bug related to record has been reported yet.
Also fixed a potential problem if record stop is called while the record thread is exiting.
This change goes with a kernel driver change that reduces the audio buffer size from 4800 bytes (~27ms) to 3072 bytes (~17ms).
- The AudioFlinger modifcations in change 0bca68cfff161abbc992fec82dc7c88079dd1a36 have been removed: the short sleep period was counter productive when the AudioTrack is using the call back thread as it causes to many preemptions.
- AudioFlinger mixer thread now detects long standby exit time and in this case anticipates start by writing 0s as soon as a track is enabled even if not ready for mixing.
- AudioTrack::start() is modified to start call back thread before starting the IAudioTrack so that thread startup time is masked by IAudioTrack start and mixer thread wakeup time.
Reduce sleep time in AudioFlinger mixer thread when no data has been written to output to speed up startup time when exiting standby.
The rest of the modifications for this issues is in kernel driver:
commit 0dbb0ee136ed8de757df1ae26d84556c1751deae for buffer size modification from 8192 to 4800 bytes.
Another kernel improvement that is not submitted yes will reduce delay when audio output is exiting standby.
Added setVoiceVolume() method to AudioSystem, AudioFlinger, IAudioFlinger, AudioPolicyService.
Removed call to AudioHardwareInterface::setVoiceVolume() from AudioFlinger::setStreamVolume().
This change is a complement to the main fix in kernel driver for the same issue (partner change #1250).
It removes clicks sometimes heard after the end of the tones while audio flinger is sending 0s to the audio output stream.
The problem was that the sleep time between two writes was more than the duration of one audio output stream buffer which could cause some underrun.
Also fixed a recent regression in ToneGenerator that made that the end of previous tone was repeated at the beginning of current one under certain timing circumstances when the maximum tone duration was specified.
Wait for the parameter set completed condition with a time out in ThreadBase::setParameters().
Also lock AudioFlinger mutex before accessing thread list in AudioFlinger::setParameters() and keep a strong reference
on the thread being used in case it is exited while processing the request.
There was a regression introduced in AudioFlinger by change 24114 for suspended output:
The suspended output was not reading and mixing audio tracks.
When the phone is ringing, the A2DP output is suspended if the SCO headset and A2DP headset are the same. As the ringtone is played over the duplicated output, the fact that the A2DP output was not reading data was causing the hardware output to be stalled from time to time.
The fix consists in locking AudioFlinger::mLock mutex in the TrackBase destructor before clearing the strong pointer to the shared memory client. The mutex is not locked in removeclient() any more which implies that we must make sure that the Client destructor is always called from the TrackBase destructor or that we hold the mLock mutex before calling deleting the Client.
The problem comes from the fact that when the duplicated output is closed after BT headset disconnection, the OUTPUT_CLOSED notification is not sent to AudioSystem. Then the mapping between notification stream and duplicated output cached in AudioSystem is not cleared and next time a notification is played, the duplicated output is selected and the createTrack() request is refused by AudioFlinger as the selected output doesn't exist.
The notification is ignored by AudioFlinger because when it is sent by the terminating playback thread, the thread has already been removed from the playback thread list.
The fix consists in sending the notification in closeOutput() and not when exiting the playback thread.
The same fix is applied to record threads.
This is due to a regression introduced by change 24114: when no audio tracks are ready for mixing, 0s are written to audio hardware. However this should only happen if tracks have already been mixed since the audio flinger thread woke up.
Also do not write 0s to audio hardware in direct output threads when audio format is not linear PCM.
AudioFlinger: verify that mCblk is not null before using it in Track and RecordTrack contructors.
IAudioFlinger: check result of remote transaction before reading IAudioTrack and IAudioRecord.
IAudioTrack and IAudioRecord: check result of remote transaction before reading IMemory.
Add a parameter to ToneGenerator.startTone() allowing the caller to specify the tone duration. This is used by the phone application to have a precise control on the DTMF tone duration which was not possible with the use of delayed messaged.
Also modified AudioFlinger output threads so that 0s are written to the audio output stream when no more tracks are ready to mix instead of just sleeping. This avoids an issue where the end of a previous DTMF tone could stay in audio hardware buffers and be played just before the beginning of the next DTMF tone.
Do not ramp volume if the first frame of a track is processed after the track was stopped.
In the case of very short sounds, the track stop request can be received by AudioFlinger just after the start request before the first frame is mixed by AudioMixer. In this case, the track is already in stopped state and initial volume is applied with a ramp for the first frame processed which should not be the case: initial volume change is always applied immediatelly.
In AudioFlinger::MixerThread::putTracks(), change the mFillingUpStatus flag to FS_FILLING for active tracks so that mute request is executed without ramping volume down when the track is moved from A2DP to hardware output.
Also modified AudioFlinger::setStreamOutput() so that the notification of the change is sent only once to AudioSystem.
This is because the AudioFlinger duplicating thread is closed while the output tracks are still active. This cause the output tracks objects to be destroyed at a time where they can be in use by the destination output mixer.
The fix consists in adding the OutputTrack to the track list (mTracks) of its destination thread so that a strong reference is help during the mixer processed and the track is detroyed only when safe by destination thread.
Also added detection of problems when creating the output track (e.g. no more tracks in mixer). In this case the output track is not added to output track list of duplicating thread.
When changing the audio output stream sampling rate with setParameters() make sure that all tracks have a sampling rate less or equal to 2 times the new output sampling rate.
The BT headset detection now makes the difference between car kits and headsets, which can be used by audio policy manager.
The headset connection is also detected earlier, that is when the headset is connected and not when the SCO socket is connected as it was the case before. This allows the audio policy manager to suspend A2DP output while ringing if a SCO headset is connected.
There was no garanty that the corresponding thread destructor had been already called when exiting the closeOutput() or closeInput() functions.
This contructor could be called by the thread after the exit condition is signalled. By way of consequence, closeOutputStream() could be called after
we exited closeOutput() function.
To solve the problem, the call to closeOutputStream() or closeInputStream() is moved to closeOutput() or closeInput().
The function checkForNewParameters_l() is called with the ThreadBase mutex mLock locked. In the case where the parameter change implies
an audio parameter modification (e.g. sampling rate) the function sendConfigEvent() is called which tries to lock mLock creating a deadlock.
The fix consists in creating a function equivalent to sendConfigEvent() that must be called with mLock locked and does not lock mLock.
Also added the possibility to have more than one set parameter request pending.
Use integers instead of void* as input/output handles at IAudioFlinger and IAudioPolicyService interfaces.
AudioFlinger maintains an always increasing count of opened inputs or outputs as unique ID.
Initial commit for review.
Integrated comments after patch set 1 review.
Fixed lockup in AudioFlinger::ThreadBase::exit()
Fixed lockup when playing tone with AudioPlocyService startTone()
Patch supplied on advice of partner. This causes us to send suspend_sink to
Bluez via socket interface, so we enter suspend on the A2DP link faster.
This is especially important when switching to SCO so that we come closer to
whitepaper recommendations to suspend A2DP before setting up SCO.
We have another patch set to add DBUS A2DP suspend and resume calls to Bluez
that will do a better job of following whitepaper recommendations for
A2DP -> SCO -> A2DP, but this small patch is still an improvement.
