[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [fluid-dev] Thread safety long-term thoughts
Re: [fluid-dev] Thread safety long-term thoughts
Thu, 19 Nov 2009 15:24:59 -0800
Internet Messaging Program (IMP) H3 (4.1.6)
Quoting David Henningsson <address@hidden>:
Btw; with the sample timers we also have the entire midi file player
(with its fopen calls etc) inside the audio thread. I'm guilty of that,
and it should be fixed by inserting a sequencer between the player and
the audio thread, at least in real-time use cases. Perhaps something
This scenario is less critical IMO, since the audio buffers could be
increased without issue (so what if the MIDI file plays 100ms later).
Good to know though. We may want FluidSynth to be smarter about
selecting default buffer sizes in the future, depending on the use case.
I'm sure the biggest time consumer in this regard is the note-on processing.
Assuming the note-on call makes no system calls, and the soundfont
isn't paged out by the OS, the note-on event should complete within a
good fixed time, or is there something else bugging us?
What bugs me about it is that the current note-on processing isn't
very efficient and is affected by the complexity of the underlying
instruments. I'm not sure how much of an issue it really is, but if a
lot of note-on events occur in a short period of time, it could lead
to xruns. Prior versions of FluidSynth processed the note-on event in
whatever thread was sending the note-on, which was usually the MIDI
thread which was usually running lower priority, so note-on processing
could not starve the audio thread.
It would be good to improve the efficiency of the internal note-on
event handlers, as I described previously, but it may also make sense
to process these outside of the synthesis thread in the multi-thread
case, especially since the note-on handler could be supplied by
Here is an example of what I was talking about, in regards to the
state machine depending heavily on the existing voices. Say for
example the pitch bend controller changes on a channel. All voices
are then scanned and those which are active on the given channel
are modulated in respect to the pitch bend controller. The active
voices and their parameters are private to the synthesis thread. I
don't YET see what additional processing could be moved outside of
the synth context for most events, from what it is right now.
Though I could see perhaps grouping controller changes, so only one
Perhaps there is not that much to gain performance-wise from moving
things out of audio thread context then. It would just feel less messy,
I guess, if we splitted the synth object in two parts, one with strict
real-time and one without.
I kind of felt like this sort of separation occurred with the thread
safety changes, at least I have a pretty good view of them being
separate, having delved into what was previously much messier. I
think there are improvements that can be made, but I think its a lot
better off than before. A lot of the calculations we are talking
about, in response to events, are pretty minimal. All of the ones
that require mutex locking or other OS calls are handled in
non-realtime. One exception which should be fixed is in regards to
SYSEX MIDI tuning events and Jack MIDI.
And instead of creating shadow variables, all variables should belong
the state machine, unless explicitly needed by the voices directly.
For clarification: Shadow variables were provided in only 2 cases for
the purpose of returning the most recent value assigned when querying
their values. The shadow values are used when querying the value
only. The 2 cases are presets, since atomic operations can't be used
in that case (might be possible if there was preset reference
counting) and polyphony, since polyphony is accessed fairly often in
the synthesis thread and I thought it was better to shadow the value
than have to atomically read it every time and also deal with issues
that might arise if the value is changed while looping over voices.
In the case above, the current pitch bend controller current value
belongs to the state machine. The new value is sent to the audio
thread. Btw, if someone tries to read the current pitch bend controller
value just after having set it, will it work?
Yes it will work. As far as I know, all values which can be queried
should now work, as far as appearing immediately. For all parameters,
except presets and polyphony, the value is set and accessed atomically
by all threads. When an event occurs, it assigns the new value to the
parameter and sends an update event to the synthesis thread. The
update event does not contain the value, but just tells the synth
thread to update the parameter based on the latest value. If an event
is set from within the synth thread, its pretty much the same, except
no queuing of the update occurs.
The way it is now, is much closer to the synth thread being a voice
renderer than it was with 1.0.9. It seems like its about
identifying additional stuff to move outside of it. Note on events
are the only thing I can think of at the moment, that could use
some improvement. Can you think of any others?
You probably have a better overview over the time and locking needed by
various events than I have, but I'm thinking about presets, tuning,
soundfont loading etc, but I guess they are already moved out of
Yes, they are already outside of synth context. The stuff that is
still inside synth context is within the _LOCAL function variants.
The midi thread and shell threads will do the state machine work, when
they make calls against the fluid_synth object.
We still need to do something about the Jack MIDI case, like
queuing the events back to another thread (return queue for
example). Like what is now being done with program changes.
Something like that, let the non-realtime stuff complete when it's
done, yet we must then queue all simple events only if there is
non-realtime processing in progress, to prevent reordering... (Sigh.)
Re-ordering is indeed one to watch out for. I hope there aren't any
more cases like that currently. Program changes are one case where
there is some expectation that a device might take 10s of milliseconds
or so, to load up a patch, so hopefully we can kind of assume that
note-ons wont occur for a little time after a program change. If its
all happening from outside of synth context, then it will all be
queued, without re-ordering issues, its only the Jack MIDI case that
might be a problem.
Perhaps an option to let the libfluidsynth user call a function
periodically if he wants to skip the additional thread.
Well it isn't just for garbage collection now. Its also being used to
handle program changes, which should happen ASAP.
Could we do something in create_audio_driver/delete_audio_driver to
register with the synth, and say that it is now operating with an audio
driver, and that means real-time operation? And if that is not the
case, handle events directly. Likewise, we could make the MIDI drivers
(and shell thread) register that we now have additional threads
referencing the state machine, so the state machine must be
Yeah, I think something like that could be good, rather than trying to
auto detect it. A simple API function like:
void fluid_synth_multi_thread_enable(fluid_synth_t *synth, int enable);
And/Or, like I mentioned before, we could try to improve the
multi-thread use auto-detection. I haven't fully looked into the
scenario, but if we made all the public API event functions
automatically enable multi-threading and then provide an alternative
to fluid_synth_handle_midi_event, say
fluid_synth_handle_midi_event_noqueue() that might suffice and
somewhat simplify things.
(Note: potential screwup with libfluidsynth users creating their own
audio drivers, although I assume it was screwed up in 1.0.9 the same
way as well.)
In regards to thread safety you mean? As things are now, it just
assumes it is multi-threaded, which means slightly less efficiency.
- [fluid-dev] Thread safety long-term thoughts, David Henningsson, 2009/11/16
- Re: [fluid-dev] Thread safety long-term thoughts, josh, 2009/11/16
- Re: [fluid-dev] Thread safety long-term thoughts, Ebrahim Mayat, 2009/11/17
- Re: [fluid-dev] Thread safety long-term thoughts, David Henningsson, 2009/11/18
- Re: [fluid-dev] Thread safety long-term thoughts, Ebrahim Mayat, 2009/11/18
- Re: [fluid-dev] Thread safety long-term thoughts, josh, 2009/11/18
- Re: [fluid-dev] Thread safety long-term thoughts, David Henningsson, 2009/11/19
- Re: [fluid-dev] Thread safety long-term thoughts,
- Re: [fluid-dev] Thread safety long-term thoughts, David Henningsson, 2009/11/21
- Re: [fluid-dev] Thread safety long-term thoughts, josh, 2009/11/24
- Re: [fluid-dev] Thread safety long-term thoughts, David Henningsson, 2009/11/26
- Re: [fluid-dev] Thread safety long-term thoughts, josh, 2009/11/26