Re: [Denemo-devel] GSoC proposal to improve MIDI and audio capabilities

[Philipp Überbacher]

Excerpts from Dominic Sacré's message of 2011-04-11 01:31:53 +0200:
> Hi Richard,
> 
> On Thursday 07 April 2011 18:45:58 Richard Shann wrote:
> > On Thu, 2011-04-07 at 15:07 +0200, Dominic Sacré wrote:
> > > I didn't mean to imply that Denemo is full of broken features, just
> > > that my proposal is mainly about restructuring and cleanup, not
> > > about removing things.
> > 
> > yes, I didn't take offence, I understood that, my question is would
> > the re-structuring you propose allow the adjustment of the MIDI
> > timing during feedback, as exemplified by the Tempo slider. That is
> > if you could describe the sequence of events that would happen when
> > someone moves the tempo slider during playback then I would be more
> > confident that we weren't getting ourselves into a trap.
> 
> I'm not sure if I can describe the sequence of events that would happen, 
> as I'm still struggling to understand how the current codebase works, or 
> more importantly why it was written the way it is.
> What I can tell you is that things like the tempo slider would just keep 
> working like they do now. With the changes I'm proposing, during 
> playback all MIDI events would still be read one by one from the SMF 
> structure (see fluidsynth_play_smf_event() and 
> jackmidi_play_smf_event()). The difference would just be that currently 
> it's up to FluidSynth to create its own output ports (or access the 
> soundcard, depending on the underlying audio API), while I'm proposing 
> to do that by ourselves in Denemo. Nevertheless, the way MIDI events are 
> fed from the SMF structure to the FluidSynth engine would remain more or 
> less the same.
> 
> That said, there's of course also the timing issue that Philipp 
> described. Now that I'm thinking about it, this is probably what 
> prompted the suggestion of "making midi/audio output realtime-capable" 
> as a GSoC project. Having written some JACK clients I automatically read 
> that as "make JACK support realtime-safe", but bad MIDI timing is 
> actually a different issue that has nothing to do with realtime-safety 
> per se.
> 
> As far as I can see, both the FluidSynth and JACK MIDI backends use 
> g_idle_add() to queue MIDI events for playback. It's pretty obvious that 
> timing-wise this is a terrible idea. Could anyone explain why the code 
> was written that way?
> A proper multi-threaded model seems to be the way to solve this, and 
> again, Philipp explained this much better than I did (in fact, I didn't 
> mention in at all...). I still believe the best approach is to tackle 
> all these problems at once in one major overhaul.
> 
> > > It has also been suggested that audio input is only semi-usable
> > 
> > I use it every week to tune my harpsichord. But I have not heard
> > reports of people using it for inputting notes to Denemo - it
> > certainly still does that - due to noises in the room I usually get
> > a few notes inserted before I hit the Tuning button - but it will
> > surely have some bit rot by now - it has lots of fancy pattern
> > following features ... (I got a MIDI keyboard a couple of years ago
> > and have never looked back).
> > 
> > > and
> > > should not be a high priority. I'm not so sure about that, and
> > > would rather fix it than remove it, as it's a pretty nifty
> > > feature. Incidentally, I haven't been able to test it yet. At the
> > > moment all my usable hardware audio inputs are JACK-only
> > > (firewire), and audio input in Denemo only works with PortAudio.
> > > Did I mention I was planning to fix that? ;)
> > 
> > yes you did, and I only mentioned that if you needed to cut corners
> > to achieve the goal audio input could be left untouched, it would
> > not break anything and would not be broken if it was left untouched.
> 
> Fortunately, removing the PortAudio dependency from the audio input code 
> doesn't look very difficult. It may even be easier to just do that, 
> rather than working around this part of the code just to be able to 
> leave it untouched, while the rest of the audio/MIDI I/O code is turned 
> upside down...
> 
> As for cutting corners, in the latest (final) version of my GSoC 
> proposal I de-emphasized support for Windows and OS X. I wouldn't 
> consider my work finished until those are supported (as they are now), 
> but building and testing Denemo on all these different platforms is a 
> can of worms I'd rather not open while I'm on such a tight schedule.
> 
> 
> Best regards,
> 
> Dominic

I just noticed that LAC will have some possibly useful talks this year:
http://lac.linuxaudio.org/2011/?page=program&mode=table&day=3
"FluidSynth real-time and thread safety challenges" and
"Low-Latency Audio on Linux by Means of Real-Time Scheduling"

Regards,
Philipp