|Subject:||Re: [fluid-dev] Real-time Controls + Audio Streaming|
|Date:||Sat, 3 Nov 2012 11:59:47 -0700|
- USER EXPERIENCE
- Web app (i.e. runs in most Mac + Windows desktop browsers) that plays MIDI files with high-quality grand piano SoundFont
- Real-time controls for speed and pitch (along with typical controls for volume, pause/play, etc.)... so there's no option to pre-generate audio files since you can't anticipate what pitch/key combination will be requested in the middle of playing the song.
- My assumption is that it is NOT a good idea to have the softsynth running in the browser (computationally intense, large SoundFont download, install fat client vs. web app, etc.)
- So this leads me to believe that the softsynth should be running in real-time on the server, generating audio that can be streamed to the browser app, which would be very lightweight since all it would need to do is play streaming audio
- The controls for speed + pitch would actually go back to the server, and in real-time cause the softsynth to generate the corresponding audio which would be streamed to the web clientSo my questions are:
- Can Fluidsynth be installed on a server and generate real-time audio fast enough to give up with playback, i.e. given typical server CPU and single piano instrument, is it reasonable to expect that FluidSynth can generate audio in faster than real-time?
- Can the FluidSynth API be accessed in mid-song to change the pitch and velocity, or does it have to start over from the beginning of the song?
- Do you know of anyone who has taken the audio output from FluidSynth and streamed it to another client?I greatly appreciate your time to review these questions and hopefully point me in the right direction. And for those who are interested, I'm willing to pay for a short-term development contract to help get this project started.Thanks!-- Dave
My own thoughts around the architecture of such a system is the following:
* Would be a server based solution, with a server application written in C.
* Server application would handle spawning FluidSynth rendering threads using libFluidSynth to stream to users.
* Server would provide a FastCGI interface for control FluidSynth instances.
* Server application would interface to a Shoutcast server to stream encoded MP3 data.
One thing about such a system is it would require quite a lot of CPU and probably would not scale very well. So in cases where there were a lot of users (say more than a couple), a distributed server farm like topology would be needed (synthesis servers, MP3 encoding servers, etc).
I am interested to hear the details of your offer to pay for such a project.
|[Prev in Thread]||Current Thread||[Next in Thread]|