My issue is actually fixed: the default audio.period-size parameter (64) was too low. 128 was still to low, 256 gave a perfect sound! The whole synthesizer sounds very reactive, no latency to my ears.
A period of 256 at 44100 gives you about 5ms of latency. Multiply that with the number of periods to get the actual minimal latency. If you want lower latency after all, then you could go for a custom kernel with preempt-rt patches. I'm running fluidsynth on similar hardware with 64 period size and 2 periods without any glitches, based on a real-time preempt-rt kernel and polyphony limit of 64.
Also, as your device seems to be a quad-core, you might benefit from using multiple render threads, especially if you need higher polyphony. Simply set the number of render threads with the -o synth.cpu-cores setting.
odrodroid@odroid:~$ aplay -L
null
Discard all samples (playback) or generate zero samples (capture)
pulse
PulseAudio Sound Server
sysdefault:CARD=ODROIDHDMI
This device (card 0, device 0 = hw:0,0) seems to be your default device. In other words, by default it outputs sound via the HDMI interface?
sysdefault:CARD=ODROIDDAC
This is card number 1, device 0 = hw:1,0. That seems to be the DAC that you want to use for output. So -o audio.alsa.device=hw:1,0 should output directly to your DAC, without alsa doing any kind of extra processing.
Cheers
Marcus