Is it possible to use libmicrohttpd to transfer large amounts of binary data (e.g. 10GB) incrementally using a thread pool model? By incrementally I mean that after we receive 10MB we'd like to process the 10MB internally before allowing more data to be transferred to the web server-- we don't have enough memory to receive the whole 10GB first. Using MHD_USE_THREAD_PER_CONNECTION we can achieve this by blocking until we process each 10MB chunk. However, for scaling reasons we'd prefer to use the thread pool model, but I can't find a way to "block" acknowledgement of processing some data without synchronously blocking one of the threads in the pool. It looks like the POST request handlers are close to what I need, except my data is not form-encoded.
Any suggestions would be greatly appreciated.
Thanks,
Jared