Re: [libmicrohttpd] Reading data from named pipe

[Christian Grothoff]

On 03/27/2017 08:03 PM, Alex Reynolds wrote:
> I have an API that reads a subset of data from an archive and writes it to
> a FILE*. I can use this API to write this data subset to a temporary file
> via mkstemp() and fdopen().
> 
> I am currently using MHD_create_response_from_callback() to read chunks of
> bytes from this temporary file and write them to the web client. This works
> well on small files.
> 
> To avoid the I/O cost of writing a larger temporary file, and then reading
> from it, I set up a named pipe via mkfifo() and a FILE* that points to it.
> When I have MHD_create_response_from_callback() try to read from this named
> pipe, in the same way that it is set up to read from the temporary file,
> the server and client hang.
> 
> If this is possible, what is the correct way to serve data from a named
> pipe?

The answer depends on the style of event loop you are using.

For external select, do a non-blocking (!) read from it, and return
whatever you got to MHD via response_from_callback(), returning 0 if the
pipe is still operational but you have no more data. To keep the event
loop going, you need to manually add your pipe to the read-set, so that
select/poll/epoll unblock as needed.

For thread-per-connection, use a blocking (!) read from the pipe, and
always return data to MHD.

For thread-pool / internal select, you would have to suspend the
connection when the pipe runs dry, and then have some _other_ thread
check for the pipe becoming again available and calling resume on the
connection. This one is usually messy.


If you want to go experimental, you could try to get
MHD_create_response_from_fd64() to work and use fileno() on your FILE*
to get the underlying FD.  I have _never_ tested this, and suspect it'll
blow up in your face the moment the 'fd' blocks as MHD expects it to be
a file and not a pipe. But, maybe it is possible to hack MHD to handle
blocking 'sendfile()' --- I am pretty sure it does not manage this case
today --- and thereby unify the 3 cases above, simplify the client and
improve performance.  Still, this is more like adding a medium-size
feature to MHD than the "correct way" you asked for, so don't take this
as advice for how to do it, but more as a pointer in case you ever need
something even better and have too much time ;-).

signature.asc
Description: OpenPGP digital signature