Re: [libmicrohttpd] MHD_create_response_from_fd using a pipe

[Denis Dowling]

Hi,

The 'MHD_create_response_from_fd' almost does what I want but I need a 
different MHD_ContentREaderFreeCallback that does more then just close the fd. 
When the pipe is finished reading the process needs to be reaped with a 
pclose() call. Not sure if there is another way to have a function called when 
the response finishes reading the file.

I am testing this now. I have added a new function to libmicrohttpd

struct MHD_Response *
MHD_create_response_from_callback_fd (uint64_t size,
                                      size_t block_size,
                                      MHD_ContentReaderCallback crc,
                                      void *crc_cls,
                                      MHD_ContentReaderFreeCallback crfc,
                                      int fd)
{
  struct MHD_Response *response;

  response = MHD_create_response_from_callback (size,
                                                block_size,
                                                crc,
                                                crc_cls,
                                                crfc);
  if (NULL == response)
    return NULL;
  response->fd = fd;
  response->fd_off = 0;
  return response;
}

Then in my code when I want to return data from a process I do something like

    FILE *pf = popen(cmd, "r");
    int fd = fileno(pf);

    response = MHD_create_response_from_callback_fd(-1, 4096, &popen_reader,  
pf, &popen_free, fd);

The popen_reader function function looks like

static ssize_t popen_reader (void *cls, uint64_t pos, char *buf, size_t max)
{
  FILE *fd = (FILE *)cls;
  ssize_t n = read (fileno(fd), buf, max);
  if (0 == n)
    return MHD_CONTENT_READER_END_OF_STREAM;
  if (n < 0)
    return MHD_CONTENT_READER_END_WITH_ERROR;
  return n;
}

The popen_free function looks like

static void popen_free(void *cls)
{
    FILE *fd = (FILE *)cls;
   pclose(fd);
}

This seems to work however I am using SSL so the code path in 
send_param_adapter does not use sendfile(). I will have to dig deeper on the 
limitations here if not using SSL. I am on a 64bit platform so size_t is not an 
issue. Not sure if this will be an issue with a 32bit platform though as the 
individual block sizes will be smaller then 2^32 and there is no seeking needed.

Regards,
Denis

-----Original Message-----
From: address@hidden [mailto:address@hidden On Behalf Of Christian Grothoff
Sent: Tuesday, 2 June 2015 5:45 PM
To: address@hidden
Subject: Re: [libmicrohttpd] MHD_create_response_from_fd using a pipe

Hi!

Why don't you just use 'MHD_create_response_from_fd'?  I'm not sure how well 
pipes (or socketpairs) work with 'sendfile', but short of that, this should 
work at least to set the FD.

You wouldn't be able to do chunked encoding on systems where 'size_t != 
uint64_t' (because we have 'size_t' for the size argument, which is admittedly 
not nice). Other than those limitations, this should give you the desired 
result.

-Christian

On 06/02/2015 08:07 AM, Denis Dowling wrote:
> Hi,
>
>
>
> I am looking at generating some content from the result of a zip
> process and it would be neat if I could get the popen() pipe to push
> data directly into MHD_create_response. It looks like this would
> almost work by using MHD_create_response_from_callback but also
> setting the
> response->fd to be the fd of the pipe returned by popen(). Read
> response->callback
> would be mostly the same as happens for a file. The size would be set
> to
> -1 for chunked output. pclose() call could be run from the free callback.
>
>
>
> Has anyone done something like this before? There is an issue that the
> response->fd is opaque from user code and would need some function to
> expose.
>
>
>
> Regards,
>
> Denis
>
> ----------------------------------------------------------------------
> --
>
> The information contained in this message and any attachments is
> strictly confidential and intended solely for the use of the intended
> recipient(s). The copyright in this communication belongs to *(HSD)*.
> This message and any attachments are confidential and may be subject
> to legal or other professional privilege. Any confidentiality or
> privilege is not waived or lost because this e-mail has been sent to
> you by mistake. If you have received this transmission in error,
> please notify
> *HSD* on +61 3 9875 5900 immediately and destroy all copies of this
> e-mail. The contents of this email message may not be quoted, copied,
> reproduced or published in part or in whole, without the written
> authorisation of *HSD*.
>
________________________________


The information contained in this message and any attachments is strictly 
confidential and intended solely for the use of the intended recipient(s). The 
copyright in this communication belongs to  (HSD). This message and any 
attachments are confidential and may be subject to legal or other professional 
privilege. Any confidentiality or privilege is not waived or lost because this 
e-mail has been sent to you by mistake. If you have received this transmission 
in error, please notify HSD on +61 3 9875 5900 immediately and destroy all 
copies of this e-mail. The contents of this email message may not be quoted, 
copied, reproduced or published in part or in whole, without the written 
authorisation of HSD.