libmicrohttpd
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [libmicrohttpd] Big incoming requests


From: Ken Zangelin
Subject: Re: [libmicrohttpd] Big incoming requests
Date: Fri, 30 Oct 2015 00:53:57 +0100
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0

ok, I get it, I acknowledge each chunk I get in what I call "Part II" of the 
MHD_AccessHandlerCallback.
Before, I only acknowledged (and copied the data) once I had the entire buffer.
Easy change, already implemented and working like a charm.

Thank you again,

/KZ

On 29/10/15 19:35, Christian Grothoff wrote:
Look at connection.c:1032, which is the only place where this error is
generated:

           if (connection->read_buffer_offset ==
               connection->read_buffer_size)
             {
               if ((MHD_YES != try_grow_read_buffer (connection)) &&
                   (0 != (connection->daemon->options &
                          (MHD_USE_SELECT_INTERNALLY |
                           MHD_USE_THREAD_PER_CONNECTION))))
                 {
                   /* failed to grow the read buffer, and the
                      client which is supposed to handle the
                      received data in a *blocking* fashion
                      (in this mode) did not handle the data as
                      it was supposed to!
                      => we would either have to do busy-waiting
                      (on the client, which would likely fail),
                      or if we do nothing, we would just timeout
                      on the connection (if a timeout is even
                      set!).
                      Solution: we kill the connection with an error */
                   transmit_error_response (connection,
                                         MHD_HTTP_INTERNAL_SERVER_ERROR,
                                            INTERNAL_ERROR);


The situation: you are called to handle upload data, but did not
(effectively, your callback failed to update *upload_data_size when
called). Additionally, the IO buffer is full. So your application
doesn't handle the data that is in the buffer, and we can't read more to
give your app a bigger chunk. That's deadly, as MHD shouldn't then go
into busy waiting, hence the 500 internal error.

You could specify a larger memory limit, but the better solution is to
properly handle the 'upload_data' *incrementally*.  The
MHD_PostProcessor logic is an example for how to do this.

Happy hacking!

Christian


On 10/29/2015 03:20 PM, Ken Zangelin wrote:
On 29/10/15 14:13, Christian Grothoff wrote:
It limits the length of the URL and the headers, not the size of
uploaded POST data. -Christian

On 10/29/2015 12:08 PM, Ken Zangelin wrote:
Hi, I have a basic question regarding the
MHD_OPTION_CONNECTION_MEMORY_LIMIT option to MHD_start_daemon.

Will the size given to MHD_OPTION_CONNECTION_MEMORY_LIMIT limit the
maximum permitted size of any request?

Thank you,

/KZ

OK, in that case I have a follow-up question.

We recently started to play with this connection memory limit option
(MHD_OPTION_CONNECTION_MEMORY_LIMIT)
and we have seen that requests with larged payload stops working for low
memory limits.

To give some detail, changing MHD_OPTION_CONNECTION_MEMORY_LIMIT to 64k,
the max accepted size of payload seems to be 33509 bytes.

If a bigger payload is sent, then MHD responds with the following message:

     HTTP/1.1 500 Internal Server Error
     Content-Length: 132
     Date: Thu, 29 Oct 2015 13:53:15 GMT


     <?xml version="1.0"?>
     <html>
       <head>
         <title>Internal server error</title>
       </head>
       <body>Some programmer needs to study the manual more
carefully.</body>
     </html>


Our MHD_AccessHandlerCallback function is called twice with payload
(apart from the first call), and then
the MHD_OPTION_NOTIFY_COMPLETED callback is called and MHD responds by
itself.

With a bigger value for MHD_OPTION_CONNECTION_MEMORY_LIMIT this did not
happen.

The version of the MHD we use:
#define MHD_VERSION 0x00093300



So, we might be doing something the wrong way.
Please tell us what to do to be able to receive large payloads with a
"small" MHD_OPTION_CONNECTION_MEMORY_LIMIT.

Thank you again,

/KZ


________________________________

Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario,
puede contener información privilegiada o confidencial y es para uso
exclusivo de la persona o entidad de destino. Si no es usted. el
destinatario indicado, queda notificado de que la lectura, utilización,
divulgación y/o copia sin autorización puede estar prohibida en virtud
de la legislación vigente. Si ha recibido este mensaje por error, le
rogamos que nos lo comunique inmediatamente por esta misma vía y proceda
a su destrucción.

The information contained in this transmission is privileged and
confidential information intended only for the use of the individual or
entity named above. If the reader of this message is not the intended
recipient, you are hereby notified that any dissemination, distribution
or copying of this communication is strictly prohibited. If you have
received this transmission in error, do not read it. Please immediately
reply to the sender that you have received this communication in error
and then delete it.

Esta mensagem e seus anexos se dirigem exclusivamente ao seu
destinatário, pode conter informação privilegiada ou confidencial e é
para uso exclusivo da pessoa ou entidade de destino. Se não é vossa
senhoria o destinatário indicado, fica notificado de que a leitura,
utilização, divulgação e/ou cópia sem autorização pode estar proibida em
virtude da legislação vigente. Se recebeu esta mensagem por erro,
rogamos-lhe que nos o comunique imediatamente por esta mesma via e
proceda a sua destruição



________________________________

Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede 
contener información privilegiada o confidencial y es para uso exclusivo de la 
persona o entidad de destino. Si no es usted. el destinatario indicado, queda 
notificado de que la lectura, utilización, divulgación y/o copia sin 
autorización puede estar prohibida en virtud de la legislación vigente. Si ha 
recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente 
por esta misma vía y proceda a su destrucción.

The information contained in this transmission is privileged and confidential 
information intended only for the use of the individual or entity named above. 
If the reader of this message is not the intended recipient, you are hereby 
notified that any dissemination, distribution or copying of this communication 
is strictly prohibited. If you have received this transmission in error, do not 
read it. Please immediately reply to the sender that you have received this 
communication in error and then delete it.

Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinatário, pode 
conter informação privilegiada ou confidencial e é para uso exclusivo da pessoa 
ou entidade de destino. Se não é vossa senhoria o destinatário indicado, fica 
notificado de que a leitura, utilização, divulgação e/ou cópia sem autorização 
pode estar proibida em virtude da legislação vigente. Se recebeu esta mensagem 
por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e 
proceda a sua destruição



reply via email to

[Prev in Thread] Current Thread [Next in Thread]