Re: [libmicrohttpd] MHD_OPTION_PER_IP_CONNECTION_LIMIT behaviour behind a reverse proxy

[Evgeny Grin]

Hi Kelson,

On the one hand this is something that could be implemented in MHD.

On the other hand, the value of such implementation is questionable.
The per-ip limit was implemented to avoid getting of all (or too much 
of) servers resources for serving a single client (or just a few 
clients). When clients are connected directly, we MHD can get 
information about the client's IP from the sockets, so requests above 
the limits are just refused. In situation when the information must be 
extracted from the headers, MHD should:
* accept the new connection (the new socket),
* allocate memory and other resources (start the new thread in 
thread-per-connection mode),
* receive the data on the new connection,
* parse received data to find end of the request headers,
* parse headers to find "X-Forwarded-For" header (or headers if 
reverse-proxies are chained, then find the right one header),
* parse "X-Forwarded-For" value.
Such processing already took a lot of resources. If the response is 
small, then it would be better to send the response anyway to save 
resources of processing repeated request from the same client.
And this is something that can be easily implemented in the application 
logic. You should just return "MHD_NO" when access handler callback is 
called. It would be exactly the same effect: accept connection, parse 
headers, close connection.

I don't see any noticeable advantage of implementing it at the MHD level.
Ideally it should be implemented at reverse proxy level, then you would 
save resources on rejected connections.
If you implement it on MHD level, that resource usage can be even higher 
compared to processing without per-ip limiting, because clients will 
repeat requests (and repeated request would need to be accepted and 
parsed before rejected).

--
Evgeny (AKA Karlson2k)


-------- Original Message --------
From: Emmanuel Engelhart <kelson@kiwix.org>
Sent: Sunday, February 6, 2022, 16:58 UTC+3
Subject: [libmicrohttpd] MHD_OPTION_PER_IP_CONNECTION_LIMIT behaviour 
behind a reverse proxy

> On 06.02.22 12:39, Christian Grothoff wrote:
> > You're misreading the code, "5" is the numeric value of the option in
> > the enum, not the actual default limit. The default limit is indeed zero
> > as per the documentation (and a limit for zero means 'unlimited').
>
> Thank you for the clarification. It makes sense.
>
> > I think your suggestion to check against X-Forwarded-For is interesting,
> > but I wonder if in that case it would not be better/simpler to simply
> > let the reverse proxy do this check? At least I expected that to be the
> > case, and I think _if_ we implemented an X-Forwarded-For based check,
> > we'd need a new option, new data structure, and it'd also imply a
> > (minor) additional performance hit. So really the question is: if you're
> > behind a reverse proxy, why not configure the reverse proxy to limit the
> > number of connections per IP?
>
> If this is not doable at microhttpd level, we will have to do it in the 
> reverse-proxy indeed. I just though that because there is already an 
> option in microhttpd, and because this is not unusual for Web apps to 
> take care of this case, I would put a message about it.
>
> My personal opinion is that it would be neat and straight forward to 
> have microhttpd just doing the job. If you agree on the principle, we 
> could even maybe propose a patch. But, there is for sure good arguments 
> for both include and not include such a feature. Therefore, whatever 
> your decision on it, I'm sure I will understand it.
>
> Kelson

OpenPGP_signature
Description: OpenPGP digital signature