Re: [libmicrohttpd] race condition on 0.9.73

[Evgeny Grin]

On 23.09.2021 22:35, José Bollo wrote:
> On Thu, 23 Sep 2021 19:57:23 +0300
> Evgeny Grin <k2k@yandex.ru> wrote:
>
> > On 23.09.2021 18:43, José Bollo wrote:
> > > > For sure you can run MHD with "external poll" mode, but you must
> > > > ensure that only single thread is calling MHD_run() at any given
> > > > moment of time.
> > >
> > > Okay, that is a good hint, thanks. I'm going to first track
> > > concurrent use of MHD_run. FYI, I tried the latest git and got the
> > > same issue.
> > > > If you want to call several copies of MHD_run(), make sure that you
> > > > have individual copies of MHD for each thread (start MHD by
> > > > MHD_start_daemon() for every thread).
> > >
> > > I don't want that, because it implies differents port I suppose.
> >
> > Even if MHD is doing internal accept(), you still can use
> > MHD_OPTION_LISTENING_ADDRESS_REUSE parameter to bind to the same
> > port. Or provide your own socket.
> > However as I can see below, it's not applicable for your case as you
> > are not using MHD to listen for new connections.
> >
> >
> > > > Alternatively, you can use MHD with its own threading mode. MHD can
> > > > run treads per connection or use thread pool.
> > > >
> > > > Can you give more details how you use MHD? Which flags are used?
> > >
> > >         flags = MHD_USE_EPOLL
> > >                         | MHD_ALLOW_UPGRADE
> > >                         | MHD_USE_TCP_FASTOPEN
> > >                         | MHD_USE_NO_LISTEN_SOCKET
> > >                         | MHD_ALLOW_SUSPEND_RESUME;
> >
> > Do you use suspend/resume of connections?
>
> yes for websocket upgrade but it is not relevant in the case.

Don't you use MHD built-in "upgrade" functions?

> > >    
> > > > Why do you use external threads instead of MHD internal treads?
> > >
> > > The program has a job scheduler that manages threads on its own. It
> > > also serves other protocols that HTTP(S).
> >
> > It is still hard to see where is the problem. Actually responses are
> > thread-safe and shouldn't create any such kind of problems.
> >
> > Could you share you access handler callback code?
> > Where do you generate responses?
> > Where do you queue responses?
>
> I think that the clue about re-entering MHD_run is the good one. I'm
> going to follow that path tomorrow and will let you know. The code is
> public but you know, it is a pain to read someone else code and this
> part is poorly commented.

Please share you results.
We are close to releasing the next MHD version and want to make sure 
there are no know bugs in the code.

I'm wondering, why do you run the same MHD instance in different threads?
Every MHD_run() processes *all* connections served by particular MHD 
instance. This looks like work duplication (or multiplication).
If you process specific connections by each thread and you add 
connections to MHD by MHD_add_connection(), just use thread-specific MHD 
instance so you will always know which connections are processed by 
MHD_run().

--
Evgeny