Re: [libmicrohttpd] chunked encoding & mjpeg stream

[Evgeny Grin]

To summarize my findings (and interpretation):
1. Any "multipart/*" type is a "Content-Type". It is not a kind of 
"Transfer-Encoding". It is a property of payload.
2. HTTP/1.1 requires sender to specify the end of the payload by 
specifying either "Content-Length" or "Transfer-Encoding" (with final 
"chunked" encoding) (see remark 1 below).
3. The "boundary" delimiters are part of payload. If "Content-Length" is 
used, the size must include the size of delimiters (see remark 2 below).
4. Additional headers after "boundary" delimiters are part of payload, 
not part of HTTP headers (see remark 2 below).

Remark 1: RFC directly requires HTTP client to use "Content-Length" or 
"Transfer-Encoding" in 
https://datatracker.ietf.org/doc/html/rfc7230#section-3.3.2, but similar 
requirement for server is not so explicit and mentioned only in 
https://datatracker.ietf.org/doc/html/rfc7230#section-3.3.3. Probably 
it's good idea to document in RFCs similar requirements together, for 
both HTTP client and HTTP server.

Remark 2: RFCs are not very clear about it. On one hand 
https://datatracker.ietf.org/doc/html/rfc7233#appendix-A specified 
together HTTP headers alongside with additional headers in body, the one 
can read it as headers in body are part of HTTP headers. On the other 
hand, specification of "multipart/form-data" (RFC7578) is based on 
RFC2046, which explicitly specified that "multipart" messages are 
encapsulating other messages. 
https://datatracker.ietf.org/doc/html/rfc2046#section-3
Moreover https://datatracker.ietf.org/doc/html/rfc2046#section-5.1.1 and 
https://datatracker.ietf.org/doc/html/rfc2046#section-4.1.1 require only 
CRLF to be used as line breaks (unlike HTTP, which allow bare LF to be 
used).
I think it would be nice to update RFCs to explicitly specify that all 
versions of "multipart" Content-Type are still just payloads for HTTP.


Conclusions:
When "multipart/x-mixed-replace" is used with HTTP protocol, it must be 
treated as additional encapsulation layer transmitted by HTTP. 
HTTP-compatible clients must read HTTP payload as specified by RFCs and 
then extract binary data from "multipart" payload.
Clients must be able to correctly process "multipart" payload even if it 
is transmitted with chunked encoding, even if chunk borders do not match 
"boundary" delimiters.

Looks like current versions of Firefox and Chrome are trying to process 
"multipart" encapsulation as part of HTTP encapsulation which is not 
correct.

--
Evgeny


On 28.02.2022 19:14, Evgeny Grin wrote:
> Hello,
>
> Looks like Chrome and Firefox just unable to process chunked encoding 
> together with multipart/x-mixed-replace.
>
> Actually "multipart/x-mixed-replace" was never standardized for HTTP.
> We have other forms of "multipart/" defined for HTTP:
> * "multipart/form-data" for requests, see 
> https://datatracker.ietf.org/doc/html/rfc7578
> * "multipart/byteranges" for "206 Partial Content", see 
> https://datatracker.ietf.org/doc/html/rfc7233#section-4.1 and 
> https://datatracker.ietf.org/doc/html/rfc7233#appendix-A
>
> However, all these forms need to be compliant with HTTP specifications, 
> which require either "Content-Length" or "Transfer-Encoding" with 
> "chunked" as the final encoding. See 
> https://datatracker.ietf.org/doc/html/rfc7230#section-3.3.3.
>
> So, if you are sending this kind of response, you must either declare 
> the payload size in the header or use chunked-encoding.
>
> Both Chrome and Firefox are slowly deprecating and removing support for
> "multipart/x-mixed-replace".
> See
> https://bugs.chromium.org/p/chromium/issues/detail?id=249132 (dated 2013)
> https://developer.mozilla.org/en-US/docs/Mozilla/Firefox/Releases/22 
> (released in 2013).
>
> If you really need to use "multipart/x-mixed-replace" with browsers and 
> you don't know the size in advance, there is a workaround: use 
> MHD_create_response_from_callback() with MHD_SIZE_UNKNOWN as the size 
> and apply MHD_RF_HTTP_1_0_COMPATIBLE_STRICT flag 
> (MHD_RF_HTTP_VERSION_1_0_ONLY for older MHD versions). This will force 
> MHD to not use the proper chunked encoding and result should be readable 
> by browsers.
>
> Let us know please if this solves the issue for you.

OpenPGP_signature
Description: OpenPGP digital signature