Re: [libmicrohttpd] POST data processing

[Austin Einter]

Dear All
I am getting data in access content callback.

I can do my own parsing.

To get body data, independent chunks through iterate callback (NOT access content callback), I have issues.

In some cases, post process create fails (returns null).
In some cases, post process is created, but when I call post process api, iterate callback is not called.

Are you telling that

1) always post process create will succeed (irrespective of xml / json / binary body)

2) if post process is created, on calling post process api, iterate callback is called.

Once I get sometime (sorry, but really occupied), will put a small code and share.

Regards

Austin

On Thu, May 17, 2018 at 8:24 PM, Kenneth Mastro <address@hidden> wrote:

Unless something broke in a very recent version, POST definitely works with JSON data.  I do it all the time.  I could be wrong, but I suspect MHD does not care about the data type at all.

I very strongly suspect the problem is in your code or your test.  Since it works with form data, are you sure your request is getting sent as a proper 'POST' when you're trying with XML or JSON?

Just a thought - are you just trying to use MHD's post-processor to process the data like you do with form data?  In that case - no, MHD does not PARSE the JSON or XML (like it can with form data).  You have to do that yourself or use a third party library (but either way, you have to capture the data from MHD and do something with it on your own).

Ken

On Thu, May 17, 2018 at 10:33 AM, Austin Einter <address@hidden> wrote:

Hello
When I tried, if content type is xml, post processor crate failed (it returned null).

When content type is binary, it created post processor, however when I call post process api, the iterate callback not called.

That made me to search google.
Somewhere I read it does not support xml and json.

Data I am getting in access content callback, however my iterate callback is not called.
Please note that same code is working for text data (file upload or form data).

Thanks

Austin

On Thu, May 17, 2018 at 6:56 PM, silvioprog <address@hidden> wrote:

Hello Austin,

As Christian explained, via "upload_data" you can receive any payload kind. I'm writting a library which maps MHD callbacks to "objects" and it needs to support the following features:

1. receive one or more files on demand (to receive large files about 5 GB+ without "frying" CPU/RAM) in same request (files via form-data);

2. receive form fields (HTML-form fields via x-www-form-urlencoded);

3. 1 and 2 (both files and fields via form-data);

4. payload contents (raw-data, json/xml and so on via application/<type>);

5. handlers to create customized 'body-parser' if none of the above options fits.

it seems the option 4 above is the one you are looking for. For example, supposing you need to send a JSON "{"foo":"bar"}" to the server, in the client:

curl -X POST -H "Content-Type: application/json" -d '{"foo":"bar"}' http://localhost:8080

and in a minimal snipped server example, you can receive it as:

...

static void req_cb(void *cls, struct bk_httpreq *req, struct bk_httpres *res) {

    struct bk_str *payload = bk_httpreq_payload(req);

    printf("Payload string: %s; Size: %zd", bk_str_content(payload), bk_str_length(payload));

    ...

}

int main(void) {

    struct bk_httpsrv *srv = bk_httpsrv_new(req_cb, NULL);

    bk_httpsrv_listen(srv, 8080, false);

    ...

}

...

then it prints "Payload string: {"foo":"bar"}; Size: 13" in the terminal. The "payload" variable above is an instance of the object "bk_str" which contains useful "methods" for string handling like "content", "length", "clear", "printf" and more.

If you want to take a look how it was implemented, please use the branch "new_api" and specifically the line 78 from bk_httpuplds.c file:

https://github.com/risoflora/libbrook/blob/new_api/src/bk_httpuplds.c#L78

hope this help you.

(the library is under development, so there is parts undocumented yet but feel free to send any questions via Github issues)

On Thu, May 17, 2018 at 1:52 AM, Austin Einter <address@hidden> wrote:

Hello
I am using libmicrohttpd and facing issue with POST processing.

The POST message received contains xml or json data. libmicrohttpd does not support xml / json. So I can not use post process or iterate callbacks for body data processing. In fact I tried and it does not work (iterate callbacks not called even when I call post process).

So I am left with two options.

1) Either write body parser, where I need to take care of content-length case, chunked data case etc. 

2) OR use http-parser kind of open source

Is there any other option available?

I am not very pro to write my own parser, instead I would like to use http-parser. I just looked at http-parser. To use http-parser, we need to provide the entire message to http-parser, either single time or one chunk at a time.

Is there anyway in libmicrohttpd, I can collect the entire POST message as it comes / received at socket level?

If possible, kindly provide required steps / info.

Best Regards

Austin

 

--

Silvio Clécio