[libmicrohttpd] [GSoC Update] Week 5

[Didik Setiawan]

================================= SUMMARY ======================================
My public fork of Wget2 project is available here [1][2]. I will continuously 
keep
pushing my work so anyone interested can track me there. Feel free to
participate in the discussions going on merge request with my mentors. Your
feedback are highly appreciated.


=============================== INTRODUCTION ===================================
The purpose of this project is to use Libmicrohttpd as test suite for Wget2.  I
plan to do this by do some changes on function wget_test_start_server() also
wget_test_stop_server() on src/libtest.c of Wget2. With this approach, I don't
need to change existing test suite which call the internal server code through
functions mentioned above.
I've count there are 38 test file which use wget_test_start_server(). I must
ensure all the test passed when I've done the transition.
For installation prerequisite, I make that Libmicrohttpd support will needed
when we need to run `make check` or run test suite. Then I need to modify
configure.ac. I will give proper warning about this requirement. There is a
section in README.md where I must explain to user to provide Libmicrohttpd to
make all test running correctly.
With Libmicrohttpd I can add new test using feature that not yet implemented in
old server code, but ready on Libmicrohttpd, such as HTTP authentication and
concurrent request checking. 

Mentors:
Darshit Shah <address@hidden>
Ander Juaristi <address@hidden>


================================== UPDATES =====================================
Things which were done until this week:

 * Started working on wget_test_start_server() to use Libmicrohttpd as HTTP
   server. Workflow to resolve this:
   - Remove initial process for HTTP server socket
   - Create _http_server_start() function, wrapper for Libmicrohttpd. There is
     also function answer_to_connection() which use to create proper HTTP
     response
   - Use select method (MHD_USE_SELECT_INTERNALLY) for threading model in
     Libmicrohttpd to get better compatibility
   - http_server_port seized automatically using Libmicrohttpd function by
     passing MHD_DAEMON_INFO_BIND_PORT or MHD_DAEMON_INFO_LISTEN_FD parameter to
     MHD_get_daemon_info() 
   - Using iteration to parse urls data in answer_to_connection(). This
     guarantee we can pass any variadic data to Libmicrohttpd and prevent
     segmentation fault 
   - Fix answer_to_connection() function to create proper HTTP response:
     - Handle arguments (query string) on URL
     - Handle redirection
     - Handle chunked transfer encoding
     - Handle directory creation
     - Handle URL with IRI object
     - Handle URL with IDN hostname
     - Handle URL with query string which contains space
     - Handle If-Modified-Since header
   - Fix segmentation fault error when build on Fedora/Clang CI runner. This
     caused by former variable "http_server_tid" that forget to removed


================================= NEXT STEPS ===================================
Things which would be done in the coming week:

 * There are still problem occurred when running `make check-valgrind` on some
   tests. This prevent me to pass the CI test. I still found some difficulties
   to resolve this, so any helps are appreciated
 * Fix all remaining unfinished test covered by Libmicrohttpd code:
   - test-wget1.c: add capability for HTTP 406 Range Not Satisfiable
   - test-auth-basic.c: add basic authentication test
   - test-metalink.c: looping
   - test-i-https.c: add HTTPS test
 * Fix Libmicrohttpd depedency on Wget2, just required when need to run test
   suite


[1]: https://gitlab.com/dstw/wget2
[2]: https://gitlab.com/dstw/wget2/tree/use-mhd

Regards,
Didik Setiawan