Re: [Bug-wget] Metalink support

[Hubert Tarasiuk]

Here is a small fix to Metalink support in main function.
The bug was that if the Metalink file parsing failed, there would be a
free() call for non-initialized memory pointer.

W dniu 24.06.2015 o 08:06, Hubert Tarasiuk pisze:
> I have prepared a Metalink patch for Wget. The two main features are:
> 
> - support Metalink v3 and Metalink v4 XML files:
> https://tools.ietf.org/html/rfc5854
> using libmetalink parser: https://launchpad.net/libmetalink
> 
> - support Metalink in HTTP headers: http://tools.ietf.org/html/rfc6249
> 
> 
> Specifically what the patch implements (for both XML and HTTP):
> - keep downloading from consecutive resource URLs until a successful
> retrieval
> - verify SHA256 digest (this digest is mandatory for Metalink, thus it
> should be included in all Metalink documents)
> - verify OpenPGP signatures (using public keys from user's keyring)
> using GPGME and GnuPG
> 
> Checksum mismatch means download failure and it proceeds to try with
> another resource (if available).
> If the signature cannot be verified (missing public key would be the
> most common reason), we do NOT assume download failure.
> If the signature can be verified and the verification fails (ie. data
> does not match signature), we assume download failure.
> 
> Please note that the PGP signatures are only working for
> Metalink-over-HTTP at this time due to a bug in libmetalink.
> 
> 
> Following options were added to Wget:
> 
> --input-metalink=FILE - download files described in Metalink file FILE
> (like --input-file)
> 
> --metalink-over-http - when downloading from HTTP URLs:
> -> issue a HEAD request and check for Metalink metadata in reponse
> -> if found: switch to Metalink-mode
> -> if not found: fall back to ordinary HTTP download
> 
> _Test suite_
> I have made two modifications to Python test suite:
> - allow multiple SendHeaders with same name by using a Python list as
> dictionary value
> - do not start the HTTP test in constructor; do it in the begin() method
> instead (as the method name would suggest); original behaviour was to
> run the test in object constructor and the begin() method would just
> return the result
> 
> Please let me know what do you think about the patches. Some test cases
> are included. If you would like to test it on actual servers, here is
> what I found:
> - Metalink files with PGP signatures: http://curl.haxx.se/download.html
> - Metalink in HTTP headers:
> https://download.gnome.org/apps/3.0/3.0.0/sources/
> 
> The commits are also available via Github interface:
> https://github.com/jy987321/Wget/commits/metalink
> 
> Hubert
> 
> W dniu 28.05.2015 o 00:49, Hubert Tarasiuk pisze:
>> I have talked with Giuseppe and he suggested that we might not do TCP
>> Fast Open support for FTP at this time (he argued that FTP is slow
>> either way :).
>>
>> Instead I might focus on implementing some basics of Metalink protocol
>> for HTTP and FTP resources in Wget.
>>
>> Do you have any thoughts about that?
>>

0001-Do-not-free-Metalink-structure-if-not-initialized.patch
Description: Text Data

signature.asc
Description: OpenPGP digital signature