Re: [Bug-wget] http server responding with 416 but file was not transferred completely

[Josef Moellers]

On 15.09.2017 09:34, Josef Moellers wrote:
> On 14.09.2017 17:06, Tim Rühsen wrote:
>> On 09/14/2017 12:11 PM, Josef Moellers wrote:
>>> On 14.09.2017 10:12, Tim Rühsen wrote:
>>>> On 09/14/2017 09:53 AM, Josef Moellers wrote:
>>>>> Hi,
>>>>>
>>>>> We have seen (at least) one server who has
>>>>> Accept-Ranges: bytes
>>>>> in his header but, upon receiving a request with
>>>>> Range: bytes=23068672-
>>>>> responds with
>>>>> HTTP/1.1 416 Requested Range Not Satisfiable
>>>>> although the file was not transferred completely!
>>>>>
>>>>> wget then claims that
>>>>> The file is already fully retrieved; nothing to do.
>>>>>
>>>>> E.g.
>>>>> run
>>>>>   wget https://downloads.dell.com/FOLDER02721216M/1/SUU_14.12.200.69.ISO
>>>>> the, after a couple of MB, abort the transfer and then continue the
>>>>> download:
>>>>>   wget --continue
>>>>> https://downloads.dell.com/FOLDER02721216M/1/SUU_14.12.200.69.ISO
>>>>>
>>>>> Maybe the check in src/http.c:
>>>>> 3821   if (statcode == HTTP_STATUS_RANGE_NOT_SATISFIABLE
>>>>> 3822       || (!opt.timestamping && hs->restval > 0 && statcode ==
>>>>> HTTP_STATUS_OK
>>>>> 3823           && contrange == 0 && contlen >= 0 && hs->restval >= 
>>>>> contlen))
>>>>>
>>>>> should be changed and any HTTP_STATUS_RANGE_NOT_SATISFIABLE with an
>>>>> incomplete file should show something like
>>>>>
>>>>> "download continue failed, file incomplete"
>>>>
>>>> Well, that would be ok for this special server.
>>>>
>>>> Normally 416 together with a server timestamp matching the file's
>>>> timestamp means that the file is complete (as far as the client can
>>>> judge from HTTP).
>>>>
>>>> IMO, if the server is broken (or misbehaves) then the server should be
>>>> repaired. Except it is a very common misbehavior. In which case we could
>>>> consider a work-around on the client side.
>>>>
>>>
>>> So I humbly propose the attached patch.
>>> I tried to create a pull request, but got a 403.
>>
>> Thanks for the patch - I'll test it in the next days.
> 
> Hold the press ...
> While the fix catches some mis-behaviour of the remote server, we have
> another issue:
> when "-N" is specified, "--continue" apparently does not contine when
> the file's modification date has not changed.

Maybe deciding that "timestamping" and "always_rest" are mutually
exclusive or disabling "timestamping" if "always_rest" is set?

Josef

signature.asc
Description: OpenPGP digital signature