Re: [Bug-wget] Timestamp behaviour with modified local files

[Ander Juaristi]

Hi Ian,

On 07/28/2015 07:32 AM, Ian Wienand wrote:
> Hi,
>
> The manual says
>
>    "If the local file does not exist, or the sizes of the files do not
>     match, Wget will download the remote file no matter what the
>     time-stamps say."
>
> In two cases I'm not seeing this:
>
> 1) With if-modified-since I don't believe the content-length is
>     checked at all
This is tricky. The behaviour is correct since the draft encourages to test the 
conditions before serving any request. From section 14.35.2 ("Range Retrieval 
Requests"):

        The presence of a Range header in a conditional GET (...) modifies what
        is returned if the GET is otherwise successful and the
        condition is true. It does not affect the 304 (Not Modified)
        response returned if the conditional is false.

Thus, if the content hasn't been changed, the server just acts as if no Range 
header was sent.

But IMO this is a problem, since the If-Modified-Since header is always sent by 
default and '-c' is a widely used option. What's more, I'm not really sure now, 
but I think Wget asserts '-c' automatically after a previous run that got 
cancelled with Ctrl-C. I'm afraid this issue you describe might often cause 
unexpected behaviour.

To me, the only sensible solution seems to be not to send If-Modified-Since 
when resuming downloads. Because if you send a conditional GET and the 
condition is met, the server will go no further.

> 2) Without if-modified-since, if the remote end returns a 416 we don't
>     re-download if the file-on-disk is larger than the remote end.
Just thinking loudly... Maybe If-Range would be a solution here?

What do you think?

Regards,
- AJ