[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: CRITICAL BUG: wget -N is leaving corrupted files
From: |
Derek Martin |
Subject: |
Re: CRITICAL BUG: wget -N is leaving corrupted files |
Date: |
Wed, 12 Jun 2024 14:14:13 -0400 |
On Sat, Jun 08, 2024 at 07:21:14PM +0200, Tim Rühsen wrote:
> What other options do we have to make --if-modified-since workable in your
> scenario? (Apart from switching --if-modified-since off)
>
> a) When you download a file, use a temporary file name. After wget exists,
> check the return status and if it is 0, rename the file.
>
> The downside is that you always have download the file, even if it didn't
> change.
I think this is probably the right solution, except:
1. ALWAYS rename the file, even if the download fails / is
interrupted.
2. BEFORE the rename, set the timestamp appropriately:
- set it to the original local file's timestamp if the transfer did
not complete successfully
- set it to the upstream file's timestamp if it did complete
successfully.
3. To successfully do that for the most possible cases, you'll need to
catch signals and delay their handling until the above is done, in
addition to whatever other error handling is already required.
And probably:
4. Document that in cases where clean-up procedures can't catch every
last case, temporary files may be left behind, so the user can expect
them on errors and manually clean them up. Probably also name the
temporary file something like ${original_file_name}_tmp.XXXXXX so that
the user can, if they so choose, rename it to ${original_file_name}
and manually reset the time stamp to get wget to resume/redownload or
whatever.