Re: Refreshing a mirror throws 'Cannot write to .... (Is a directory)' error, when the directory is alreary downloaded

[Tim Rühsen]

Hi Boris,

I would leave -R "index.html" away. The "index.html" are helpful when
running wget multiple times.

The directory errors come from Location: headers within 301 redirections.

[
explanation:
The scanned URL says 'subdir' and when asking the server for it, we get
a redirection 301 to 'subdir/'. Wget keeps with 'subdir' as a mitigation
for https://www.cvedetails.com/cve/CVE-2010-2252/.
]

To switch this security feature off, use --trust-server-names.

Regards, Tim

On 11/26/19 11:39 AM, Boris Behrens wrote:
> Am Di., 26. Nov. 2019 um 11:30 Uhr schrieb Tim Rühsen <address@hidden>:
>> If you mirror into an existing directory, you normally only want to
>> update changes files. Like keeping the server's and your copy in sync.
>>
> Ah ok, that is not what I want.
> 
>> If wget now creates .<N> files each time you sync, the files will stuff
>> up - even if they didn't change on the server. This is normally not what
>> a user wants. So, what do *you* exactly want ?
>>
> As I expect that the files do not change, but that there will be added
> more files I would like to just download the files that are not
> present.
> Downloading changed (in this case indicated by a timestamp from the
> web server) files would be nice but no need for me.
> 
>> Maybe I was wrong with --clobber, another try would be --no-timestamping
>> after --mirror.
> Already did this in the first tests.
> Here are the options I already tried:
> WGETOPTIONS='--mirror --no-parent -R "index.html*"
> --no-host-directories --cut-dirs=1'
> WGETOPTIONS='--mirror --timestamp --recursive --no-parent -R
> "index.html*" --no-host-directories --cut-dirs=1'
> WGETOPTIONS='--mirror --no-parent -R "index.html"
> --no-host-directories --cut-dirs=1 --no-clobber'
> WGETOPTIONS='--mirror --no-parent -R "index.html"
> --no-host-directories --cut-dirs=1'
> WGETOPTIONS='--mirror --no-parent -R "index.html*"
> --no-host-directories --cut-dirs=1'
> WGETOPTIONS='--mirror --no-parent -x -R "index.html"
> --no-host-directories --cut-dirs=1'
> WGETOPTIONS='--mirror --no-parent -x -R "index.html*"
> --no-host-directories --cut-dirs=1'
> WGETOPTIONS='--no-host-directories --cut-dirs=1 -N -r'
> WGETOPTIONS='--no-parent --no-host-directories --cut-dirs=1 -N -r -c -l inf'
> WGETOPTIONS='--no-parent -R "index.html" --no-host-directories
> --cut-dirs=1 --no-clobber -N'
> WGETOPTIONS='--no-parent -R "index.html" --no-host-directories
> --cut-dirs=1 --no-clobber'
> WGETOPTIONS='--no-parent -R "index.html" --no-host-directories
> --cut-dirs=1 -N -r'
> WGETOPTIONS='-r -N -l inf --no-parent -R "index.html*"
> --no-host-directories --cut-dirs=1'
> WGETOPTIONS='-r -N -l inf --no-parent -R "index.html*"
> --no-host-directories --cut-dirs=1'
> WGETOPTIONS='-r -N -l inf --no-parent -R "robots.txt,index.html*"
> --no-host-directories --cut-dirs=1'
> 
> Cheers
>  Boris
>

signature.asc
Description: OpenPGP digital signature