[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Bug-wget] wget fails to parse existing .listing files for FTP download
From: |
Mark |
Subject: |
[Bug-wget] wget fails to parse existing .listing files for FTP download |
Date: |
Tue, 2 Oct 2012 12:49:26 +0100 |
User-agent: |
SquirrelMail/1.4.21 |
Hi,
I'm using wget 1.13.4 and have come across a problem. Does this happen for
anyone else?
Situation:
- Use wget to recursively mirror some directories on an FTP site. Use
--no-remove-listing so the .listing files generated for each directory
are preserved.
- Part-way through the downloading process, break wget.
- Later come back and resume mirroring using the same wget command.
- wget fails to parse the already-downloaded .listing files and because
of that does not download any more files. (It actually tries to download
an empty/null filename.)
I'm using a command like this:
wget -x -S --no-clobber --recursive --level=inf --debug
--no-remove-listing --append-output=wget_log.txt --progress=dot:mega
"ftp://ftp.example.com/path/to/files/"
Would a workaround for the not-able-to-continue-mirroring problem be to
use --continue and not --no-clobber? Then wget should re-download the
.listing files, and because the already-downloaded files are all fully
downloaded, it should skip downloading them again.
Here's some example output.
DEBUG output created by Wget 1.13.4 on linux-gnu.
URI encoding = `UTF-8'
Using `ftp.microsoft.com/.listing' as listing tmp file.
File `ftp.microsoft.com/.listing' already there; not retrieving.
UNKNOWN; perms 0;
Skipping.
UNKNOWN; perms 0;
Skipping.
UNKNOWN; perms 0;
Skipping.
UNKNOWN; perms 0;
Skipping.
UNKNOWN; perms 0;
Skipping.
UNKNOWN; perms 0;
Skipping.
UNKNOWN; perms 0;
Skipping.
UNKNOWN; perms 0;
Skipping.
UNKNOWN; perms 0;
Skipping.
UNKNOWN; perms 0;
Skipping.
UNKNOWN; perms 0;
Skipping.
UNKNOWN; perms 0;
Skipping.
UNKNOWN; perms 0;
Skipping.
--2012-10-02 12:28:21-- ftp://ftp.microsoft.com/
=> `ftp.microsoft.com/index.html'
Resolving ftp.microsoft.com (ftp.microsoft.com)... 64.4.30.33
Caching ftp.microsoft.com => 64.4.30.33
Connecting to ftp.microsoft.com (ftp.microsoft.com)|64.4.30.33|:21...
connected.
Created socket 4.
Releasing 0x0000000001a9e430 (new refcount 1).
Logging in as anonymous ...
220 Microsoft FTP Service
--> USER anonymous
331 Anonymous access allowed, send identity (e-mail name) as password.
--> PASS Turtle Power!
230-Welcome to FTP.MICROSOFT.COM. Also visit
http://www.microsoft.com/downloads.
230 User logged in.
--> SYST
215 Windows_NT
--> PWD
257 "/" is current directory.
--> TYPE I
200 Type set to I.
==> CWD not needed.
--> SIZE
501 Invalid number of parameters.
conaddr is: 64.4.30.33
--> PASV
227 Entering Passive Mode (64,4,30,33,57,113).
trying to connect to 64.4.30.33 port 14705
Created socket 5.
--> RETR
501 Invalid number of parameters.
No such file `'.
Closed fd 5
Closed fd 4
The already-present .listing file in question is this (wget was able to
parse it the first time):
04-28-10 07:21PM <DIR> bussys
04-28-10 10:17PM <DIR> deskapps
04-28-10 11:14PM <DIR> developr
04-28-10 11:15PM <DIR> KBHelp
04-28-10 11:15PM <DIR> MISC
04-29-10 06:54AM <DIR> MISC1
04-29-10 08:18AM <DIR> mscomtest
04-29-10 08:47AM <DIR> peropsys
04-29-10 05:10PM <DIR> Products
04-29-10 05:13PM <DIR> PSS
04-29-10 05:22PM <DIR> ResKit
04-29-10 07:51PM <DIR> Services
04-30-10 08:37AM <DIR> Softlib
-- Mark
- [Bug-wget] wget fails to parse existing .listing files for FTP download,
Mark <=