[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Was "[Bug-wget] using wget to mirror based on filesize only not"
|
From: |
TPCwget |
|
Subject: |
Was "[Bug-wget] using wget to mirror based on filesize only not" |
|
Date: |
Fri, 19 Nov 2010 20:02:20 +0000 (GMT) |
>
> The ftp server is actually a data logger which has a Compact Flash card and
> an ftp style structure.
>
> When I download the files they look like this:
>
> Name Size Type Date Modified
> 01-00001.NDF 19kb NDF File 1/11/2010 12:00AM
> 02-00001.NDF 19kb NDF File 2/11/2010 12:00AM
> 03-00001.NDF 5kb NDF File 3/11/2010 12:00AM
>
> As data is written to the file eg. 03-00001.NDF the file size grows
> throughout the day reach a size of 19kb but the "Date Modified" date remains
> at when it was created 3/11/2010 12:00AM
This reminded me of a similar problem I have. I have a data logger box
(ABB SM1000) which runs a version of Windows CE. Its FTP server produces
a directory listing format which wget does not understand, and hence 'wget
-N' does not work -- that is the datestamps from the box are not preserved.
Here is an example using wget 1.10.2 with the command,
wget -N -v --no-remove-listing --server-response --no-passive-ftp -r
ftp://username:address@hidden/
--------------------------------------------------------------------------------------------------------------------------------------------
--18:59:21-- ftp://username:address@hidden/
=> `phsm1000-1.rhul.ac.uk/.listing'
Resolving phsm1000-1.rhul.ac.uk... 134.219.128.77
Connecting to phsm1000-1.rhul.ac.uk|134.219.128.77|:21... connected.
Logging in as username ...
220 WinCE GkWare FTP Service (Version 1.3 Jun 1 2004)
--> USER username
331 OK, password required
--> PASS Turtle Power!
230 OK
--> SYST
215 Windows_NT (compatible) / Windows CE
--> PWD
257 "/" is current directory.
--> TYPE I
200 TYPE set to I.
==> CWD not needed.
--> PORT 134,219,128,55,157,229
200 PORT command successful.
--> LIST
150 Opening BINARY mode data connection for /bin/ls.
0K 9.76 KB/s
226 Transfer complete.
01-01-1998 12:00PM <DIR> Flash_Disk
01-01-1998 12:00PM <DIR> Storage_Card
08-06-2010 08:25AM <DIR> www
08-06-2010 08:25AM <DIR> My Documents
08-06-2010 08:25AM <DIR> Program Files
08-06-2010 08:25AM <DIR> Temp
08-06-2010 08:25AM <DIR> Windows
18:59:22 (9.76 KB/s) - `phsm1000-1.rhul.ac.uk/.listing' saved [348]
phsm1000-1.rhul.ac.uk/Flash_Disk: corrupt time-stamp.
phsm1000-1.rhul.ac.uk/Storage_Card: corrupt time-stamp.
phsm1000-1.rhul.ac.uk/www: corrupt time-stamp.
phsm1000-1.rhul.ac.uk/My Documents: corrupt time-stamp.
phsm1000-1.rhul.ac.uk/Program Files: corrupt time-stamp.
phsm1000-1.rhul.ac.uk/Temp: corrupt time-stamp.
phsm1000-1.rhul.ac.uk/Windows: corrupt time-stamp.
--18:59:22-- ftp://username:address@hidden/Flash_Disk/
=> `phsm1000-1.rhul.ac.uk/Flash_Disk/.listing'
--> CWD /Flash_Disk
250 CWD command successful.
--> PORT 134,219,128,55,206,193
200 PORT command successful.
--> LIST
150 Opening BINARY mode data connection for /bin/ls.
0K . 13.46 KB/s
226 Transfer complete.
11-19-2010 07:00PM 31220 AELog.lgf
10-31-2010 02:00AM 2724 TotLog.lgf
11-19-2010 07:01PM 30020 AudLog.lgf
12-23-2008 08:56PM 20 BatLog.lgf
12-23-2008 08:55PM 30776 Company standard.cfg
11-05-2010 07:34PM 30776 SM1000.cfg
12-23-2008 08:55PM 30776 QSDemo.cfg
12-23-2008 08:55PM 30776 QSMilliamp.cfg
12-23-2008 08:55PM 30776 QSFlow.cfg
12-23-2008 08:55PM 30776 QSTHC_C.cfg
12-23-2008 08:55PM 30776 QSTHC_F.cfg
12-23-2008 08:55PM 30776 QSRTD_C.cfg
12-23-2008 08:55PM 30776 QSRTD_F.cfg
12-23-2008 08:55PM 30776 QSDairy.cfg
12-23-2008 08:55PM 80 TotalizerCode.ts
12-23-2008 08:55PM 80 MLCode.mls
11-19-2010 07:01PM 88612 CH1_1_Analogue.dat
11-19-2010 07:01PM 88612 CH1_2_Analogue.dat
11-19-2010 07:01PM 88612 CH1_3_Analogue.dat
11-19-2010 07:01PM 88612 CH1_4_Analogue.dat
11-19-2010 07:01PM 88612 CH1_5_Analogue.dat
11-19-2010 07:01PM 88612 CH1_6_Analogue.dat
11-05-2010 07:34PM 12 info.dps
12-23-2008 08:56PM 80 BCode.mls
11-19-2010 07:01PM 2400 SavedData
05-20-2009 04:28PM 30776 17_28_13 200509 SM2000.cfg
11-06-2009 04:53PM 30776 16_53_12 061109 SM2000.cfg
11-19-2010 07:01PM 88612 CH2_1_Analogue.dat
11-19-2010 07:01PM 88612 CH2_2_Analogue.dat
11-19-2010 07:01PM 88612 CH2_3_Analogue.dat
11-19-2010 07:01PM 88612 CH2_4_Analogue.dat
11-19-2010 07:01PM 88612 CH2_5_Analogue.dat
11-19-2010 07:01PM 88612 CH2_6_Analogue.dat
11-06-2009 10:23PM 30776 22_23_18 061109 SM2000.cfg
01-15-2010 05:23PM 30776 17_23_17 150110 SM2000.cfg
11-05-2010 07:34PM 30776 19_34_02 051110 SM2000.cfg
18:59:24 (13.46 KB/s) - `phsm1000-1.rhul.ac.uk/Flash_Disk/.listing' saved [2032]
--18:59:24-- ftp://username:address@hidden/Flash_Disk/AELog.lgf
=> `phsm1000-1.rhul.ac.uk/Flash_Disk/AELog.lgf'
--> CWD /Flash_Disk
250 CWD command successful.
--> PORT 134,219,128,55,193,190
200 PORT command successful.
--> RETR AELog.lgf
150 Opening BINARY mode data connection for AELog.lgf
Length: 31,220 (30K)
0K .......... .......... .......... 100% 34.93 KB/s
226 RETR command successful.
18:59:25 (34.93 KB/s) - `phsm1000-1.rhul.ac.uk/Flash_Disk/AELog.lgf' saved
[31220]
phsm1000-1.rhul.ac.uk/Flash_Disk/AELog.lgf: corrupt time-stamp.
--------------------------------------------------------------------------------------------------------------------------------------------
I downloaded the latest wget v.1.12 and repeated the test which fails as
follows,
--------------------------------------------------------------------------------------------------------------------------------------------
--2010-11-19 19:05:16-- ftp://username:address@hidden/
=> `phsm1000-1.rhul.ac.uk/.listing'
Resolving phsm1000-1.rhul.ac.uk (phsm1000-1.rhul.ac.uk)... 134.219.128.77
Connecting to phsm1000-1.rhul.ac.uk
(phsm1000-1.rhul.ac.uk)|134.219.128.77|:21... connected.
Logging in as username ...
220 WinCE GkWare FTP Service (Version 1.3 Jun 1 2004)
--> USER username
331 OK, password required
--> PASS Turtle Power!
230 OK
--> SYST
215 Windows_NT (compatible) / Windows CE
--> PWD
257 "/" is current directory.
--> TYPE I
200 TYPE set to I.
==> CWD not needed.
--> PORT 134,219,128,55,195,129
200 PORT command successful.
--> LIST -a
150 Opening BINARY mode data connection for /bin/ls.
0K 0.00 =0s
226 Transfer complete.
2010-11-19 19:05:18 (0.00 B/s) - `phsm1000-1.rhul.ac.uk/.listing' saved [0]
--2010-11-19 19:05:18-- ftp://username:address@hidden/
=> `phsm1000-1.rhul.ac.uk/index.html'
==> CWD not required.
--> SIZE
502 Command not implemented/recognized
--> PORT 134,219,128,55,131,36
200 PORT command successful.
--> RETR
425 OpenLocalFileForRead failed.
Error in server response, closing control connection.
Retrying.
--------------------------------------------------------------------------------------------------------------------------------------------
Is there a workaround for wget-1.12 to make downloading work and any chance
(presuming not already implemented) of incorporating the box's FTP directory
listing format into wget?
Many thanks
Tom Crane.
--
Tom Crane, Dept. Physics, Royal Holloway, University of London, Egham Hill,
Egham, Surrey, TW20 0EX, England.
Email: address@hidden
Fax: +44 (0) 1784 472794