Re: [Duplicity-talk] FTP not working

[Kenneth Loafman]

Kenneth Loafman wrote:
> Tim Riemenschneider wrote:
>> Tim Riemenschneider schrieb:
>>> Hi,
>>>
>>> I think I have a problem with the ftp-backend. collection-status shows
>>> as empty, but there are backups on the server. I think ncftpls is not
>>> working correctly.
>>>   
>> Ok, trying further revealed the problem: the ftp-server seems to choke
>> on the default options of ls, since the first call of ls (long listing)
>> works, but the second doesn't:
>> So, after hacking ftpbackend.py:
>> --- ftpbackend.py.orig  2009-04-02 17:51:52.000000000 +0200
>> +++ ftpbackend.py       2009-04-03 13:33:50.000000000 +0200
>> @@ -109,7 +109,7 @@
>>          if not l:
>>              return l
>>          # if long list is not empty, get short list of names only
>> -        commandline = "ncftpls %s '%s'" % \
>> +        commandline = "ncftpls %s -x '' '%s'" % \
>>              (self.flags, self.url_string)
>>          l = self.popen_persist(commandline).split('\n')
>>          l = [x.split()[-1] for x in l if x]
>>
>> Can my change break something? (Otherwise it could probably go into the
>> source?)
> 
> No, it looks like the removal of "-x ''" from the original source broke
> duplicity for your case, and fixed it in others.  There was a bug this
> closed, but it looks like another solution is needed now.
> 
> Oh, what I wouldn't give for standardization in FTP servers...

Here is a patch that will work for, I think, all the FTP servers we know
about so far.  It's probably better than the two-pass approach to list
that we use now since it gets all its names on the first pass.  Please
give this a try and let me know if it works.  I'll get it into the next
release if it does.

...Thanks,
...Ken

Index: duplicity/backends/ftpbackend.py
===================================================================
RCS file: /sources/duplicity/duplicity/duplicity/backends/ftpbackend.py,v
retrieving revision 1.19
diff -u -r1.19 ftpbackend.py
--- duplicity/backends/ftpbackend.py    2 Apr 2009 14:47:16 -0000       1.19
+++ duplicity/backends/ftpbackend.py    3 Apr 2009 14:20:39 -0000
@@ -22,6 +22,7 @@
 import os
 import os.path
 import urllib
+import re
 
 import duplicity.backend
 from duplicity import globals
@@ -84,6 +85,11 @@
         if parsed_url.port != None and parsed_url.port != 21:
             self.flags += " -P '%s'" % (parsed_url.port)
 
+        if not globals.short_filenames:
+            self.filename_re = re.compile("duplicity-[^\s]+")
+        else:
+            self.filename_re = re.compile("(?:df|di|dfs|dns)\\.[^\s]+")
+
     def put(self, source_path, remote_filename = None):
         """Transfer source_path to remote_filename"""
         remote_path = 
os.path.join(urllib.unquote(self.parsed_url.path.lstrip('/')), 
remote_filename).rstrip()
@@ -102,18 +108,26 @@
     def list(self):
         """List files in directory"""
         # try for a long listing to avoid connection reset
-        commandline = "ncftpls %s -l '%s'" % \
-            (self.flags, self.url_string)
+        commandline = "ncftpls %s -l '%s'" % (self.flags, self.url_string)
         l = self.popen_persist(commandline).split('\n')
         l = filter(lambda x: x, l)
         if not l:
             return l
-        # if long list is not empty, get short list of names only
-        commandline = "ncftpls %s '%s'" % \
-            (self.flags, self.url_string)
-        l = self.popen_persist(commandline).split('\n')
-        l = [x.split()[-1] for x in l if x]
-        return l
+        # we have a non-empty list, look for our files
+        names = []
+        for x in l:
+            try:
+                parts = x.split()
+            except:
+                continue
+            # The name is either the first or the last part
+            # of the line.  Try the last part first since
+            # that's the most common.
+            if self.filename_re.match(parts[-1]):
+                names.append(parts[-1])
+            elif self.filename_re.match(parts[0]):
+                names.append(parts[0])
+        return names
 
     def delete(self, filename_list):
         """Delete files in filename_list"""

signature.asc
Description: OpenPGP digital signature