Re: [Duplicity-talk] local_path.exists() fails when testing a new backend

[address@hidden]

Hi guys!

I've some good news :)

>From the first tests that me and Skylable team did, it looks like the
backend works as expected!
We were able to: make a full backup, incremental backup, list and restore!

We will make more tests in the next days, then I think I could prepare
a "pull request" or a patch for your original code base. If you have
any preference, please let me know :)

I'm going to write a blog post too, to explain how I implemented the
backend. Some informations are in your README, some other I gathered
from all the questions I asked to you. I think it would be nice for
other people to find them all in a single blog post as future
reference.

In the mean time thank you so much for all the help you provided!

Cheers.


On 10 July 2014 21:08, address@hidden <address@hidden> wrote:
> Thank you so much Michael!
>
> I've done a quick test and I'm not getting anymore the error I was
> getting last time I tried.
> I will need to do more tests with the help of some Skylable people
> (who know the original client better than me) and we will let you know
> as soon as possible.
>
> Thank you so much in the mean time, upgrading the branch really helped a lot!
>
> Cheers,
>
>
> On 10 July 2014 21:03, Michael Terry <address@hidden> wrote:
>> Devel and master are the same in this context.  And you have the right
>> branch, as well as the right revisions, specifically 992.  Please try your
>> test again and let me know if there is still a problem.
>>
>> -mt
>>
>>
>> On 10 July 2014 16:00, address@hidden <address@hidden> wrote:
>>>
>>> Hi,
>>>
>>> even if I'm working on a personal git repository, I kept your .bzr folder.
>>> So now I've done a "bzr pull" and this is what I have as latest commits:
>>>
>>> (duplicity)andrea-Inspiron-660:duplicity andrea [master] $ bzr log | more
>>> ------------------------------------------------------------
>>> revno: 995
>>> committer: Kenneth Loafman <address@hidden>
>>> branch nick: duplicity-src
>>> timestamp: Sat 2014-06-28 10:48:11 -0500
>>> message:
>>>   * Fixed bug 1327550: OverflowError: signed integer is greater than
>>> maximum
>>>     - Major and minor device numbers are supposed to be one byte each.
>>> Someone
>>>       has crafted a special system image using OpenVZ where the major and
>>> minor
>>>       device numbers are much larger (ploop devices).  We treat them as
>>> (0,0).
>>> ------------------------------------------------------------
>>> revno: 994 [merge]
>>> committer: Kenneth Loafman <address@hidden>
>>> branch nick: duplicity-src
>>> timestamp: Sat 2014-06-28 09:48:21 -0500
>>> message:
>>>   * Merged in lp:~antmak/duplicity/0.7-par2-fix
>>>     - Useful fix for verbatim par2cmdline options (like "-t" in
>>> par2-tbb version)
>>> ------------------------------------------------------------
>>> revno: 993
>>> committer: Kenneth Loafman <address@hidden>
>>> branch nick: duplicity-src
>>> timestamp: Sat 2014-06-28 09:45:09 -0500
>>> message:
>>>   Restore accidental deletion.
>>> ------------------------------------------------------------
>>> revno: 992 [merge]
>>> committer: Kenneth Loafman <address@hidden>
>>> branch nick: duplicity-src
>>> timestamp: Sat 2014-06-28 09:43:36 -0500
>>> message:
>>>   * Merged in lp:~mterry/duplicity/webdav-fixes
>>>     - This branch fixes two issues I saw when testing the webdav backend:
>>>     - 1) Errors like the following: "Attempt 1 failed. BackendException:
>>> File
>>>       /tmp/duplicity-LQ1a0i-tempdir/mktemp-u2aiyX-2 not found locally
>>> after get
>>>       from backend".  These were caused by the _get() method not
>>> calling setdata()
>>>       on the local path object, so the rest of the code thought it didn't
>>> exist.
>>>     - 2) Some odd issues from stale responses/data. We have a couple
>>> places in
>>>       webdavbackend.py where we close the connection before making a
>>> request
>>>       because of this problem. But I've changed it to do it every time,
>>> more
>>>       reliably, by putting a _close() call inside the request() method.
>>>     - With this, the webdav backend seems fine to me.
>>> ------------------------------------------------------------
>>>
>>> is this ok? How can I know in which branch I am? I almost forgot how
>>> to use bazaar :(
>>>
>>> This is what I get with bzr info:
>>>
>>> (duplicity)andrea-Inspiron-660:duplicity andrea [master] $ bzr info
>>> Standalone tree (format: 2a)
>>> Location:
>>>   branch root: .
>>>
>>> Related branches:
>>>   parent branch: bzr+ssh://bazaar.launchpad.net/+branch/duplicity/
>>>
>>> is this the devel branch or the master one?
>>>
>>> Thank you so much!
>>>
>>>
>>> On 10 July 2014 15:25,  <address@hidden> wrote:
>>> > it is a _development_ repository.. yes, please update your local branch.
>>> >
>>> > ..ede
>>> >
>>> > On 10.07.2014 16:24, address@hidden wrote:
>>> >> Hi,
>>> >>
>>> >> I don't remember exactly the version number that I'm using, but I took
>>> >> it recently (few weeks ago) from your upcoming 0.7 devel branch.
>>> >> I can update it again if you want, no problems.
>>> >>
>>> >> On 9 July 2014 23:45, Michael Terry <address@hidden> wrote:
>>> >>> Are you using at least r992 of trunk?  It fixed an issue with that
>>> >>> error
>>> >>> message.
>>> >>> -mt
>>> >>>
>>> >>>
>>> >>> On 9 July 2014 18:33, address@hidden <address@hidden> wrote:
>>> >>>>
>>> >>>> Hi guys,
>>> >>>>
>>> >>>> sorry if I ask again, but has anyone an idea why that backend call is
>>> >>>> failing?
>>> >>>> I would really appreciate your help. We are really close to complete
>>> >>>> the backend for Skylable client and this would be an additional
>>> >>>> opportunity also for Duplicity.
>>> >>>>
>>> >>>> Thank you so much for your help.
>>> >>>>
>>> >>>> Cheers.
>>> >>>>
>>> >>>>
>>> >>>> On 3 July 2014 21:32, address@hidden <address@hidden> wrote:
>>> >>>>> Hi guys,
>>> >>>>>
>>> >>>>> as you probably know I'm working on the Duplicity backend for
>>> >>>>> Skylable
>>> >>>>> service. I'm basing my work on your "devel" branch (0.7.x) and you
>>> >>>>> can
>>> >>>>> find my work in progress here:
>>> >>>>> https://github.com/andreagrandi/duplicity-sx
>>> >>>>>
>>> >>>>> Now, let's go to the issue. It looks like I've problems implementing
>>> >>>>> the _get method of my backend.
>>> >>>>>
>>> >>>>> On high level, this is the error I get when I execute a command like
>>> >>>>> this:
>>> >>>>>
>>> >>>>> PYTHONPATH=. ./bin/duplicity -v9
>>> >>>>> sx://indian.skylable.com/vol-andy80/
>>> >>>>> ./test-bkp
>>> >>>>>
>>> >>>>> output here: http://pastebin.com/JF4LuWXZ
>>> >>>>>
>>> >>>>> Debugging my code with ipdb I can see this:
>>> >>>>>
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> /home/andrea/Downloads/duplicity/duplicity/backends/sxbackend.py(39)_get()
>>> >>>>>      38         commandline = "sxcp {0} {1}".format(remote_path,
>>> >>>>> local_path.name)
>>> >>>>> ---> 39         self.subprocess_popen(commandline)
>>> >>>>>      40
>>> >>>>>
>>> >>>>> ipdb> n
>>> >>>>>> /home/andrea/Downloads/duplicity/duplicity/backend.py(541)get()
>>> >>>>>     540             self.backend._get(remote_filename, local_path)
>>> >>>>> --> 541             if not local_path.exists():
>>> >>>>>     542                 raise BackendException(_("File %s not found
>>> >>>>> locally after get "
>>> >>>>>
>>> >>>>> ipdb> local_path.exists()
>>> >>>>> ipdb> local_path
>>> >>>>> (() /tmp/duplicity-V8v6rE-tempdir/mktemp-0tsmWi-2 None)
>>> >>>>>
>>> >>>>> as you can see the .exists() method doesn't return anything, while
>>> >>>>> inspecting the object it contains a file that I've verified being
>>> >>>>> existing (the only two things that I don't understand are: is that
>>> >>>>> empty tumple at the beginning ok? It's the None at the end ok?):
>>> >>>>>
>>> >>>>> andrea-Inspiron-660:duplicity andrea [master] $ ls -al
>>> >>>>> /tmp/duplicity-V8v6rE-tempdir/mktemp-0tsmWi-2
>>> >>>>> -rw-rw-r-- 1 andrea andrea 4455795 lug  3 21:20
>>> >>>>> /tmp/duplicity-V8v6rE-tempdir/mktemp-0tsmWi-2
>>> >>>>>
>>> >>>>> The code of course fails because tha exists() fails:
>>> >>>>>
>>> >>>>>> /home/andrea/Downloads/duplicity/duplicity/backend.py(542)get()
>>> >>>>>     541             if not local_path.exists():
>>> >>>>> --> 542                 raise BackendException(_("File %s not found
>>> >>>>> locally after get "
>>> >>>>>     543                                          "from backend") %
>>> >>>>> util.ufn(local_path.name))
>>> >>>>>
>>> >>>>> Now my question is: why the .exists() fails if the path exists?
>>> >>>>> What's
>>> >>>>> wrong with my code?
>>> >>>>>
>>> >>>>> Thank you so much. Cheers.
>>> >>>>>
>>> >>>>> --
>>> >>>>> Andrea Grandi -  Software Engineer / Qt Ambassador / Nokia Developer
>>> >>>>> Champion
>>> >>>>> website: http://www.andreagrandi.it
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>>> --
>>> >>>> Andrea Grandi -  Software Engineer / Qt Ambassador / Nokia Developer
>>> >>>> Champion
>>> >>>> website: http://www.andreagrandi.it
>>> >>>>
>>> >>>> _______________________________________________
>>> >>>> Duplicity-talk mailing list
>>> >>>> address@hidden
>>> >>>> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>>> >>>
>>> >>>
>>> >>>
>>> >>> _______________________________________________
>>> >>> Duplicity-talk mailing list
>>> >>> address@hidden
>>> >>> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>>> >>>
>>> >>
>>> >>
>>> >>
>>> >
>>> > _______________________________________________
>>> > Duplicity-talk mailing list
>>> > address@hidden
>>> > https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>>>
>>>
>>>
>>> --
>>> Andrea Grandi -  Software Engineer / Qt Ambassador / Nokia Developer
>>> Champion
>>> website: http://www.andreagrandi.it
>>>
>>> _______________________________________________
>>> Duplicity-talk mailing list
>>> address@hidden
>>> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>>
>>
>>
>> _______________________________________________
>> Duplicity-talk mailing list
>> address@hidden
>> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>>
>
>
>
> --
> Andrea Grandi -  Software Engineer / Qt Ambassador / Nokia Developer Champion
> website: http://www.andreagrandi.it



-- 
Andrea Grandi -  Software Engineer / Qt Ambassador / Nokia Developer Champion
website: http://www.andreagrandi.it