Re: [Duplicity-talk] Out of space error while restoring a file

[edgar . soldin]

On 29.09.2012 09:50, Laurynas Biveinis wrote:
> 2012/9/26  <address@hidden>:
>> On 26.09.2012 11:42, Laurynas Biveinis wrote:
>> [SNIP]
>>>>>
>>>>>>> So, "twice the size of the restored file + one volume?" But originally
>>>>>>> it failed with 80GB free for a 30GB file, so it's even more than that.
>>>>>>
>>>>>> right, we would have to disect filesystem usage during restoring to find 
>>>>>> out what is going on here. actually the v9 dump tells you when temp 
>>>>>> files are created and deleted (though not their size) so you can work 
>>>>>> through that if you want.
>>>>>>
>>>>>> alternatively you have to observe what happens in the tmp folder during 
>>>>>> restore ;)
>>>>>
>>>>> Here's the open file snapshot around volume 704 (out of ~750):
>>>>> http://pastebin.com/pGFH6Wju
>>>>>
>>>>> Note that "Sandėlys" has been replaced with "Elements", that's another
>>>>> volume (so that locally-downloaded backup sets would fit)
>>>>>
>>>>> Thanks again,
>>>>>
>>>>
>>>> ok, just wanted to check the command line output promised above, just to 
>>>> see there was nothing attached ;)
>>>
>>> Oops. Fixed.
>>>
>>>> wrt to the lsof output. it shows
>>>> duplicity 8946 laurynas    7u   REG   8,33 25211305984     8622 
>>>> /media/Elements/Laurynas/tmp/duplicity-DL3ypc-tempdir/tmpgxn0pY (deleted)
>>>> and
>>>> duplicity 8946 laurynas   12u   REG   8,33 34465812480     8614 
>>>> /media/Elements/Laurynas/tmp/duplicity-DL3ypc-tempdir/tmpYZgzig (deleted)
>>>>
>>>> and some others, all of them have the tag (deleted), so i assume the space 
>>>> was already freed again?
>>>
>>> Nope, it's taken space.
>>>
>>>> even if not this is what we assumed: 2 times the 30GB file plus some.
>>>>
>>>> what would be interesting is a "multiplexed" output of 'duplicity -v9' 
>>>> _and_ and a periodic 'ls -la' of TMP . so one could see what files 
>>>> duplicity creates/deletes and how this reflects in the TMP folder. i 
>>>> believe this would be possible with bash magic.
>>>
>>> I'l see about producing this. But, ls -la will not show those big
>>> deleted files. I will try to include lsof as well, perhaps filtered to
>>> remove open net sockets and the like.
>>>
>>
>> first off, the v9 output does not show any tmp files named like tmpYZgzig 
>> but only mktemp-MRoS1d-741 ones. so i assume the default python tempfile 
>> does not inform us
>>
>> or
>>
>> you are not using the latest patch?
>> http://bazaar.launchpad.net/~duplicity-team/duplicity/0.6-series/revision/860
> 
> I am using the latest patch but not the latest duplicity code: I
> patched Ubuntu-shipped 0.6.18. Is that a good version to test or do
> you want me to use the latest trunk or any other version?

should be fine for the issue at hand

>> regarding the used space. did you monitor via du/ls/df the duplicity tmp 
>> folder and can confirm that it uses up much more than the 2xbiggest file 
>> plus some? how much?
> 
> Will do.
> 

please.. and come back with results. ede/duply.net