Re: [Duplicity-talk] 0.4.3-rc11 failure on FTP

[Nicolas Aspert]

Hello

Kenneth Loafman wrote:
> Yes, it is sequential: process, upload, repeat.  This saves space on the
> local system since only one chunk needs to be active at a time.  On my
> systems, it's roughly half and half, processing to uploading.  Assuming
> complete parallelization, we could almost halve the processing time for
> my case.  For yours, the limit would be processing time.
> 
>> I am them wondering if this is possible to improve this a bit by using
>> the spooling capacities of ncftp to avoid pausing in volume generation
>> (cf. ncftpspooler man page)
> 
> It would be.  I need to look into how we could track the progress of the
> uploads so we would know when to delete the chunks as they completed.
> The current duplicity would try to delete right after spooling since it
> would think the process was complete.

In this case the ncftpspooler has a convenient "delete" option that
should do the trick.

> This mechanism could then be generalized for all backends so that both
> puts and gets could be parallelized with a user settable limit on the
> amount of local storage to use.

Ideally yes but I suspect this is not too easy for all
backends/situations...

> One solution, if you have the space, is to backup locally using the file
> backend, then rsync to the backup server.  rsync has much better restart
> capabilities than anything I've seen so far.
> 

I am now a bit too short on disk space to perform this... but if things
get worse I'll think about giving this option a run :)

a+
Nicolas