[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Duplicity-talk] Restoring from PCA Backend - download preparation

From: erwan.vallee
Subject: [Duplicity-talk] Restoring from PCA Backend - download preparation
Date: Sun, 27 Jun 2021 12:04:52 +0200



I’m using duplicity with multibackend (public cloud archive and object storage from ovh), as described in this documentation from ovh :


I’ve worked on a modification to make PCA Backend work better (for my usage at least !) : allow to unseal all volumes at once when restoring content from PCA, wait for unseal to complete for all volumes before launching download. I’ve used API documentation as mentioned here :



This is a similar change to what has been implemented some time ago for Glacier cold storage :


I’ve implemented the following things available in this commit : https://gitlab.com/erwanBoka/duplicity/-/commit/822e42ba3c26a5b32377e238a9b6040b9b087052

  • Implement in BackendWrapper and MultiBackend pre_process_download (and redirect to sub-backends)
  • Implement for pcabackend pre_process_download with ovh API as a blocking call, waiting for all volumes to be unsealed.


However, I’m a bit confused by what was implemented then (new hook pre_process_download_batch), compared to existing one pre_process_download.

  • pre_process_download is called from dup_main/restore_get_patched_rop_iter with all filenames as parameters
  • it’s implemented only once in _boto_single, but with a single filename as argument…
  • pre_process_download_batch is essentially doing the same thing (call with all volume names)

I’m afraid my modifications in BackendWrapper and MultiBackend might cause pre_process_download to be called for _boto_single, and break things…


I haven’t modified anything yet in _boto_single, but my proposal would be :

  • change pre_process_download method name in _boto_single to an « internal » method used only in _get
  • keep only pre_process_download hook in dup_main
  • implement it the way pre_process_download_batch is implemented today in _boto_single

I don’t have s3 access, and cannot test it though.


Let me know your view about this, and the path forward please !

Best regards,






reply via email to

[Prev in Thread] Current Thread [Next in Thread]