I’m using duplicity with multibackend (public cloud archive and object storage from ovh), as described in this documentation from ovh :
I’ve worked on a modification to make PCA Backend work better (for my usage at least !) : allow to unseal all volumes at once when restoring content from PCA, wait for unseal to complete for all volumes before launching download. I’ve used API documentation as mentioned here :
This is a similar change to what has been implemented some time ago for Glacier cold storage :
I’ve implemented the following things available in this commit : https://gitlab.com/erwanBoka/duplicity/-/commit/822e42ba3c26a5b32377e238a9b6040b9b087052
- Implement in BackendWrapper and MultiBackend pre_process_download (and redirect to sub-backends)
- Implement for pcabackend pre_process_download with ovh API as a blocking call, waiting for all volumes to be unsealed.
However, I’m a bit confused by what was implemented then (new hook pre_process_download_batch), compared to existing one pre_process_download.
- pre_process_download is called from dup_main/restore_get_patched_rop_iter with all filenames as parameters
- it’s implemented only once in _boto_single, but with a single filename as argument…
- pre_process_download_batch is essentially doing the same thing (call with all volume names)
I’m afraid my modifications in BackendWrapper and MultiBackend might cause pre_process_download to be called for _boto_single, and break things…
I haven’t modified anything yet in _boto_single, but my proposal would be :
- change pre_process_download method name in _boto_single to an « internal » method used only in _get
- keep only pre_process_download hook in dup_main
- implement it the way pre_process_download_batch is implemented today in _boto_single
I don’t have s3 access, and cannot test it though.
Let me know your view about this, and the path forward please !