Re: [Duplicity-talk] Simple full backup verify error

[Felix Natter]

hello Edgar,

thank you for your reply! My answers are inline.

On 24.01.23 11:51, edgar.soldin--- via Duplicity-talk wrote:

> hey Felix,
>       
>       
>       see inline
>       
>       
>       On 24.01.2023 11:19, Felix Natter via Duplicity-talk wrote:
>       
> > hello Duplicity community,
> >         
> >         
> >         I am running a simple daily backup (in this case: just one full
> >         backup with just one volume):
> >         
> >         
> >         export DUPLICITY_OPTS="--verbosity=debug --tempdir=/repos/temp
> >         --archive-dir=/repos/.duplicity-cache --volsize=28000
> >         --no-compression --encrypt-key MY_KEY"
> >         
> >         
> >         duplicity $DUPLICITY_OPTS --full-if-older-than 5D
> >         /repos/dumps.backup/ file:///mnt/FOO/daily/ |& tee
> >         /tmp/duplicity.log
> >         
> >       
>       
>       not sure that you need debug verbosity here already
>       
>

do I need to debug verify only or also fullbackup?

>       
> > which succeeds. Immediately afterwards, I
> >         run a verify including compare:
> >         
> >         
> >         duplicity verify $DUPLICITY_OPTS --compare-data
> >         file:///mnt/FOO/daily/ /repos/dumps.backup/ |& tee
> >         /tmp/duplicity.log
> >         
> >         
> >         which fails with many of these:
> >         
> >         
> >         Registering (mktemp) temporary file
> >         /repos/temp/duplicity-t168pkjr-tempdir/mktemp-d9mbcjxz-1
> >         
> >         Invalid data - SHA1 hash mismatch for file:
> >         
> >           duplicity-full.20230124T082316Z.vol1.difftar.gpg
> >         
> >           Calculated hash: ccab08e9879d9598b83945c83233a9978d954c67
> >         
> >           Manifest hash: cf642123945b4f86b376815c8b8f2ce92e0f2688
> >         
> >         
> >         Registering (mktemp) temporary file
> >         /repos/temp/duplicity-t168pkjr-tempdir/mktemp-gz1sto53-2
> >         
> >         Invalid data - SHA1 hash mismatch for file:
> >         
> >           duplicity-full.20230124T082316Z.vol1.difftar.gpg
> >         
> >           Calculated hash: ccab08e9879d9598b83945c83233a9978d954c67
> >         
> >           Manifest hash: cf642123945b4f86b376815c8b8f2ce92e0f2688
> >         
> >         
> >         [...]
> >         
> >       
>       
>       duplicity creates checksums during backup and these do not match
>       anymore, vulgo they have been corrupted.
>       
>       
> > /mnt/FOO is an NFS mounted share. The
> >         target directory and cache directory are fresh
> >         
> >         (just deleted before the backup job). I haven't deleted the temp
> >         folder though.
> >         
> >         I am using the latest duplicity: 1.2.1.
> >         
> >         
> >         Here is more log output that occurs before the above:
> >         
> >         
> >         3 file(s) exists on backend
> >         
> >         3 file(s) exists in cache
> >         
> >         Extracting backup chains from list of files:
> >         ['duplicity-full.20230124T082316Z.vol1.difftar.gpg',
> >         'duplicity-full-signatures.20230124T082316Z.sigtar.gpg',
> >         'duplicity-full.20230124T082316Z.manifest.gpg']
> >         
> >         File duplicity-full.20230124T082316Z.vol1.difftar.gpg is not
> >         part of a known set; creating new set
> >         
> >         File duplicity-full-signatures.20230124T082316Z.sigtar.gpg is
> >         not part of a known set; creating new set
> >         
> >         Ignoring file (rejected by backup set)
> >         'duplicity-full-signatures.20230124T082316Z.sigtar.gpg'
> >         
> >         Processing local manifest
> > b'/repos/.duplicity-cache/956a308cc4cd0300f1ebbd0e0b7865c1/duplicity-full.20230124T082316Z.manifest'
> >         (4835)
> >         
> >         Found manifest volume 1
> >         
> >         Found 1 volumes in manifest
> >         
> >         File duplicity-full.20230124T082316Z.manifest.gpg is part of
> >         known set
> >         
> >         Found backup chain [Tue Jan 24 09:23:16 2023]-[Tue Jan 24
> >         09:23:16 2023]
> >         
> >         Local and Remote metadata are synchronized, no sync needed.
> >         
> >         Last full backup date: Tue Jan 24 09:23:16 2023
> >         
> >         Collection Status
> >         
> >         -----------------
> >         
> >         Connecting with backend: BackendWrapper
> >         
> >         Archive dir:
> >         /repos/.duplicity-cache/956a308cc4cd0300f1ebbd0e0b7865c1
> >         
> >         
> >         Found 0 secondary backup chain(s).
> >         
> >         
> >         Found primary backup chain with matching signature chain:
> >         
> >         -------------------------
> >         
> >         Chain start time: Tue Jan 24 09:23:16 2023
> >         
> >         Chain end time: Tue Jan 24 09:23:16 2023
> >         
> >         Number of contained backup sets: 1
> >         
> >         Total number of contained volumes: 1
> >         
> >           Type of backup set:                            Time:      Num
> >         volumes:
> >         
> >                          Full         Tue Jan 24 09:23:16
> >         2023                 1
> >         
> >         -------------------------
> >         
> >         No orphaned or incomplete backup sets found.
> >         
> >         Processing local manifest
> > b'/repos/.duplicity-cache/956a308cc4cd0300f1ebbd0e0b7865c1/duplicity-full.20230124T082316Z.manifest'
> >         (4835)
> >         
> >         Found manifest volume 1
> >         
> >         Found 1 volumes in manifest
> >         
> >         Processing local manifest
> > b'/repos/.duplicity-cache/956a308cc4cd0300f1ebbd0e0b7865c1/duplicity-full.20230124T082316Z.manifest'
> >         (4835)
> >         
> >         Found manifest volume 1
> >         
> >         Found 1 volumes in manifest
> >         
> >         Using temporary directory /repos/temp/duplicity-t168pkjr-tempdir
> >         
> >         Registering (mktemp) temporary file
> >         /repos/temp/duplicity-t168pkjr-tempdir/mktemp-d9mbcjxz-1
> >         
> >         
> >         So what am I missing?
> >         
> >       
>       
>       just tried locally with another key but essentially your
>       DUPLICITY_OPTS and duplicity 1.2.1 verified fine.
>       
>     
    We are using a total size of 22Gb, did you try with large files?
>       
>       how about you backup/verify to/from a local file:// target first
>       (e.g. /tmp/test.backup/) and work your way up from there. this way
>       you make sure everything is working and then you can start
>       debugging the NFS issues.
>       
>

Locally everything is ok! Can I add more than --verbosity=debug logging? Do the NFS mount
parameters make a difference? The problem does not occur every time, i.e. sometimes it works.

sidact-nas2:/volume1/export/Backup/svn    /mnt/NAS2-SVN    nfs    rw,vers=3,soft,bg,intr    0    0

I am now trying with the 'sync' nfs mount option.

Many Thanks and Best Regards,

Felix

>       
>       good luck ..ede/duply.net
>       
>       
>       _______________________________________________
>       
>       Duplicity-talk mailing list
>       
>       Duplicity-talk@nongnu.org
>       
>       https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>       
>

--

SIDACT GmbH
Simulation Data Analysis and
Compression Technologies

Felix Natter
Software Developer

Auguststraße 29
53229 Bonn
Germany

Phone	:   +49 228 5348 0430
Direct	:   +49 228 4097 7118
Email	:   felix.natter@sidact.com
Web	:   http://www.sidact.com/

OpenPGP_signature
Description: OpenPGP digital signature