Re: [Duplicity-talk] gpg key password asked for backup after verify

[edgar . soldin]

Ken,

On 20.07.2017 14:03, Kenneth Loafman wrote:
> Hi ede,
> 
> OK, I implemented it so it will output a non-fatal error message on gpg fails 
> with remote manifest files, return None, and continue.  I ignored the code 
> from the previous message because the try/except was removed completely.  
> Tests are passing and I'll get it released soon.  

it's a start, but we need to decide if this warrants a fatal failure or not! 
as long as there is no way to compare the remote manifest w/ the local one w/o 
decryption i'd say it is. actually it is already fatal to anyone not using an 
english locale ;)

btw. my suggestion was to make it fatal in 0.8 as the version raise might 
warrant major changes.

generally we need to realize that backing up to dumb backends either needs 
decryption for sync from time to time or implement a way to signal both 
location's files are identical. how to do that safely w/o proper 
decryption/signing is beyond me. 
that's why i say, duplicity usage needs at least one secret key/message and if 
it is only a secret machine key used to sign stuff.

> We need to continue this discussion on the list

done. cc'd the list.

>We keep introducing potentially fatal issues.

exactly where do we "keep introducing" them? :)

sunny greetings ..ede/duply.net


> 
> ...Ken
> 
> 
> On Wed, Jul 19, 2017 at 4:44 AM, <address@hidden <mailto:address@hidden>> 
> wrote:
> 
>     Ken,
> 
>     because now the try/except has a condition that leads it _not to_ fail at 
> all [1], but to silently return no manifest, even though there is one, but it 
> cannot be decrypted w/o the secret key.
> 
>     seen?.. ede
> 
>     [1]  when gpg spits out "secret key not available" in english
> 
>     On 18.07.2017 23:02, Kenneth Loafman wrote:
>     > ede,
>     >
>     > I still don't understand... removing the exception does exactly nothing 
> except cause it to fail in a different and more confusing manner, so why 
> bother?
>     >
>     > ...Ken
>     >
>     >
>     > On Tue, Jul 18, 2017 at 3:40 PM, <address@hidden 
> <mailto:address@hidden> <mailto:address@hidden <mailto:address@hidden>>> 
> wrote:
>     >
>     >     Ken,
>     >
>     >     kind of academic, but ok,
>     >
>     >     1. the local cache might have been deleted
>     >     2. it might have been tampered with
>     >     3. some file system condition might have reverted the cache to a 
> previous state
>     >     4. ...
>     >
>     >     stuff like that. the solution to use a local machine secret/public 
> key seems to be much more appealing than the assumption that we are 
> synchronized in case we do not have a secret key to decrypt.
>     >
>     >     also agn, we will need the secret key in case of resuming anyway or 
> the backup will fail indefinitely until a new chain is started or a key is 
> given.
>     >
>     >     ..ede
>     >
>     >     On 7/18/2017 10:32 PM, Kenneth Loafman wrote:
>     >     > ede,
>     >     >
>     >     > It's going to raise an error if it can't read/decrypt the 
> manifest, either
>     >     > in a handled exception, or an unhandled one.
>     >     >
>     >     > How could duplicity write to a backend with a more recent 
> manifest?  The
>     >     > lockfile guarantees there is only one writer at a time.
>     >     >
>     >     > ...Ken
>     >     >
>     >     >
>     >     > On Tue, Jul 18, 2017 at 3:23 PM, <address@hidden 
> <mailto:address@hidden> <mailto:address@hidden <mailto:address@hidden>>> 
> wrote:
>     >     >
>     >     >> Ken,
>     >     >>
>     >     >> how do you imagine to resolve this cleanly w/o raising an error? 
> the
>     >     >> moment a user writes a backup to a backend where a more recent 
> manifest
>     >     >> already resides, the chain is corrupted, no?
>     >     >>
>     >     >> ..ede
>     >     >>
>     >     >> On 7/18/2017 10:18 PM, Kenneth Loafman wrote:
>     >     >>> ede,
>     >     >>>
>     >     >>> Without going back through this convoluted email, why are we 
> removing the
>     >     >>> exception rather than fixing it?  gpg has defined return codes 
> and it
>     >     >> seems
>     >     >>> that would be the way to go?
>     >     >>>
>     >     >>> ...Ken
>     >     >>>
>     >     >>> On Tue, Jul 18, 2017 at 10:07 AM, <address@hidden 
> <mailto:address@hidden> <mailto:address@hidden <mailto:address@hidden>>> 
> wrote:
>     >     >>>
>     >     >>>> you could edit it manually
>     >     >>>>   >collections.py#get_remote_manifest() L232
>     >     >>>>
>     >     >>>>     def get_remote_manifest(self):
>     >     >>>>         """
>     >     >>>>         Return manifest by reading remote manifest on backend
>     >     >>>>         """
>     >     >>>>         assert self.remote_manifest_name
>     >     >>>>         manifest_buffer = self.backend.get_data(self.
>     >     >> remote_manifest_name)
>     >     >>>>         log.Info(_("Processing remote manifest %s (%s)") %
>     >     >>>> (self.remote_manifest_name, len(manifest_buffer)))
>     >     >>>>         return manifest.Manifest().from_string(manifest_buffer)
>     >     >>>>
>     >     >>>> sufficient? ..ede
>     >     >>>>
>     >     >>>> On 18.07.2017 17:01, Kenneth Loafman wrote:
>     >     >>>>> Hi ede,
>     >     >>>>>
>     >     >>>>> I thought this was fixed earlier?
>     >     >>>>>
>     >     >>>>> Could you provide a "bzr diff collections.py > 
> collections.py.patch".
>     >     >> I
>     >     >>>> don't need a branch for something like this.
>     >     >>>>>
>     >     >>>>> ...Thanks,
>     >     >>>>> ...Ken
>     >     >>>>>
>     >     >>>>>
>     >     >>>>> On Tue, Jul 18, 2017 at 9:13 AM, <address@hidden 
> <mailto:address@hidden> <mailto:address@hidden <mailto:address@hidden>> 
> <mailto:
>     >     >>>> address@hidden <mailto:address@hidden> <mailto:address@hidden 
> <mailto:address@hidden>>>> wrote:
>     >     >>>>>
>     >     >>>>>     hey Ken,
>     >     >>>>>
>     >     >>>>>     do you need a branch? or can you remove it on the fly? 
> afaics the
>     >     >>>> hack is still in there
>     >     >>>>>       
> http://bazaar.launchpad.net/%7Eduplicity-team/duplicity/0 
> <http://bazaar.launchpad.net/%7Eduplicity-team/duplicity/0> 
> <http://bazaar.launchpad.net/%7Eduplicity-team/duplicity/0 
> <http://bazaar.launchpad.net/%7Eduplicity-team/duplicity/0>>.
>     >     >>>> 8-series/annotate/head%3A/duplicity/collections.py#L241 <
>     >     >>>> http://bazaar.launchpad.net/%7Eduplicity-team/duplicity/0 
> <http://bazaar.launchpad.net/%7Eduplicity-team/duplicity/0> 
> <http://bazaar.launchpad.net/%7Eduplicity-team/duplicity/0 
> <http://bazaar.launchpad.net/%7Eduplicity-team/duplicity/0>>.
>     >     >>>> 8-series/annotate/head%3A/duplicity/collections.py#L241>
>     >     >>>>>
>     >     >>>>>     ..ede
>     >     >>>>>
>     >     >>>>>     -------- Forwarded Message --------
>     >     >>>>>     Subject: Re: [Duplicity-talk] gpg key password asked for 
> backup
>     >     >>>> after verify
>     >     >>>>>     Date: Mon, 29 May 2017 14:08:14 +0200
>     >     >>>>>     From: edgar.soldin--- via Duplicity-talk <
>     >     >> address@hidden <mailto:address@hidden> <mailto:address@hidden 
> <mailto:address@hidden>>
>     >     >>>> <mailto:address@hidden <mailto:address@hidden> 
> <mailto:address@hidden <mailto:address@hidden>>>>
>     >     >>>>>     Reply-To: Discussion about duplicity backup <
>     >     >>>> address@hidden <mailto:address@hidden> <mailto:address@hidden 
> <mailto:address@hidden>> <mailto:address@hidden <mailto:address@hidden> 
> <mailto:address@hidden <mailto:address@hidden>>>>
>     >     >>>>>     To: Kenneth Loafman <address@hidden 
> <mailto:address@hidden> <mailto:address@hidden <mailto:address@hidden>> 
> <mailto:
>     >     >> address@hidden <mailto:address@hidden> <mailto:address@hidden 
> <mailto:address@hidden>>>>,
>     >     >>>> Discussion about duplicity backup <address@hidden 
> <mailto:address@hidden> <mailto:address@hidden <mailto:address@hidden>> 
> <mailto:
>     >     >>>> address@hidden <mailto:address@hidden> <mailto:address@hidden 
> <mailto:address@hidden>>>>
>     >     >>>>>
>     >     >>>>>     nP.. as written, 0.8 would be a good chance to simply 
> remove
>     >     >>>> "missing secret key" exception as a first step, as we can 
> argue backward
>     >     >>>> compatibility breakage _but_ more importantly backup security
>     >     >> improvement!
>     >     >>>>>
>     >     >>>>>     ..ede
>     >     >>>>>
>     >     >>>>>     On 29.05.2017 14:05, Kenneth Loafman wrote:
>     >     >>>>>     > ede,
>     >     >>>>>     >
>     >     >>>>>     > Thanks for the links.  I completely forgot about all 
> that.  Yes,
>     >     >>>> we need to fix it.
>     >     >>>>>     >
>     >     >>>>>     > ...Ken
>     >     >>>>>     >
>     >     >>>>>     >
>     >     >>>>>     > On Mon, May 29, 2017 at 6:40 AM, edgar.soldin--- via
>     >     >>>> Duplicity-talk <address@hidden <mailto:address@hidden> 
> <mailto:address@hidden <mailto:address@hidden>> <mailto:address@hidden 
> <mailto:address@hidden> <mailto:address@hidden <mailto:address@hidden>>
>     >     >> .
>     >     >>>> org> <mailto:address@hidden <mailto:address@hidden> 
> <mailto:address@hidden <mailto:address@hidden>> <mailto:address@hidden 
> <mailto:address@hidden> <mailto:address@hidden <mailto:address@hidden>>.
>     >     >> org>>>
>     >     >>>> wrote:
>     >     >>>>>     >
>     >     >>>>>     >     Ken,
>     >     >>>>>     >
>     >     >>>>>     >     On 29.05.2017 13:26, Kenneth Loafman wrote:
>     >     >>>>>     >     > 2) I'm not seeing that we ignore errors in the 
> sync between
>     >     >>>> local and remote.  That would produce bad backups if we did.
>     >     >>>>>     >
>     >     >>>>>     >     that's where you are wrong ;)
>     >     >>>>>     >       
> http://bazaar.launchpad.net/~duplicity-team/duplicity/0.8- 
> <http://bazaar.launchpad.net/%7Eduplicity-team/duplicity/0.8-> 
> <http://bazaar.launchpad.net/%7Eduplicity-team/duplicity/0.8- 
> <http://bazaar.launchpad.net/%7Eduplicity-team/duplicity/0.8->>
>     >     >>>> series/annotate/head%3A/duplicity/collections.py#L241 <
>     >     >>>> http://bazaar.launchpad.net/%7Eduplicity-team/duplicity/0 
> <http://bazaar.launchpad.net/%7Eduplicity-team/duplicity/0> 
> <http://bazaar.launchpad.net/%7Eduplicity-team/duplicity/0 
> <http://bazaar.launchpad.net/%7Eduplicity-team/duplicity/0>>.
>     >     >>>> 8-series/annotate/head%3A/duplicity/collections.py#L241> <
>     >     >>>> http://bazaar.launchpad.net/%7Eduplicity-team/duplicity/0 
> <http://bazaar.launchpad.net/%7Eduplicity-team/duplicity/0> 
> <http://bazaar.launchpad.net/%7Eduplicity-team/duplicity/0 
> <http://bazaar.launchpad.net/%7Eduplicity-team/duplicity/0>>.
>     >     >>>> 8-series/annotate/head%3A/duplicity/collections.py#L241 <
>     >     >>>> http://bazaar.launchpad.net/%7Eduplicity-team/duplicity/0 
> <http://bazaar.launchpad.net/%7Eduplicity-team/duplicity/0> 
> <http://bazaar.launchpad.net/%7Eduplicity-team/duplicity/0 
> <http://bazaar.launchpad.net/%7Eduplicity-team/duplicity/0>>.
>     >     >>>> 8-series/annotate/head%3A/duplicity/collections.py#L241>>
>     >     >>>>>     >
>     >     >>>>>     >     i found it in 2010 in the more detailed thread 
> about this
>     >     >> issue
>     >     >>>>>     >       
> http://thread.gmane.org/gmane.comp.sysutils.backup 
> <http://thread.gmane.org/gmane.comp.sysutils.backup> 
> <http://thread.gmane.org/gmane.comp.sysutils.backup 
> <http://thread.gmane.org/gmane.comp.sysutils.backup>>.
>     >     >>>> duplicity.general/4245 <http://thread.gmane.org/
>     >     >>>> gmane.comp.sysutils.backup.duplicity.general/4245> <
>     >     >>>> http://thread.gmane.org/gmane.comp.sysutils.backup 
> <http://thread.gmane.org/gmane.comp.sysutils.backup> 
> <http://thread.gmane.org/gmane.comp.sysutils.backup 
> <http://thread.gmane.org/gmane.comp.sysutils.backup>>.
>     >     >> duplicity.general/4245
>     >     >>>> <http://thread.gmane.org/gmane.comp.sysutils.backup 
> <http://thread.gmane.org/gmane.comp.sysutils.backup> 
> <http://thread.gmane.org/gmane.comp.sysutils.backup 
> <http://thread.gmane.org/gmane.comp.sysutils.backup>>.
>     >     >> duplicity.general/4245
>     >     >>>>>>
>     >     >>>>>     >
>     >     >>>>>     >     ..ede
>     >     >>>>>     >
>     >     >>>>>     >     _______________________________________________
>     >     >>>>>     >     Duplicity-talk mailing list
>     >     >>>>>     >     address@hidden <mailto:address@hidden> 
> <mailto:address@hidden <mailto:address@hidden>> <mailto:address@hidden 
> <mailto:address@hidden> <mailto:address@hidden <mailto:address@hidden>>>
>     >     >>>> <mailto:address@hidden <mailto:address@hidden> 
> <mailto:address@hidden <mailto:address@hidden>> <mailto:address@hidden 
> <mailto:address@hidden> <mailto:address@hidden <mailto:address@hidden>>>>
>     >     >>>>>     >     
> https://lists.nongnu.org/mailman/listinfo/duplicity-talk 
> <https://lists.nongnu.org/mailman/listinfo/duplicity-talk> 
> <https://lists.nongnu.org/mailman/listinfo/duplicity-talk 
> <https://lists.nongnu.org/mailman/listinfo/duplicity-talk>> <
>     >     >>>> https://lists.nongnu.org/mailman/listinfo/duplicity-talk 
> <https://lists.nongnu.org/mailman/listinfo/duplicity-talk> 
> <https://lists.nongnu.org/mailman/listinfo/duplicity-talk 
> <https://lists.nongnu.org/mailman/listinfo/duplicity-talk>>> <
>     >     >>>> https://lists.nongnu.org/mailman/listinfo/duplicity-talk 
> <https://lists.nongnu.org/mailman/listinfo/duplicity-talk> 
> <https://lists.nongnu.org/mailman/listinfo/duplicity-talk 
> <https://lists.nongnu.org/mailman/listinfo/duplicity-talk>> <
>     >     >>>> https://lists.nongnu.org/mailman/listinfo/duplicity-talk 
> <https://lists.nongnu.org/mailman/listinfo/duplicity-talk> 
> <https://lists.nongnu.org/mailman/listinfo/duplicity-talk 
> <https://lists.nongnu.org/mailman/listinfo/duplicity-talk>>>>
>     >     >>>>>     >
>     >     >>>>>     >
>     >     >>>>>
>     >     >>>>>
>     >     >>>>>     _______________________________________________
>     >     >>>>>     Duplicity-talk mailing list
>     >     >>>>>     address@hidden <mailto:address@hidden> 
> <mailto:address@hidden <mailto:address@hidden>> <mailto:address@hidden 
> <mailto:address@hidden> <mailto:address@hidden <mailto:address@hidden>>>
>     >     >>>>>     https://lists.nongnu.org/mailman/listinfo/duplicity-talk 
> <https://lists.nongnu.org/mailman/listinfo/duplicity-talk> 
> <https://lists.nongnu.org/mailman/listinfo/duplicity-talk 
> <https://lists.nongnu.org/mailman/listinfo/duplicity-talk>> <
>     >     >>>> https://lists.nongnu.org/mailman/listinfo/duplicity-talk 
> <https://lists.nongnu.org/mailman/listinfo/duplicity-talk> 
> <https://lists.nongnu.org/mailman/listinfo/duplicity-talk 
> <https://lists.nongnu.org/mailman/listinfo/duplicity-talk>>>
>     >     >>>>>
>     >     >>>>>
>     >     >>>>
>     >     >>>>
>     >     >>>
>     >     >>
>     >     >>
>     >     >
>     >
>     >
> 
>