Re: [Duplicity-talk] Best Duplicity 8 backend for Google Drive?

[edgar . soldin]

no problem Peter,

you can consider using rclone too, 
http://duplicity.nongnu.org/vers8/duplicity.1.html#sect28 , which should be 
much easier to set up! ..ede/duply.net

On 18.08.2021 06:00, zga9uhnq4g--- via Duplicity-talk wrote:
> Thanks Edgar, for spending the time and writing up such a clear description.
>
> I started looking at the gdrive backend code in gitlab last night, and 
> finally managed to get it working at 1 AM this morning (before I went to 
> bed).  I had similar issues figuring out the Debian packages that provide the 
> google modules gdrive needs.  Didn't have time to reply to this thread until 
> now.
>
> The only thing I did differently is how I found the folder id.  If you 
> right-click on a folder in the Google Drive web page, you can select "Get 
> link" which brings up a dialog that allows you to copy the link.  I just 
> guessed that the part at the end was the ID, and discovered that it worked 
> (after I resolved a few other issues).
>
> My next step is to "replicate" my existing backup to Google Drive, and modify 
> my backup script to use Google Drive.
> After that I want to try to get the box backend working (my free box account 
> has more storage space than my free Google Drive account).
>
> Thanks again for your help.
>
>
> On 2021-08-17 02:38, edgar.soldin--- via Duplicity-talk 
> duplicity-talk-at-nongnu.org |duplicity-talk| wrote:
>> ok Peter,
>>
>> just wasted 2hs on this, because you got me interested :). essentially, the 
>> problem is twofold, with the first problem being the "bigger" issue
>>
>> issue1. duplicity does not provide a client API authentification. it has to 
>> be created by every user themselves on https://console.developers.google.com
>> issue2. figure out the url to use, as it is not mentioned in the man page
>>
>> if you managed to do all that _and_ pip3 installed really all (google*) 
>> modules duplicity complains about being needed, it works as expected
>>
>> so for issue1
>> 1. login to google drive in your browser
>> 2. go to https://console.developers.google.com
>> 3. create a project (!), switch to it
>> 4. Library -> add Google-Drive-API
>> 5. Credentials -> add OAuth-Client
>> 6. OAuth Consent screen -> publish (else you'll get 
>> https://stackoverflow.com/questions/65756266/error-403-access-denied-the-developer-hasn-t-given-you-access-to-this-app-despi)
>> 7. download "client_secret_<client_id>.apps.googleusercontent.com.json" and 
>> place it on your backup machine as '~/gdrive_client.json' (used below)
>>
>> wrt. to issue2
>> the url to use looks like
>>   gdrive://<client_id>.apps.googleusercontent.com/<subfolder_to place files 
>> in>?myDriveFolderID=<a_folder_id>
>> e.g.
>>   
>> gdrive://388783630241-02dsm3ijbnjnjen7266vpa4jv721mt61.apps.googleusercontent.com/test?myDriveFolderID=1v6P4LkVRhoNZOvN0R2VlR7tC55-HmroX
>>
>> you can get the folder id as described here 
>> https://ploi.io/documentation/database/where-do-i-get-google-drive-folder-id
>> didn't check if the subfolder is really needed btw.
>>
>> now you should be set and you can try running a backup like
>>   export GOOGLE_CLIENT_SECRET_JSON_FILE=~/gdrive_client.json
>>   export GOOGLE_CREDENTIALS_FILE=~/gdrive_creds
>>   duplicity-0.8.20/bin/duplicity temp/ 
>> gdrive://388783630241-02dsm3ijbnjnjen7266vpa4jv721mt61.apps.googleusercontent.com/test?myDriveFolderID=1v6P4LkVRhoNZOvN0R2VlR7tC55-HmroX
>>
>> it'll ask you to visit the url and provide a secret and fill 
>> '~/gdrive_creds' for subsequent runs.
>>
>> hope that helps.. ede/duply.net
>>
>> ps. @Ken - how about we provide a default duplicity client auth with 
>> duplicity distro to make life easier for users?
>>
>>
>> On 17.08.2021 05:19, zga9uhnq4g--- via Duplicity-talk wrote:
>>> Yes I did read it.  That's the "Note on GDrive Backend" section that I 
>>> mentioned I'm having trouble understanding.
>>>
>>> First, it talks about a "Google My Drive based shared folder" and I want to 
>>> use a Google My Drive regular (non-shared) folder (which Ken said should 
>>> work).
>>>
>>> Second, even though the "Note on GDrive Backend" section mentions using a 
>>> regular account (as an alternative to a "service account"), the example URL 
>>> in that section as well is the gdrive example in the URL Format section 
>>> only show "service account name" and "service account email address".  The 
>>> last sentence in the "Note on GDrive Backend" section seems to imply that 
>>> the OAuth client ID is alternative for the service email address, but 
>>> doesn't make it clear how/where to provide that client ID.
>>>
>>> In addition, the example URLS in the "URL Format" section and the "Note on 
>>> GDrive Backend" section look quite  different, so I'm not sure which format 
>>> to follow.
>>>
>>>      thanks,
>>>      Peter
>>>
>>> On 2021-08-16 19:41, edgar.soldin--- via Duplicity-talk 
>>> duplicity-talk-at-nongnu.org |duplicity-talk| wrote:
>>>> not familiar with gdrive, but did you read the man page section
>>>> http://duplicity.nongnu.org/vers8/duplicity.1.html#sect27 
>>>> <http://duplicity.nongnu.org/vers8/duplicity.1.html#sect27>
>>>> ?
>>>>
>>>> ..ede/duply.net
>>>>
>>>> On August 17, 2021 12:35:38 AM GMT+02:00, zga9uhnq4g--- via Duplicity-talk 
>>>> <duplicity-talk@nongnu.org> wrote:
>>>>
>>>>      Thanks for the info Ken.
>>>>
>>>>      A couple follow up questions.
>>>>
>>>>        * Is there a way to contact the GDrive backend developer with 
>>>> questions, or is this list the best way?
>>>>        * I'm having trouble understanding the Note on GDrive Backend, 
>>>> specifically what the gdrive:// URL should look like.  If my Google Drive 
>>>> account is john_doe123@gmail.com and I want the duplicity backup stored in 
>>>> the (non-shared) /backup/duplicity folder, can you  (or someone) tell me 
>>>> what the gdrive:// URL should look like?
>>>>
>>>>          thanks again,
>>>>          Peter
>>>>
>>>>
>>>>      On 2021-08-15 09:51, Kenneth Loafman kenneth-at-loafman.com 
>>>> |duplicity-talk| wrote:
>>>>>      Hi Peter,
>>>>>
>>>>>      All of our backends are written by volunteers, so I'm not the best 
>>>>> one to answer.
>>>>>
>>>>>        * gdocs is 10 years old and some still use it (to be deprecated in 
>>>>> 0.9.0).
>>>>>        * pydrive replaced gdocs 7 years ago and some still use it (to be 
>>>>> deprecated in 0.9.0).
>>>>>        * gdrive replaced pydrive 5 months ago.
>>>>>
>>>>>      As far as I know, all three still work, which is surprising since 
>>>>> gdocs is deprecated by Google.  I would suggest using gdrive.
>>>>>
>>>>>      A service account is for a team while a regular account is a normal 
>>>>> Google user.  I don't know any pros or cons.
>>>>>
>>>>>      No, My Drive folders do not need to be shared, they just need 
>>>>> different addressing than shared.
>>>>>
>>>>>      Sorry, that's all I can think of at the moment,  Hope it helps!
>>>>>
>>>>>      ...Ken
>>>>>
>>>>>
>>>>>      On Sun, Aug 15, 2021 at 12:23 AM zga9uhnq4g--- via Duplicity-talk 
>>>>> <duplicity-talk@nongnu.org <mailto:duplicity-talk@nongnu.org>> wrote:
>>>>>
>>>>>          Looking at the duplicity 8 man page, it appears that there are 
>>>>> several options for storing backups in Google Drive:
>>>>>
>>>>>            * gdocs://user[:password]@other.host/some_dir
>>>>>            * pydrive://<service account’ email 
>>>>> address>@developer.gserviceaccount.com/some_dir 
>>>>> <http://developer.gserviceaccount.com/some_dir>
>>>>>            * gdrive://<service account’ email 
>>>>> address>@developer.gserviceaccount.com/some_dir 
>>>>> <http://developer.gserviceaccount.com/some_dir>
>>>>>
>>>>>          Which of them should I expect to be the most reliable and 
>>>>> efficient?
>>>>>          Searching duplicity-talk, I saw some indication that the gdocs 
>>>>> API has been deprecated, which makes me leery of using it.  The man page 
>>>>> says "GDrive: is a rewritten PyDrive: backend ...", but I didn't see any 
>>>>> mention of gdrive in duplicity-talk, so I'm not sure how widely it is 
>>>>> being used.
>>>>>
>>>>>          I have a couple questions about the Notes on the Pydrive and 
>>>>> Gdrive backends in the man page:
>>>>>
>>>>>            * What are pros and cons of using a service account versus a 
>>>>> regular account?
>>>>>            * The Note on GDrive Backend only mentions using shared 
>>>>> folders for 'My Drive', does that mean I can't store a duplicity backup 
>>>>> in a non-shared folder?
>>>>>
>>>>>              thanks,
>>>>>              Peter
>>>>>
>>>>>          _______________________________________________
>>>>>          Duplicity-talk mailing list
>>>>>          Duplicity-talk@nongnu.org <mailto:Duplicity-talk@nongnu.org>
>>>>>          https://lists.nongnu.org/mailman/listinfo/duplicity-talk 
>>>>> <https://lists.nongnu.org/mailman/listinfo/duplicity-talk>
>>>>>
>>>>
>>>> _______________________________________________
>>>> Duplicity-talk mailing list
>>>> Duplicity-talk@nongnu.org
>>>> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>>>
>>> _______________________________________________
>>> Duplicity-talk mailing list
>>> Duplicity-talk@nongnu.org
>>> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>>>
>>
>> _______________________________________________
>> Duplicity-talk mailing list
>> Duplicity-talk@nongnu.org
>> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>
>
> _______________________________________________
> Duplicity-talk mailing list
> Duplicity-talk@nongnu.org
> https://lists.nongnu.org/mailman/listinfo/duplicity-talk