[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Duplicity-talk] A duplicity backend for Amazon S3

From: Brian Sutherland
Subject: [Duplicity-talk] A duplicity backend for Amazon S3
Date: Thu, 13 Apr 2006 20:28:38 +0200
User-agent: Mutt/1.5.9i


I've written a backend for duplicity that writes to the Amazon S3
service [1]. It uses the bitbucket.py module [2].

It allows you to do something like this:

    duplicity / s3+http://${access_key}:${secret_key}/${bucket}

and so on to store you backups on S3. Which, well is very cheap. If
others start using this, it would be nice for the code to be included in

(Beware, bitbucket.py is a fairly young module and may have some bugs)

[1] http://www.amazon.com/gp/browse.html/103-1203077-2066220?node=16427261
[2] http://www.other10percent.com/?p=18

The relevant code from the duplicity.backend module is very simple:

class BitBucketBackend(Backend):

        def __init__(self, parsed_url):
                import bitbucket
                self.module = bitbucket
                parts = parsed_url.suffix.split('/')
                bucket_name = parts[-1]
                AccessKeyID, SecretAccessKey = '/'.join(parts[:-1]).split(':')
                self.bucket = self.module.BitBucket(bucket_name,

        def put(self, source_path, remote_filename = None):
                """Transfer source_path (Path object) to remote_filename 

                If remote_filename is None, get the filename from the last
                path component of pathname.

                if not remote_filename:
                    remote_filename = source_path.get_filename()
                bits = self.module.Bits(filename=source_path.name)
                self.bucket[remote_filename] = bits

        def get(self, remote_filename, local_path):
                """Retrieve remote_filename and place in local_path"""
                bits = self.bucket[remote_filename]

        def list(self):
                """Return list of filenames (strings) present in backend"""
                self.bucket.fetch_all_keys() # XXX I don't think this should be 
                return self.bucket.keys()

        def delete(self, filename_list):
                """Delete each filename in filename_list, in order if 
                for file in filename_list:
                    del self.bucket[file]

# Dictionary relating protocol strings to backend_object classes.
protocol_class_dict = {"scp": scpBackend,
                                           "ssh": scpBackend,
                                           "file": LocalBackend,
                                           "ftp": ftpBackend,
                                           "rsync": rsyncBackend,
                                           "s3+http": BitBucketBackend}

Brian Sutherland

Metropolis - "it's the first movie with a robot. And she's a woman.
              And she's EVIL!!"

reply via email to

[Prev in Thread] Current Thread [Next in Thread]