Re: [Monotone-devel] Rarely mentioned 25-50% efficiency boost for monotone ; -)

[Nathaniel Smith]

On Fri, Apr 15, 2005 at 12:02:14PM +0200, Christof Petig wrote:
> Hi,
> 
> since monotone's performance had been a topic on this list for some days
> and nobody else mentioned this fact I will do it:
> 
> Monotone both internally and externally (database) still uses
> hexadecimal or base64 encoded data while sqlite3 already supports BLOBs.
> It also does not use query parameters (which are needed by BLOBs), but
> merges data and commands into a big SQL statement.
> 
> There is still preliminary work on prepared statements in the
> n.v.m.sqlite3 branch by Derek Scherger which has seen neither review nor
>   further work (IIRC).

Do you have any data supporting this 25-50% efficiency increase?
Obviously I'm interested if it's true :-), but I like to see profiling
data to support such ideas before suggesting people put work into
them...

(You do get a space improvement, of about 25%; I don't think we're IO
bound enough for this to matter for speed, but perhaps there are cases
where we are.)

Saving 25% disk space is also nice, of course, but there still is the
original motivation for using base64: that it's nice that you can poke
around at the database, copy/paste data pulled out of it, etc.,
without having to take precautions against binary data being blatted
out on your screen... all in all, I can imagine deciding that the
space is more important, but it doesn't seem like an urgent question?

> Migrating the internal representation to binary (instead of hex) IDs
> would be an enourmous effort (IIRC) but storing certs and data plain
> instead of base64 encoded should be easy to accomplish.
>
> Another benefit would be the easier interface to cert data via human
> initiated SQL.

It's already trivial to get decoded data with 'db execute' -- just use
the unpack() and unbase64() functions.

-- Nathaniel

-- 
So let us espouse a less contested notion of truth and falsehood, even
if it is philosophically debatable (if we listen to philosophers, we
must debate everything, and there would be no end to the discussion).
  -- Serendipities, Umberto Eco