[Monotone-devel] Re: nvm.experiment.db-compaction

[Markus Schiltknecht]

Hi,

Zack Weinberg wrote:
> Yeah, I was just looking at that myself.
>
> Rather than add lots and lots of en/decode_hexenc calls all over the
> place, I was thinking that it would be better to make operator<< and
> operator% (for formatting) on 'id' automatically convert to hexenc,
> and the 'id' constructor automatically convert from hexenc when it
> detects it (the hexenc string will be twice as long as the binary
> string, so it's unambiguous).

I thought about that as well, but it would circumvent the type system 
somewhat. I'm not sure if we want that.

> Doing this of course runs us afoul of vocab_macros ... I want to
> dynamite the entire vocab system and start over,

How would that look like?

> but that's hard, and
> hopefully not necessary for this work.  Perhaps your ATOMIC_BINARY
> approach would work.

I've been under the impression, that we sometimes want the binary 
representation. For example for netsync, but at least in database.cc we 
now need the binary representation. Just using id::inner() to get that 
seems rather confusing and doesn't help with type safety either.

Also keep in mind, that the original verify() routine for id's checked 
for length 40. Where I now check for length 20 for ids, but length 40 
for hexenc<id>s. Without that distinction, this verification would not 
be possible, and I'd have an even harder time finding the mismatches.

> >  What I wanted
> >  was to disallow operator<< for a new type ATOMIC_BINARY, but still allow
> >  (read: define) it for normal ATOMIC and ATOMIC_NOVERIFY types.
>
> Well, see above, but you probably need something not unlike the
> default template for dump() [in base.hh] visible in the header if you
> want to get a compiler error rather than a linker error.

What I didn't understand is, that it is operator<<, which the linker 
complains about. But the compiler went trough it all *without* having 
such an operator defined for id. I did even check the .ii file: no such 
operator. Is there some kind of implicit declarations those? Or does 
template instantiation of the DECORATE implicitly declare them?

> Right, my bet is that more often than not we wind up not needing to
> hex encode the majority of them, and thus it will be an unambiguous
> speed win.

Yeah, that'd be nice.

> On the longer scale I'd like to think about binary encoding of
> revisions and rosters, with an isomorphism to the text format for user
> presentation, but having the hash calculation be over the binary form

That would be nice as well, but lets take one step after another ;-)

> ... unfortunately it would break all signatures.

Uh, not necessarily, I think. We could just bump the format_version, and 
keep compatibility code in there, no?

Regards

Markus