Re: [Mldonkey-users] Disk space preallocation

[Pierre Etchemaite]

Le Tue, 6 May 2003 13:03:35 +0200, Markus Hitter <address@hidden> a écrit :

> 3) After deleting .ini files and a temp recovery MLDonkey now thinks 
> these files, broken due to short disk space, are valid and 
> re-calculated all those md4's. Worse, it starts to send out those bad 
> chunks as being valid[1].

I don't think so, at least here's the expected "path of trust":

* you get the file hash and size from the ed2k:// link. Hash is stored in
the temp file name, size is stored in the temp "file size" (the _filesystem
metadata_ of the temp file)

* you get the file subhashes from another peer, after giving it the file
hash. Since the file hash is the hash of the concatenated subhashes, you
have a way to check that the subhashes are correct (at least, as much as you
can trust the MD4 algorithm)

* each time a chunk is fully received, its hash is computed. If it matches
the known subhash, it's validated. Otherwise, it's invalidated. Hashes from
actual on-disk data are *never stored* for further use.

* MLdonkey checks that it never writes over validated chunks

* metadata (files.ini) is flushed periodically to disk. Since the downloads
are almost monotonous, the only danger is to forget that some data was
received between the flush and the crash. The only time when the download is
not monotonous is at the moment a chunk is invalidated. Going back to
previous state (chunk partially downloaded) cannot do harm, as the chunk
will be rechecked when completed.

* if you lose metadata, you should be able to recover file hash and size
from the temp file, then the subhashes from a peer, then the completed
chunks by recomputing chunks hashes. All what you should lose are partial
chunks (That's why dropping files.ini is still a bad thing to do if you have
no real reason to do so).

> Currently, I get errors like "BAD BAD BAD: number of chunks is 
> different 15/9 for D2FB464884C3D8003A083807B3B9D9DC:77828096 on peer" 
> and "217.126.70.178 is suspicious of level 1; client 86: block 17 
> corrupted"

It's not the subhashes that are wrong, but the number of subhashes. Since
the number of subhashes is computed from the temp file size, my guess is
that you lost the filesystem "file size" for the temp files. Possibly
because your archiver/whatever doesn't preserve those for spare files if the
last byte wasn't written ?
That should either be workarounded, or documented.

Also, 17 is both bigger than 15 and 9. If I'm not mistaken, and that it's
really a chunk number, I hope those messages aren't related to the same
file, otherwise it means you've done a *terrible* mess with mldonkey
assumptions, and I'm sure you could bypass any hardening... ;)

BR,
Pierre.