[Monotone-devel] Encountered a segfault on monotone pull 8-O

[Christof Petig]

I just wanted to tell you that after 81 minutes (on a 3GHz HT P4!) a
pull repeatably causes a segfault (I didn't believe it the first time):

monotone: [bytes in: 20289752] [bytes out: 812358] [certs in: 9415]
[revs in: 2543]
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 1024 (LWP 13085)]
0x0839d616 in CryptoPP::HuffmanEncoder::GenerateCodeLengths (
    codeBits=0xbfe02860, maxCodeBits=15, codeCounts=0x12539424, nCodes=286)
    at cryptopp/secblock.h:251
251                     {assert(index >= 0 && (unsigned int)index <
m_size); return m_ptr[index];}

Unfortunately gdb does not love me here:
(gdb) print index
$1 = {<text variable, no debug info>} 0x4000cf50 <strchr>
(gdb) print m_size
No symbol "m_size" in current context.
(gdb) print m_ptr
No symbol "m_ptr" in current context.

(perhaps -O0 would help with that, but 81 minutes is not something you
easily try again without optimization)

the backtrace is interesting though:
#0  0x0839d616 in CryptoPP::HuffmanEncoder::GenerateCodeLengths (
    codeBits=0xbfe02860, maxCodeBits=15, codeCounts=0x12539424, nCodes=286)
    at cryptopp/secblock.h:251
#1  0x083a0d97 in CryptoPP::Deflator::EncodeBlock (this=0x12539250,
    blockType=2) at cryptopp/secblock.h:235
#2  0x083a1899 in CryptoPP::Deflator::EndBlock (this=0x12539250, eof=true)
    at cryptopp/zdeflate.cpp:746
#3  0x083a06a0 in CryptoPP::Deflator::Put2 (this=0x12539250, str=0x0,
    length=0, messageEnd=-1, blocking=true) at cryptopp/zdeflate.cpp:497
#4  0x0833ca37 in CryptoPP::BufferedTransformation::ChannelPut2 (
    this=0x12539250, address@hidden, begin=0xbfe00f00 "",
    length=3219132160, messageEnd=-1075835136, blocking=true)
    at cryptopp/cryptlib.cpp:221
#5  0x0833d3bf in CryptoPP::BufferedTransformation::TransferMessagesTo2 (
    this=0xbfe02f74, address@hidden, address@hidden,
    address@hidden, blocking=true) at cryptopp/cryptlib.h:910
#6  0x0833d515 in CryptoPP::BufferedTransformation::TransferAllTo2 (
    this=0xbfe02f74, address@hidden, address@hidden, blocking=true)
    at cryptopp/cryptlib.cpp:416
#7  0x0814fe3d in
CryptoPP::SourceTemplate<CryptoPP::StringStore>::PumpAll2 (
    this=0xbfe02f60, blocking=104) at cryptopp/filters.h:677
#8  0x0814abfb in xform<CryptoPP::Gzip> (address@hidden) at 
cryptopp/filters.h:648
#9  0x080f57be in encode_gzip<data> (address@hidden, address@hidden)
    at vocab_terms.hh:15
#10 0x080f4fee in pack<data> (address@hidden, address@hidden)
    at transforms.hh:80
#11 0x082ef295 in write_revision_set (address@hidden, address@hidden)
    at revision.cc:1332
#12 0x0818d71e in database::put_revision (this=0xbfffe8b0,
address@hidden,
    address@hidden) at database.cc:1319
#13 0x0818dcdc in database::put_revision (this=0xbfe00f00,
address@hidden,
    address@hidden) at database.cc:1349
#14 0x081e670d in packet_db_writer::consume_revision_data (this=0xbfffdd88,
    address@hidden, address@hidden) at /usr/include/c++/3.3/memory:271
#15 0x081d319c in delayed_revision_data_packet::apply_delayed_packet (
    this=0x12149130, address@hidden) at packet.cc:265
#16 0x081d2f46 in delayed_packet::prerequisite_satisfied
(this=0x12149130, p=
        {px = 0x85c25d0, pn = {pi_ = 0x85c25f0}}, address@hidden)
    at packet.cc:173
#17 0x081d2c85 in prerequisite::satisfy (this=0x85c25d0, self=
        {px = 0x85c25d0, pn = {pi_ = 0x85c25f0}}, address@hidden)
    at /usr/include/boost/shared_ptr.hpp:199
#18 0x081de713 in packet_db_writer::impl::accepted_revision
(this=0x8529e80,
    address@hidden, address@hidden) at
/usr/include/boost/shared_ptr.hpp:253
#19 0x081e6723 in packet_db_writer::consume_revision_data (this=0xbfffdd88,
    address@hidden, address@hidden) at /usr/include/c++/3.3/memory:271
#20 0x081d319c in delayed_revision_data_packet::apply_delayed_packet (
    this=0x121667b8, address@hidden) at packet.cc:265
#21 0x081d2f46 in delayed_packet::prerequisite_satisfied
(this=0x121667b8, p=
        {px = 0x961c738, pn = {pi_ = 0x961c758}}, address@hidden)
    at packet.cc:173
#22 0x081d2c85 in prerequisite::satisfy (this=0x961c738, self=
        {px = 0x961c738, pn = {pi_ = 0x961c758}}, address@hidden)
    at /usr/include/boost/shared_ptr.hpp:199
#23 0x081de713 in packet_db_writer::impl::accepted_revision
(this=0x8529e80,
    address@hidden, address@hidden) at
/usr/include/boost/shared_ptr.hpp:253
#24 0x081e6723 in packet_db_writer::consume_revision_data (this=0xbfffdd88,
    address@hidden, address@hidden) at /usr/include/c++/3.3/memory:271
#25 0x081d319c in delayed_revision_data_packet::apply_delayed_packet (
    this=0x12149200, address@hidden) at packet.cc:265
#26 0x081d2f46 in delayed_packet::prerequisite_satisfied
(this=0x12149200, p=
        {px = 0xa97a718, pn = {pi_ = 0xa79f018}}, address@hidden)
    at packet.cc:173
#27 0x081d2c85 in prerequisite::satisfy (this=0xa97a718, self=
        {px = 0xa97a718, pn = {pi_ = 0xa79f018}}, address@hidden)
    at /usr/include/boost/shared_ptr.hpp:199
#28 0x081de713 in packet_db_writer::impl::accepted_revision
(this=0x8529e80,
    address@hidden, address@hidden) at
/usr/include/boost/shared_ptr.hpp:253
#29 0x081e6723 in packet_db_writer::consume_revision_data (this=0xbfffdd88,
    address@hidden, address@hidden) at /usr/include/c++/3.3/memory:271
#30 0x081d319c in delayed_revision_data_packet::apply_delayed_packet (
    this=0x12149298, address@hidden) at packet.cc:265

....

#5051 0x081d2f46 in delayed_packet::prerequisite_satisfied
(this=0x12274350, p=
        {px = 0x90960d0, pn = {pi_ = 0x90960f0}}, address@hidden)
    at packet.cc:173
#5052 0x081d2c85 in prerequisite::satisfy (this=0x90960d0, self=
        {px = 0x90960d0, pn = {pi_ = 0x90960f0}}, address@hidden)
    at /usr/include/boost/shared_ptr.hpp:199
#5053 0x081de713 in packet_db_writer::impl::accepted_revision
(this=0x8529e80,
    address@hidden, address@hidden) at
/usr/include/boost/shared_ptr.hpp:253
#5054 0x081e6723 in packet_db_writer::consume_revision_data
(this=0xbfffdd88,
    address@hidden, address@hidden) at /usr/include/c++/3.3/memory:271
#5055 0x081d319c in delayed_revision_data_packet::apply_delayed_packet (
    this=0x12274ab0, address@hidden) at packet.cc:265
#5056 0x081d2f46 in delayed_packet::prerequisite_satisfied
(this=0x12274ab0, p=
        {px = 0x860ae98, pn = {pi_ = 0x860aeb8}}, address@hidden)
    at packet.cc:173
#5057 0x081d2c85 in prerequisite::satisfy (this=0x860ae98, self=
        {px = 0x860ae98, pn = {pi_ = 0x860aeb8}}, address@hidden)
    at /usr/include/boost/shared_ptr.hpp:199
#5058 0x081de713 in packet_db_writer::impl::accepted_revision
(this=0x8529e80,
    address@hidden, address@hidden) at
/usr/include/boost/shared_ptr.hpp:253
#5059 0x081e6723 in packet_db_writer::consume_revision_data
(this=0xbfffdd88,
    address@hidden, address@hidden) at /usr/include/c++/3.3/memory:271
#5060 0x081d319c in delayed_revision_data_packet::apply_delayed_packet (
    this=0x12274f38, address@hidden) at packet.cc:265

....

this certainly looks like a far too deep recursion to me, though it
finally ends before consuming all available memory.

#5378 0x081de713 in packet_db_writer::impl::accepted_revision
(this=0x8529e80,
    address@hidden, address@hidden) at
/usr/include/boost/shared_ptr.hpp:253
#5379 0x081e6723 in packet_db_writer::consume_revision_data
(this=0xbfffdd88,
    address@hidden, address@hidden) at /usr/include/c++/3.3/memory:271
#5380 0x081d319c in delayed_revision_data_packet::apply_delayed_packet (
    this=0x122810c0, address@hidden) at packet.cc:265
---Type <return> to continue, or q <return> to quit---
#5381 0x081d2f46 in delayed_packet::prerequisite_satisfied
(this=0x122810c0, p=
        {px = 0xc947808, pn = {pi_ = 0xc947828}}, address@hidden)
    at packet.cc:173
#5382 0x081d2c85 in prerequisite::satisfy (this=0xc947808, self=
        {px = 0xc947808, pn = {pi_ = 0xc947828}}, address@hidden)
    at /usr/include/boost/shared_ptr.hpp:199
#5383 0x081de713 in packet_db_writer::impl::accepted_revision
(this=0x8529e80,
    address@hidden, address@hidden) at
/usr/include/boost/shared_ptr.hpp:253
#5384 0x081e6723 in packet_db_writer::consume_revision_data
(this=0xbfffdd88,
    address@hidden, address@hidden) at /usr/include/c++/3.3/memory:271
#5385 0x081d319c in delayed_revision_data_packet::apply_delayed_packet (
    this=0x122812a8, address@hidden) at packet.cc:265
#5386 0x081d2f46 in delayed_packet::prerequisite_satisfied
(this=0x122812a8, p=
        {px = 0x860bd78, pn = {pi_ = 0x860bd98}}, address@hidden)
    at packet.cc:173
#5387 0x081d2c85 in prerequisite::satisfy (this=0x860bd78, self=
        {px = 0x860bd78, pn = {pi_ = 0x860bd98}}, address@hidden)
    at /usr/include/boost/shared_ptr.hpp:199
#5388 0x081de713 in packet_db_writer::impl::accepted_revision
(this=0x8529e80,
    address@hidden, address@hidden) at
/usr/include/boost/shared_ptr.hpp:253
#5389 0x081e6723 in packet_db_writer::consume_revision_data
(this=0xbfffdd88,
    address@hidden, address@hidden) at /usr/include/c++/3.3/memory:271
#5390 0x081d319c in delayed_revision_data_packet::apply_delayed_packet (
    this=0x122817d8, address@hidden) at packet.cc:265
---Type <return> to continue, or q <return> to quit---
#5391 0x081d2f46 in delayed_packet::prerequisite_satisfied
(this=0x122817d8, p=
        {px = 0xabd2bf8, pn = {pi_ = 0xabd2c18}}, address@hidden)
    at packet.cc:173
#5392 0x081d2c85 in prerequisite::satisfy (this=0xabd2bf8, self=
        {px = 0xabd2bf8, pn = {pi_ = 0xabd2c18}}, address@hidden)
    at /usr/include/boost/shared_ptr.hpp:199
#5393 0x081de713 in packet_db_writer::impl::accepted_revision
(this=0x8529e80,
    address@hidden, address@hidden) at
/usr/include/boost/shared_ptr.hpp:253
#5394 0x081e6723 in packet_db_writer::consume_revision_data
(this=0xbfffdd88,
    address@hidden, address@hidden) at /usr/include/c++/3.3/memory:271
#5395 0x081d319c in delayed_revision_data_packet::apply_delayed_packet (
    this=0x12281950, address@hidden) at packet.cc:265
#5396 0x081d2f46 in delayed_packet::prerequisite_satisfied
(this=0x12281950, p=
        {px = 0xac972d0, pn = {pi_ = 0xac972f0}}, address@hidden)
    at packet.cc:173
#5397 0x081d2c85 in prerequisite::satisfy (this=0xac972d0, self=
        {px = 0xac972d0, pn = {pi_ = 0xac972f0}}, address@hidden)
    at /usr/include/boost/shared_ptr.hpp:199
#5398 0x081de713 in packet_db_writer::impl::accepted_revision
(this=0x8529e80,
    address@hidden, address@hidden) at
/usr/include/boost/shared_ptr.hpp:253
#5399 0x081e6723 in packet_db_writer::consume_revision_data
(this=0xbfffdd88,
    address@hidden, address@hidden) at /usr/include/c++/3.3/memory:271
#5400 0x081d319c in delayed_revision_data_packet::apply_delayed_packet (
    this=0x122819e8, address@hidden) at packet.cc:265
---Type <return> to continue, or q <return> to quit---
#5401 0x081d2f46 in delayed_packet::prerequisite_satisfied
(this=0x122819e8, p=
        {px = 0x9351738, pn = {pi_ = 0x9351758}}, address@hidden)
    at packet.cc:173
#5402 0x081d2c85 in prerequisite::satisfy (this=0x9351738, self=
        {px = 0x9351738, pn = {pi_ = 0x9351758}}, address@hidden)
    at /usr/include/boost/shared_ptr.hpp:199
#5403 0x081de713 in packet_db_writer::impl::accepted_revision
(this=0x8529e80,
    address@hidden, address@hidden) at
/usr/include/boost/shared_ptr.hpp:253
#5404 0x081e6723 in packet_db_writer::consume_revision_data
(this=0xbfffdd88,
    address@hidden, address@hidden) at /usr/include/c++/3.3/memory:271
#5405 0x081d319c in delayed_revision_data_packet::apply_delayed_packet (
    this=0x12281c78, address@hidden) at packet.cc:265
#5406 0x081d2f46 in delayed_packet::prerequisite_satisfied
(this=0x12281c78, p=
        {px = 0x12281e28, pn = {pi_ = 0x12281e48}}, address@hidden)
    at packet.cc:173
#5407 0x081d2c85 in prerequisite::satisfy (this=0x12281e28, self=
        {px = 0x12281e28, pn = {pi_ = 0x12281e48}}, address@hidden)
    at /usr/include/boost/shared_ptr.hpp:199
#5408 0x081df633 in packet_db_writer::impl::accepted_file (this=0x8529e80,
    address@hidden, address@hidden) at
/usr/include/boost/shared_ptr.hpp:253
#5409 0x081dfbea in packet_db_writer::consume_file_data (this=0xbfffdd88,
    address@hidden, address@hidden) at /usr/include/c++/3.3/memory:271
#5410 0x08255443 in session::process_data_cmd (this=0xbfffdcc0,
    type=3221213920, address@hidden, address@hidden) at netsync.cc:2393
#5411 0x0825cc6a in session::dispatch_payload (this=0xbfffdcc0,
    address@hidden) at netsync.cc:2643
#5412 0x0826214c in session::process (this=0xbfffdcc0) at netsync.cc:2722
#5413 0x082636b0 in call_server (role=sink_role, address@hidden,
    address@hidden, address@hidden, address@hidden,
    default_port=5253, timeout_seconds=3219132160) at netsync.cc:2847
#5414 0x0826f9cd in run_netsync_protocol (voice=client_voice,
role=sink_role,
    address@hidden, collections=
          {<std::_Vector_base<utf8, std::allocator<utf8> >> =
{<std::_Vector_alloc_base<utf8, std::allocator<utf8>, true>> = {_M_start
= 0x8502808, _M_finish = 0x8502810, _M_end_of_storage = 0x8502810}, <No
data fields>}, <No data fields>}, address@hidden) at netsync.cc:3273
#5415 0x080a634c in commands::cmd_pull::exec (this=0x84d75a8,
address@hidden,
    address@hidden) at /usr/include/c++/3.3/bits/stl_alloc.h:242
#5416 0x0806468e in commands::process (address@hidden, address@hidden,
    address@hidden) at /usr/include/c++/3.3/bits/stl_map.h:312
#5417 0x0833381c in cpp_main (argc=-1073747664, argv=0xbfffed74)
    at monotone.cc:298
#5418 0x08336042 in main_with_signal_handlers (argc=6, argv=0xbfffed74)
    at main.cc:283
#5419 0x0833609b in main_with_many_flavours_of_exception (argc=-1075835136,
    argv=0xbfe00f00) at main.cc:337
#5420 0x08336421 in main (argc=-1075835136, argv=0xbfe00f00) at main.cc:457


I will try to rerun it with debug output enabled, perhaps this gives a
better clue. Do you, Graydon or Nathaniel, have another command to throw
at gdb?

top gives
VIRT  RES  SHR %MEM   TIME
174m 174m 3612 17.3  81:06
and cat /proc/../maps does not give any hint of memory overflow

monotone --full-revision
monotone 0.16 (base revision: 6fee8fa36ba4cc600ca7b7bf96f8efe192627827)
Running on: Linux 2.4.26 #1 SMP Fre Apr 23 15:55:07 CEST 2004 i686
Changes since base revision:

new_manifest [c2252bcd0f9b70f024bbfd7699384c94ce438546]

old_revision [6fee8fa36ba4cc600ca7b7bf96f8efe192627827]
old_manifest [962286ee8de871962bef39221e26dfe6673bcd73]

patch "INSTALL"
 from [56f96e84e147d2b1691593d0fd24146477dfa2b1]
   to [98ddba3c54f8cd72a6c937b47a7faf1ed62899b8]

patch "mkinstalldirs"
 from [058b9b499887b5df4bc7348e713e7566e4234039]
   to [1fd9153d0537d46ce44fb7cbe105d1538c6d1d45]

(I reran autoreconf -i -f)

   Christof

<weeping> And I just wanted to look whether the 250MB database file
resulting from a cvs_pull would shrink by netsyncing it into a fresh
database </>

signature.asc
Description: OpenPGP digital signature