lmi
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[lmi] Limits on pastable census size


From: Greg Chicares
Subject: [lmi] Limits on pastable census size
Date: Fri, 9 Feb 2018 14:06:14 +0000
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.5.2

Yesterday we had to work with a census with over ten thousand cells.
(We've seen one that large only once before.) It seems to work now,
but it was difficult to create with "Census | Paste census", which
ran out of memory (it threw bad_alloc). We used a brute-force
workaround: paste it in two five-thousand-cell chunks into separate
'.cns' files, then combine them with a text editor, carefully (it's
not just a matter of concatenation).

I imagine this problem wouldn't arise with a 64-bit build (though I
haven't tested that). Maybe this long-deferred PR:
  https://github.com/vadz/lmi/pull/18
would help marginally (we should apply it, regardless, when we have
the time), but I think this is really a 32-bit limitation.

Using the patch below, I tracked the failure down to this line:
    std::copy(cells.begin(), cells.end(), iip);
Maybe we could do something cleverer there--in particular, in the
most common case, where this unmodified line in the patch:
    cell_parms ().clear();
has been executed, it seems that we could "move" cell_parms() to
'cells'. But, given that the std::copy fails for size N=10000 but
succeeds for N=5000, I'd guess that a "move" would fail for
N=20000, so that's not a general solution.

BTW, calling DoSaveDocument() was an idea that seemed worth trying,
but it doesn't DTRT: pasting 5000 cells and saving produced a real
'.cns' file that was about ten percent larger than file "enormous".
That doesn't seem to be worth investigating further now that we
know that's not the point of failure.

---------8<--------8<--------8<--------8<--------8<--------8<--------8<-------
diff --git a/census_view.cpp b/census_view.cpp
index ea21aa5f6..e683202b4 100644
--- a/census_view.cpp
+++ b/census_view.cpp
@@ -1763,6 +1763,7 @@ void CensusView::UponPasteCensus(wxCommandEvent&)
         return;
         }
 
+warning() << "okay" << LMI_FLUSH;
     if(!document().IsModified() && !document().GetDocumentSaved())
         {
         case_parms ().clear();
@@ -1772,12 +1773,20 @@ void CensusView::UponPasteCensus(wxCommandEvent&)
         cell_parms ().clear();
         }
 
+warning() << "okay" << LMI_FLUSH;
     auto selection = cell_parms().size();
     std::back_insert_iterator<std::vector<Input>> iip(cell_parms());
+warning() << "okay" << LMI_FLUSH;
     std::copy(cells.begin(), cells.end(), iip);
+warning() << "okay" << LMI_FLUSH;
     document().Modify(true);
+warning() << "okay" << LMI_FLUSH;
+document().DoSaveDocument("enormous");
+warning() << "okay" << LMI_FLUSH;
     list_model_->Reset(cell_parms().size());
+warning() << "okay" << LMI_FLUSH;
     Update();
+warning() << "okay" << LMI_FLUSH;
     // Reset() leaves the listview unreachable from the keyboard
     // because no row is selected--so select the first added row
     // if possible, else the row after which no row was inserted.
--------->8-------->8-------->8-------->8-------->8-------->8-------->8-------



reply via email to

[Prev in Thread] Current Thread [Next in Thread]