Re: [lmi] change file formats to XML

[Greg Chicares]

[revisiting benchmarks...]

On 2010-03-06 00:16Z, Greg Chicares wrote:
> 
> Here's a (non-GUI) timing benchmark:
> 
> /lmi/src/lmi[0]$time make system_test >../log 2>&1
> make system_test > ../log 2>&1  84.73s user 67.96s system 45% cpu 5:21.33 
> total
> 
> I'm using msw, so 'total' time is the most relevant measure, and it's
> not materially different from HEAD:
>   5:19.62 http://lists.nongnu.org/archive/html/lmi/2010-02/msg00001.html
>   5:21.33 today with patch above
> That's misleading, though, because the 1400 or so cells in the system
> test are largely sorted by "Product", and IIRC the most-recently-read
> '.*db4' file read is cached. Furthermore, in an automated test, the time
> it takes to load product files is only a small proportion of the total;
> but in the GUI, the delay in switching products is perceptible. To
> scroll through seventy-six products (start at the top, and hold down the
> down-arrow key) takes four seconds in HEAD, but thirty-five seconds
> with the above patch. However, it takes only six seconds with my local
> tree [...]

Now it takes seven seconds. Much has changed, of course; for example,
xml product files now include annotations. Anyway, that's a secondary
measure--with somewhere between half a gross and fourscore products,
one would seldom switch sequentially through each of them.

With HEAD as of right now, I observe:

/lmi/src/lmi[0]$time make system_test >../log 2>&1
make system_test > ../log 2>&1  83.07s user 66.48s system 44% cpu 5:38.96 total
  [of which 231346 milliseconds is spent running lmi itself]

which is about eighteen seconds or six percent slower: not bad,
considering the inefficiency of xml.

Here are portions of the database benchmarks in 'input_test.cpp',
before recent changes:
  Query(scalar)     : 1.781e-06 s =       1781 ns, mean of 5616 iterations
  Query(vector)     : 6.061e-07 s =        606 ns, mean of 16501 iterations
and after:
  Query(vector)     : 5.464e-07 s =        546 ns, mean of 18304 iterations
  Query(scalar)     : 4.155e-07 s =        415 ns, mean of 24071 iterations
Those measurements have nothing to do with xml; they show the effect of
removing inefficiency that had been there forever. Thus, loading a database
has become much slower, but using it has become faster--these measurements:
  system_test(): 253345 milliseconds [original, inefficient]
  system_test(): 231346 milliseconds [current, more efficient]
show a nine-percent reduction in the time it takes to run fourteen hundred
illustrations.

Along the way, about seven percent of marked defects have been resolved:
  20100224T1904Z: 762 marked defects
  20100508T1927Z: 710 marked defects
and of course we've gained the great advantages that xml offers in return
for its relative slowness.