[Octave-bug-tracker] [bug #55909] Performance of hist

[Michael Leitner]

Follow-up Comment #3, bug #55909 (project octave):

The test passes now. The solution was trivial, just converting to double where
it is necessary. Of course this is undesirable, because it increases memory
requirements, but the most efficient algorithm of scaling, shifting and
rounding y for direct use as indices just does not work in integer
arithmetics. 

It would be possible to use the old code path for integer input, but I think
this would be worse: after all, there is an obvious work-around for keeping
the memory down with the present proposal (just computing histograms of parts
of the data and adding up), while there would be no possibility to choose the
efficient algorithm in the converse case. I admit that one could have it both
by doing the piece-meal histogramming in an inner loop inside hist.m itself in
the case of very large inputs, but I do not feel this to be warranted (yet).

(file #46532)
    _______________________________________________________

Additional Item Attachment:

File name: hist_diff2                     Size:2 KB
    <https://savannah.gnu.org/file/hist_diff2?file_id=46532>



    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?55909>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/