Re: large arrays - how to store?

[Paul Schneider]

Guy Harrison wrote:
> Paul Schneider wrote:
>
>
> > Ulrich Lauther wrote:
> >
> > > Guy Harrison <swamp-DEL-dog@ntlworld.com> wrote:
> > > : Paul Schneider wrote:
> > >
> > > : > Hello,
> > > : > 
> > > : > for my program I need a large multidimensional array the dimensions
> > > : > of which I know at compile time. Specifically it is of size 500 x
> > > : > 5000 x 10 x 6. I am looking for guidance of how to treat this array
> > > : > on a linux system with 2 Gigs of memory.
> > > : > 
> > > are all the 500x5000x10x6 memory lactions actually used? Or is the data
> > > structure sparse? In the latter case, you should consider storing just
> > > the used elements.
> >
> > It#s a dense array. All f the elements are used extensively. Suggestions?
>
>
> Important info but still not enough. Used extensively? In what fashion?
>
> "Dynamic allocation" is still the stock answer. Consider also grabbing some
> older books which cover "batch processing". Most processing is boring
> repetitive stuff. Half the battle with programming is losing conventional
> thought. Sparse arrays are a bonus. Don't program for them unless you have
> reason to.
Thanks for your answer. My problem is definitely not even remotely 
sparse. It's all dense vector/matrix matrix/matrix stuff.


I don't know much about compilers and operating systems. Maybe this is 
why it seems so unintuitive to me to throw away the advantage? to know 
everything at compile time. In all the experiments I performed, static 
allocation itself and working with statically allocated structures was 
much faster than the dynamic counterpart.