Re: Review of Thomas's >2GB ext2fs proposal

[Neal H. Walfield]

At 17 Aug 2004 02:38:15 -0700,
Thomas Bushnell BSG wrote:
> 
> "Neal H. Walfield" <neal@cs.uml.edu> writes:
> 
> > You need to drain the cache once in a while.  You need some kind of
> > strategy.
> 
> As I said, LRU is easy.
> 
> > Fine, but this is extra memory per region and it needs to be updated
> > fairly often.  This is all gotten rid of when we just clear a mapping
> > when the kernel does.
> 
> You still need the ref count!  Where are you going to put it?

In memory.  But I don't need a linked list.

> > Of course you are right and I have taken this into account.  If a
> > mapping has user references, the mapping is not removed.  It is likely
> > that the page will be pulled back into memory in the near future
> > anyways because of the user mapping.  But likely is certainly not
> > enough of a reassurance.  So, in the case where the mapping cache is
> > completely full and we need to force some pages out, we search for
> > pages that have no user references.  Clearly this is an unoptimal
> > eviction strategy, however, it should be rare enough satisfies the
> > correctness requirement.
> 
> I thought you always allowed the kernel to do the pageouts....this is
> additionaly complexity if you are sometimes forcing pages out.

This only happens if the mapping cache (which will be about 2GB large)
is full.  And it will only be full when the kernel cache contains that
much data (minus any of these pages which should be small in number).

> > > > The interesting corner case is what happens if there are user
> > > > references and the kernel drops the page.  Well, in this case, we
> > > > just leave the mapping in place based on the assumption that the
> > > > page will be faulted back in momentarily (otherwise, why would there
> > > > by a reference?).
> > > 
> > > This is incorrect.  You might be just about to drop the reference,
> > > having already completed all the uses of the page, when the page gets
> > > paged out.
> > 
> > Fine, it is not rigorously correct, however, leaving it in the cache,
> > as I describe above, does not hurt the system.
> 
> Assuming that it does eventually get dropped, otherwise, you have a
> memory leak.  The kernel isn't going to be handing it back, so how
> will you check?

It is not a memory leak as it can still be reclaim.  It will be
reclaimed (because it has no references) when the mapping cache is
full.  Or, and perhaps this is a better approach, we add a check in
the release function to see if the kernel has a copy and if not, drop
it at that point.

> (I think the solution is to have a proper interlock when you drop the
> last reference, but this is very tricky to get right.)

I am not sure what you envision here.

Neal