Re: Documentation and ideas for related work

[Paul Boddie]

On Tuesday 16. April 2019 15.44.10 Richard Braun wrote:
> On Tue, Apr 16, 2019 at 03:12:21PM +0200, Paul Boddie wrote:
> > What interests me most at the moment is how filesystem content is paged
> > into processes, whether there is some kind of sharing of accessed pages
> > between processes (some kind of page cache, maybe), and how such things
> > are organised.

> That question relates much more to the Mach kernel than to the Hurd.
> And since the Mach VM is the ancestor of most "modern" operating systems
> (the BSDs in particular, but the Linux memory management code uses a
> mostly similar design), documentation shouldn't lack.

I will have to dig around for informative references.

> I recommend the NetBSD UVM paper, which extensively describes the BSD
> VM, and in turn much of the Mach VM.

This did appear to be rather interesting, thanks! Here is the paper, as far as 
I can tell:

"The UVM Virtual Memory System"
https://www.usenix.org/legacy/event/usenix99/full_papers/cranor/cranor.pdf

Much of it appears to discuss mechanisms that, for my experiments with L4Re at 
least, are already designed by others. It seems quite likely that these may 
also be informed by this and other papers.

(For instance: memory maps probably correspond to the role of the region 
mapper/manager in L4Re; memory objects resemble dataspaces in L4Re. The 
mechanisms involved may differ, obviously.)

> What's not described, obviously, is external memory managers, but it's
> similar to how kernel memory managers (file systems) interface with
> the VM.

I don't really follow the distinction between external and kernel memory 
managers, but perhaps this is pertinent to NetBSD but not to a microkernel-
based system. Please let me know if I missed something.

In any case, I can envisage the layering being broadly similar:

* A collection of pages to be handed out to things wanting to populate them
  (this normally being kernel or VM policy)
* Filesystem accessors and other things that populate and store pages
* Filesystem object abstractions that provide views using populated pages

Indeed, the messages previously mentioned seem to echo this general 
arrangement:

http://lists.gnu.org/archive/html/l4-hurd/2002-06/msg00001.html

Currently, my experiments are quite simple in that they take pages from the 
collection, populate them, export them, and when they are either discarded or 
needed elsewhere, they are flushed and recycled.

It seems to me that the page collections can remain oblivious to what the 
pages are used for, the accessors need only know how to load and store data, 
and a general mechanism can be used to orchestrate the supply of pages.

But really, I am still in the earliest stages of prototyping, hence my 
interest in what other systems do and what people may have intended to do in 
the past.

Thanks once again for the reference!

Paul