gnumach kernel memory map (was: Re: 64bit startup)

[Luca]

Il 09/08/23 11:20, Samuel Thibault ha scritto:
> Sergey Bugaev, le mer. 09 août 2023 12:12:29 +0300, a ecrit:
> > It should just reuse whatever memory the bootloader has already loaded
> > the module into, no need to copy it out anywhere.
>
> (my guess is that it's not so simple because possibly we somehow get rid
> of the memory allocated for modules, otherwise we'd get stuck with that
> piece of memory area that we cannot reuse for other system memory uses)

From what I understand so far it's a bit tricky, especially in the 
early boot phase. The current kernel map on x86_64 is very very simple, 
and inherits some constraints from the 32-bit map for simplicity. It's 
also partly limited by the simple bootstrapping procedure in boothdr.S. 
Some issues that I found so far are e.g.:
* decouple the bootstrap map created in boothdr.S from the runtime map 
and initalize it properly in pmap_bootstarap() (see also _phystokv vs 
phystokv). There is some memory allocated in the early C code that might 
need to be remapped (e.g the kernel page table map and the biosmem heap)
* the kernel .text/.bss/.data etc seem to be the only parts required to 
be in the upper 2G, due to mcmodel=kernel. Everything else could be 
anywhere.
* highmem is somehow limited by VM_KERNEL_MAP_SIZE, which itself is 
currently limited by the 2G range
* in some places it seems there is the assumption that memory is 
directly mapped (i.e. with a simple offset between virtual/physical 
address), I'm not sure how pervasive this assumption is and how 
difficult it would be to remove it where it's not needed.

Some time ago I started a first step in this direction, attempting to 
move down the kernel map so the highmem/directmap regions could be made 
much bigger, and still keeping the static data in the upper 2G. I 
uploaded it here [0] but it's really just a first step.

My idea here was to have as a first step the various 
dma/directnap/highmem regions between -4G and -2G and still contiguous, 
and then try to leave some red zone between them and make the regions 
bigger, once the limits could be freely moved.

For the bootstrap modules, it might be enough to explicitly reserve the 
memory in pmap_bootstrap(), although that would consume "low" memory. 
Once we have bigger memory regions, it shouldn't be too hard to move the 
bootstrap module somewhere else.

Any thoughts? I have probably overlooked some other issues, so please 
correct me if I'm wrong.

Luca


[0] 
https://gitlab.com/luckyd/gnumach/-/commit/f0564e8ed63b82b4e8b0342821c37f2461625438