Re: %gs:0 thread pseudoregister in oskit-mach

[Espen Skoglund]

[Farid Hajji]
>>> The UTCB is a central information page for L4 userspace threads
>>> and it's pretty difficult to find another x86 register that can be
>>> used to point to it.
>> 
>> What's wrong with %fs?  But anyway, that is fine if the one word at
>> %gs:0 can be made available for user use.  Ideally that word would
>> be writable (which you could manage along with read-only use by
>> situating the segment base on a page boundary, one page writable
>> and one not).  But the only thing that would write it is pthreads
>> innards, so it would be workable to have a system call to set the
>> %gs:0 word for a thread.

%fs in the Win32 ABI points to a per-thread Thread Information
Block (TIB).  Don't ask me details about exactly what this segment
contains or how it is used.  I have no idea.  Point is that if we want
to be compatible with the Win32 ABI we should probably leave %fs
alone.
 
> Hmmm, the L4 X.2 spec only says that %gs:0 points to the UTCB area
> and that it should be read out.

What the spec does not directly say is that it also needs to be able
to modify %gs:0.  This is needed for doing LIPC (i.e., IPC without
entering kernel mode).  See the HotOS 2001 "Lazy Process Switching"
publication for more details.

> If userspace threads can change %gs:0 without interfering with L4,
> fine.  We could just squirrel %gs:0 away in crt0.c before glibc
> takes over and changes %gs.

> BUT, I'm not sure if
>   * %gs:0 can be written to by userspace threads
>   * the L4 API _implementation_ reads %gs:0 afterwards.

Yes.  As I said, %gs:0 can indeed be changed by user-level code.
However, changing it to arbitrary values can have strange side
effects.  Basically, if the kernel detects that the value has been
changed it will believe that a LIPC has been performed and will
complete the lazy context switch performed by the LIPC operation.  If
the value in %gs:0 happens to point to another valid L4 UTCB in the
current address space, a context switch to this thread will be
performed (with side effects I do not yet fully comprehend).  If the
value in %gs:0 does not point to a valid L4 UTCB, the kernel will
assume that the user messed up something and behaviour is then pretty
much undefined (i.e., same effect as if the user messed up his own
stack or code pages).

>> I don't know the L4 interfaces, but why not just make this page
>> accessible at a canonical address accessible from the standard user
>> %ds segment?

%gs:0 in L4 is a pointer to the currently running thread's UTCB.
Using a canonical address is not really feasable since it would have
to be defined by the L4 ia32 ABI (at address zero or some other magic
location).  This is not a very general solution (i.e., it will most
certainly cause trouble for somebody), and defining a canonical
address above 4MB also rules out the possibility of using small
spaces.

Furthermode, the page frame containing the UTCB pointer should be
mapped as a kernel object (i.e., mapping can not be flushed by
user-level code).  If the user is responsible for the mapping, the
kernel can not make the assumption that it can always read the UTCB
pointer without generating pagefaults.  This wastly complicates the
kernel design and forces us to implement loads of special cases in
time critical code (e.g., IPC).

Now, realising that the page must be mapped by the kernel, we
basically need to eat up 4KB of the user virtual-address space.  This
might be fine, but it also prohibits the user to have a 4MB mapping to
the same area (1023 4KB mappings are needed instead).  Once again, we
put restrictions on what the user is allowed to do.

Those who have read the X.2 spec will probably now counterargue
against my last argument by saying that: "hey, this is also true for
the Kernel Interface Page and UTCB areas!"  The answer to this is
that, yes, this is indeed a restriction, but it is a restriction that
is controlled at user-level.  That is, the application creating a new
address space specifies exactly which virtual areas the KIP and UTCBs
should go into.

> %ds and other registers are needed for the L4 syscalls.
> The problem is that especially x86 has so few registers :-(

%ds, %ss, and %cs are not really needed for L4 syscalls.  They are,
however, needed implicitly by most instructions in the ia32 ISA.  %es
can not be used either as it is used in string operations, and using
%fs is not desireable because of Win32 incompatibilities.

First solution that comes to mind is to use %gs:4 (or something) for
addressing the UTCB.  I'm not too happy with this solution, though
(what stops the next guy to come and claim %gs:4 for some other use).
Another problem with this solution is that %gs:0 must be treated as a
regular register and saved/restored on context switches.

The optimal solution from L4's point of view would be to use the
UserDefinedHandle virtual register.  I do realize, however, that this
is not possibe if the compiler/linker can create code like "movl
%gs:0, %eax".  It also makes TLS lookup more expensive as it needs
another indirection to get to the wanted memory contents.


Reading the "ELF Handling For Thread-Local Storage" document, I'm not
sure what the following paragraph on page 24 is supposed to mean:

     "The movl instruction at address 0x06 loads the thread pointer
  for the current thread in the %eax register.  This step eventually
  has to be adjusted to the method the platform is using to access the
  thread pointer."

Does this imply that using %gs:0 is not really pinned down in the
specification yet?  And why isn't the ELF ABI using %fs like Win32
does?

        eSk