Re: status of l4-hurd

[Farid Hajji]

> > > Glibc needs also be ported if I'm right.
> > This is a tough one. It depends on the way we're going to follow. If we
> > base everything on top of VK, then we'd need a POSIX emulation library
> > that would probably be outside of glibc (for max. portability).
> 
> I'm not sure I believe in that idea. The POSIX implementation of the
> Hurd depends on *both* the kernel and various Hurd servers. For
> instance, take the implementation of open(). It sends an rpc to the
> port associated with either the cwd or the root (depending on whether
> the filename is relative or absolute). When translators are
> encountered, it will send an rpc representing som subdir. In the end,
> it gets back a port that supports read and write rpc. All the rpc:s
> are Hurd-specific.
You're right and that's why the current implementation puts all this
in <glibc>/hurd/* and <glibc>/sysdeps/mach/hurd (for understandable
reasons).

Ironically, this is exactly what prevents us from porting the Hurd
to non-glibc based systems with ease. I tried to port a small subset
of the Hurd to FreeBSD (background: booting the Hurd in a Lites/RTMach
environment running on a system with FreeBSD binaries, besides Lites
itself). This worked to some (poor) extend, as long as the compiled
glibc was copied in the /lib directory of the FreeBSD system and
hacking around in ld.so and so on. But due to this linkage, all Hurd
binaries needed glibc and it was not possible to either
  * run a FreeBSD/Lites binary from a Hurd shell, nor
  * run a Hurd binary from a FreeBSD/Lites shell.
Both systems were completly isolated from each other.

I tried to relieve this problem by manually extracting <glibc>/hurd/*
and <glibc>/sysdeps/mach/hurd into a hurdposix.so library and then
relink/recompile Hurd programs with FreeBSD's libc+libhurdposix.so.
It even worked (somewhat) and this is what encourages me to keep
advocating a clean separation of glibc and Hurd-specific implementation
of POSIX calls.

Another reason I'd like to see this separation is that I would like to
use OSKit's libc together with L4 to rapidly pull the components together
that would consitute a collection of servers that behaves like the Hurd.
Here too, it would be extremely useful if all of the Hurd's code were
separated from glibc and relinked to OSKit's libc instead.

> So if you implement open() on top of L4 or the VK, independent of the
> Hurd, that implementation won't be terribly useful for the Hurd.
> 
> If there's a "VK", it would make sense to let glibc use the
> message-passing functions/macros offered by the VK though, rather than
> the micro-kernel's functions.
I'm currently investigating both ways and didn't made up my mind yet.
However, I partially agree with you: iff we want to reuse as much code
of the current Hurd implementation, then we should probably stick to
glibc and add yet another adaptation layer to it.

I'm just not sure wether this would be wise. Currently, I'd prefer
that we rewrite key components from scratch, paying close attention to
separating the functionality as much as possible. It's nice to have
POSIX emulation, but it would be even nicer to be able to replace either
the POSIX emulation code with stubs on systems that are already POSIX
compliant (the VK-over-Unix case) or even use something completely
different from POSIX (the VK-over-something-else case), for tasks that
would run independently of current Hurd-emulated POSIX model [this
obviously needs more explanations; more on this later on request]

As I noted earlier, there are hardware architectures out there (massively
parallel machines, NUMA clusters, ...) that could perfectly well run
the Hurd servers (even distributed). But to be able to do so, we must
use their existing extended POSIX libc's (not glibc) for the following
reasons:

  * The vendor's libc generally provide POSIX compatibility, including
    pthreads and everything else as usual. The added benefit of those
    vendor libc's is that they fully take advantage of the native OS
    that runs on those beasts. Its somewhat comparable to glibc's
    Hurd sysdeps ;-). Put in other words: Hardware parallelism and
    even virtualization may only be accessible though this vendors libc.

  * Owners of those big clusters/machines are bound by legal and
    economic factors that prohibits them from taking the system down
    just to try to boot an experimental Hurd version. They bought
    or leased those systems to run real-world number crunching software
    that would itself only run [optimally] with the native-OS.

  * Some, if not most, exotic machines are so poorly documented, that
    porting a kernel to them seems nearly impossible, so we may want
    to run the Hurd as guest OS (say guest task(s)) in the first stage,
    before some genius finds a way to port VK itself on the bare hardware.

Would _you_ be willing to first write a microkernel (or port Mach, "VK",
L4{or L5?} to such an architecture first, then port glibc to this new
context? I have no idea how much difficulties L4 people are currently
experiencing with SMP-L4, let alone L4 for exotic processors!

BTW, I doubt that glibc is really that OS-portable: <glibc>/README lists:

                *-*-gnu                 GNU Hurd
                i[3456]86-*-linux-gnu   Linux-2.x on Intel
                m68k-*-linux-gnu        Linux-2.x on Motorola 680x0
                alpha*-*-linux-gnu      Linux-2.x on DEC Alpha
                powerpc-*-linux-gnu     Linux and MkLinux on PowerPC systems
                sparc-*-linux-gnu       Linux-2.x on SPARC
                sparc64-*-linux-gnu     Linux-2.x on UltraSPARC 64-bit
                arm-*-none              ARM standalone systems
                arm-*-linux             Linux-2.x on ARM
                arm-*-linuxaout         Linux-2.x on ARM using a.out binaries
                mips*-*-linux-gnu       Linux-2.x on MIPS
                ia64-*-linux-gnu        Linux-2.x on ia64
                s390-*-linux-gnu        Linux-2.x on IBM S/390
                s390x-*-linux-gnu       Linux-2.x on IBM S/390 64-bit
                sh-*-linux-gnu          Linux-2.x on Super Hitachi
                cris-*-linux-gnu        Linux-2.4+ on CRIS

Besides the Hurd and ARM standalone systems, I see only Linux here.
The following OSes seem all to be dead (?) by now:

                alpha-dec-osf1
                i[3456]86-*-bsd4.3
                i[3456]86-*-isc2.2
                i[3456]86-*-isc3
                i[3456]86-*-sco3.2
                i[3456]86-*-sco3.2v4
                i[3456]86-*-sysv
                i[3456]86-*-sysv4
                i[3456]86-force_cpu386-none
                i[3456]86-sequent-bsd
                i960-nindy960-none
                m68k-hp-bsd4.3
                m68k-mvme135-none
                m68k-mvme136-none
                m68k-sony-newsos3
                m68k-sony-newsos4
                m68k-sun-sunos4
                mips-dec-ultrix4
                mips-sgi-irix4
                sparc-sun-solaris2
                sparc-sun-sunos4

Sure, most of those OSes are defunct by now, some are missing altogether
like say HP-UX and I'm wondering about the status of sparc-sun-solaris2!

More importantly, we don't even have a glibc port to any of the free
*BSD kernels (Free-, Open- and NetBSD). The docs say it's easy do port
glibc, but nobody seems to be able or willing to do it at all. This is
even more puzzling, since all *BSDs support Linux binaries linked
against glibc through syscall redirection (BTW, it's an incredible
easy trick, see NetBSD sources for binary compat to other OSes even on
non x86-platforms as well).  I didn't hear of a Linux kernel module
(or whatever) that would relink *BSD binaries into BSD-based
libc. Why?

You may argue that due to *BSD's Linux binary compatibility feature,
a port of glibc to *BSD is not necessary. This is true, but I'm still
missing a proof of concept, that glibc-2.x is really so portable than
what they claim.

Please not that even if the above sounds like a harsh critique against
glibc, this is not my intent. Glibc is as good as it can get and you
may use it if you want. I'm just questioning the necessity to tie the
Hurd to a C library in such an intertwined way. That's all I'm wanting
to say here.

> > This is just an idea right now. In the meantime, I'd suggest that we
> > use OSKit's libc (and other components) while running under L4.
> 
> Can you explain what this libc is? My guess is that it's library
> support for a subset ANSI-C, but no POSIX functions, no FILE *, etc?
OSKit comes with two libc's: A smallish mini-C library that contains
only minimal functionality like str*(), etc... and a full fledged libc
borrowed from FreeBSD (3.x) with support for pthreads and all. Of course
I mean the full-fledged libc here ;-). Please have a look at OSKit
sources and docs regarding libc.

> Regards,
> /Niels

Thanks,

-Farid.

-- 
Farid Hajji -- Unix Systems and Network Admin | Phone: +49-2131-67-555
Broicherdorfstr. 83, D-41564 Kaarst, Germany  | address@hidden
- - - - - - - - - - - - - - - - - - - - - - - + - - - - - - - - - - - -
One OS To Rule Them All And In The Darkness Bind Them... --Bill Gates.