Re: [Qemu-devel] get_func() hangs with gcc 3.4.2 on MinGW and WinXP host

[John R. Hogerhuis]

On Sun, 2004-12-19 at 06:45, Johannes Schindelin wrote:
> > The op.c primitives are, well, primitive... at some point consideration
> > should be given to simply giving up.
> >
> > That is, code the primitives in native assembler for each target CPU.
> 
> NOOOOO! Did you -- even once -- look at op.s? It is a *big* file. A
> quick glance at my reasons for the "NOOOOO!":
> 

Yes, I've looked at op.s when I was working on a patch some time back.
You're right, it's really big. Hand written though, with macros and
such, I think it would be much much smaller.

> - it is a *lot* of work.

> - you loose *all* of C's type safety.

You have to admit GCC is already being a little abused here. I don't
think any writer of the C compiler would have anticipated the way QEMU
makes use of it. It is designed to generate object modules for a linker,
that's it.

It's a clever hack, but if it didn't work, and no one had done it, I
would have said it likely won't work, you'll end up needing to hack up a
custom compiler just for this purpose.

Turned out I would have been wrong for old versions of GCC. That's
changed with newer ones. I think it will only get worse over time.

> - it is much too easy to miss (even unintentional) bugs.

It is too easy to miss bugs in any language. On the other side of it,
assembly language programmers know they are working without a net and
tend to be much more careful than C programmers.

> - it is not at all portable.

It's just a different level of portablility. Sort of like the
portability of Forth from one architecture to another. You just need to
port the core wordset and you get the high level words for free.

> - it is a maintenance nightmare.

Any sizable piece of software is a maintenance nightmare. I think you
are overstating the negative case (there certainly is a negative
case...) I think the complexity of a core set of primitives in assembler
is manageable, especially since for most CPUs we can start out with
autogenerated versions.

To my mind the status quo is the bigger nightmare. In the current
situation, QEMU breaks through a layer of abstraction/encapsulation that
writers of GCC ought to be able to expect.

I doubt casual programmers would have any necessity to touch op_xxx.s. 
The point is to solve the current maintenance issue (I won't call it a
nightmare yet). QEMU loses its implementation depedencies on GCC in
order that it becomes more easily maintainable given that GCC changes
will no longer break the build.

> 
> > Certainly the C compiler is being well leveraged at present to do smart
> > things like allocating registers, but a decent assembly programmer will
> > make just as good or better decisions than gcc.
> 
> No. That's just not true. It might have been true with 80x86 up to 80286,
> but even the earliest Pentia had several pipeline and caching quirks which
> made it virtually impossible to beat very good compilers.
> 

That's pretty much true. But with primitives we are rearranging in
arbitrary order at runtime, I would *guess* the compiler can't really
help us much.


> And this is only for Intel platforms. Ask *anybody* with MIPS experience.
> There are probably just under 20 people (some claim it's more like 5) on
> this planet who could beat the compiler -- and this only in case of short
> functions.
> 

Well in the case of MIPS, we start out with an autogenerated op.s, the
best that GCC can do. It can only get better with hand tuning and
benchmark testing.

And I would submit that the assembly language programmer will *always*
do the same or better than the compiler. Why? Simply because the
assembly language programmer always has the option of starting with
generated code, hand optimize it, and test the performance to see
whether it got any better.


> One very strong point about QEmu is that it's very portable. Not only is
> it easier to find strange bugs using different platforms (if you don't
> believe me, I can give you very good examples), but it is important for
> the main purpose of an emulator: to run on newer machines.
> 
> I agree with you on your point about least resistance. But as a Bazaar
> project we have the big advantage that our quality doesn't suffer from
> silly deadlines or plain dumb suits (yes, I mean all those business types
> who don't have a clue about the subject they're deciding). We should make
> it clear to the gcc people that there is a demand for the feature
> we're seeking, namely to force the use of exactly one ret.
> 

Yes, but deadlines and suits aren't the issue. The issue is dependency
on another project, gcc. GCC changes are going to continue to break the
build, unless they choose to support the kind of controls on code
generation QEMU and perhaps other projects (interpreters?) needs. QEMU
can't just stay on an old verson of GCC, since new CPUs can and will be
released. Should we fork GCC? Port the existing version ourselves to
each new architecture? I think porting op.s to each new arch will be
much easier. I submit that OSS projects normally take the path of least
resistance, get something working and make improvements/optimizations
later. That "it works" attracts more people to the project, and thus
more bugfixes improvements.

That's philosophy though. The reality is that qemu build doesn't work
with new versions of gcc. You have to remove the direct dependency
altogether: hand code, as I suggest, or use another code generator, c--,
or a generic assembly language as discussed in other threads, that is
actually designed for this purpose.

If we have reason to believe that the GCC folks are genuinely interested
in providing documented code generation facilities of the kind that QEMU
and perhaps other projects really need, that's great.

> BTW is there a way to declare an external label? We could replace all
> returns in op.c by "jmp exit_label", and parse for this instead of "ret",
> if that works.

Perhaps it would make sense to split QEMU into a QEMU Porting Kit and
QEMU. QEMU porting kit could be used to generate op.s files for QEMU.
Then QEMU is no longer dependent on gcc implementation. If gcc breaks
the porting kit it might still generate some .s that could be adjusted
with a text editor or arbitrarily complex evil scripts to make something
useful. But at least that mess would be kept separate from the main
build and no one would feel that pain until a new CPU architecture comes
out to port to.

-- John.