Re: [Lightning] JIT binary translation using lightning possible?

[Paulo César Pereira de Andrade]

2015-10-20 4:01 GMT-02:00 Kaz Nishimura <address@hidden>:
> Thank you for the quick reply.  I would like to implement a JIT binary code
> translation in an emulator of an existing CPU.  Thanks to compiler
> optimizations, I cannot rely on the program structure on the "guest" system,
> and I would have to synthesize a "function" from an unstructured code
> sequence.
>
> My current plan is to keep "guest" hardware registers in a context structure
> and to generate "host" code to manipulate it in a synthesized "function".
> Redundant code elimination is a key for better performance in the "host"
> code since the condition code can be overridden by each instruction.  If
> such condition code computation could be automatically eliminated, I expect
> code translation would become much easier.
>
> Each "guest" instruction will be translated to a sequence like the
> followings.
> 1. Load operands from the "guest" context to JIT registers.
> 2. Execute an operation or two on the JIT registers.
> 3. Store the result to the "guest" context.
> 4. Compute the condition code for the result.
>
> Where step 4 could be redundant as it might be overridden by the next
> instruction so I am looking for a better JIT code generation infrastructure.

  You should be able to convert to jit sequences of code, at least until
a branch is reached, and while doing it, simply do not generate step 4;
read ahead at least one instruction.
  The larger the code chunk you can generate the better, otherwise
cache sync, and jit generation time may cost too much.
  The big advantage of jit is loops, so, you should have a translator
capable of underestanding simple loops in the "guest" code, so that
they are converted to loops in the "host" code.
  A few condition codes can be generated somewhat cheaply in lightning.
  You should also try to map as much "guest" registers to "host" registers
as possible, and only update the context, that I suppose is a structure
in memory, when there is no other option.

> On Tue, Oct 20, 2015 at 11:41 AM Paulo César Pereira de Andrade
> <address@hidden> wrote:
>>
>> 2015-10-19 21:48 GMT-02:00 Kaz Nishimura <address@hidden>:
>>
>>   Hi,
>>
>> > I am new to GNU lightning and am now looking for a "light-weight" (that
>> > is
>> > not so much slower than direct instruction emulation) JIT code generator
>> > for
>> > a runtime binary translator.  Though I could not find detailed
>> > documentation, is it easy to use lightning for such a use case?  I feel
>> > I
>> > must handle register allocation myself, and that lightning does not have
>> > redundant code elimination (or optimization).  They are my concerns.
>>
>>   There is a project using lightning for binary translation at
>> https://github.com/pcercuei/lightrec/ that you may find interesting.
>> Paul may want to comment about it :)
>>
>>   GNU lightning uses a concept of contexts, where one first creates
>> a stream of instructions, and when done, it is converted to native
>> code. It is possible to generate code to jump from one context to
>> another, and there are some abstractions to help in such cases; what
>> is helpful when translating byte code to native code, or a plain parser
>> directly to native code.
>>
>>   It currently only generates code for the same architecture lightning
>> was compiled to.
>>
>>   You must see registers as "descriptors". They are not opaque, and
>> there are some code time generation predicates, like to know if it is
>> a callee save register, or for function arguments, if it is an argument
>> in a register.
>>
>>   There is basic optimization done at code generation time, doing
>> jump threading, simple redundancy removal, and on a few cases,
>> simple strength reduction, not on all backends, doing things like
>> converting multiplications by shifts, etc, when applicable.
>>
>>   Lightning does not overwrite registers, but non callee save registers
>> are not saved across function calls. When generating jit it creates
>> bitmaps of live registers, following jumps, so, it knows all the time
>> what registers are live. The exception is a non local jump (jump to
>> fixed address or register contents), where it will treat non callee
>> save registers as dead, like a function call.
>>
>>   If you can describe a bit better what your needs are, I should be
>> able to better tell what problems may arise, or, when simple needs,
>> I may implement any missing features :) Lightning does not have
>> any dependencies; but binutils is desirable, at least on an
>> intermediate build, for easier debugging, as it uses binutils for
>> an optional disassembler interface.


Thanks,
Paulo