I also tried the complex test:
a = b = 1+1i;
for ii=1:5
a = a + b;
endfor
unfortunately it generates a segmentation fault (works fine without JIT). If the goal is to enforce calling convention, then I suggest you use the appropriate modifiers instead of extern "C". If you want I can give it a quick try here.
No luck, still crashing (I replaced all 'extern "C"' in pt-jit.cc with __cdecl equivalent). Debuggin shows it's crashing in octave_jit_cast_complex_any, when trying to access obv. It looks like obv is not a valid pointer, indicating a potential stack corruption.
I'm not sure this is related, but did you know that even with __cdecl, there's an ABI difference between MSVC and GCC when returning aggregate objects larger than 8 bytes (typically returning a double complex object). The object to return is provided through a hidden pointer pushed on the stack, but MSVC assumes the caller pops the hidden argument, while GCC assumes the callee pops it (note: this has been fixed in gcc-4.7.0, which is now compatible with MSVC). Another source of problems is stack-alignment, although I don't whether it's relevant in this case (MSVC aligns the stack on 4 bytes, GCC aligns it on 16 bytes).
I have no idea how the JIT is working, but if binary code is compiled on the fly for functions returning a large aggregate object, maybe the wrong ABI is being used, leading to stack corruption.
Michael.