Re: [Tinycc-devel] Porting TCC to RTLinux

[Calin A. Culianu]

On Wed, 22 Nov 2006, Rob Landley wrote:

> The ability to build tcc as a kernel module makes a certain amount of sense.

Cool, I am glad you see that.  For a very specific need like mine it makes 
all the sense in the world.

> I'm under the vague impression that tccboot already did that.

I didn't know about tccboot and yes, apparently it is non-userspace, which 
is almost perfect.  However it has a lot of stuff I don't need so anyway I 
would have had to do some porting..

> Somebody wanted to add a facility to tcc so it could accept C code from stdin,
> and somebody else _did_ add a -E option so it could spit out a preprocessed
> file.  Having a userspace tcc pass a preprocessed file to a kernel tcc module
> sounds amusing...

Haha that would be amusing, if a bit redundant I guess.

I thought of compiling all the code in userspace and sending the object 
file to the kernel as a module, basically, which gets linked by the 
kernel's own module loader.

I may have to resort to that approach if I find it too difficult to debug 
tcc to behave ok as an embedded feature inside the kernel.

> The accept-from-stdin thing was just a request for feature, I didn't see
> anybody come up with a patch.  I haven't been particularly active since
> Fabrice rendered my tree moot last month, but I've still been reading the
> list and haven't seen it wander by...

Is it that hard to accept stdin?  I mean just but the data in a big string 
buffer then pass it to tcc_compile_string, I suppose.  No?  Doesn't sound 
like a very difficult patch..

> It would be nice if you could stick all this in a .c or .h file that could be
> optionally #included (#define printf printk and so on) so that the baseline
> version could build as a module.
>
> Heck, if you came up with that I'd dust off my tree and merge it there.

Yes, it would.  Hmm.  I will consider this since if would be nice 
wouldnt' it?

> Ok, cleaning that up is one patch.
>
> Since you already replaced malloc(), it should be easy enough to wrap your new
> calls to that with something that records where each allocation was made
> from.
>
> Make a macro that adds the pointer returned by each allocation to a linked
> list along with the __FILE__ and __LINE__ of the call site, and then a
> similar one for free() that traverses the linked list to remove that
> allocation: yes this is hideously inefficient but after a run the list
> contains all your leaks and this works _everywhere_.

Yes, this is what I will have to do.  It never occurred to me to actually 
do this for the mainline tcc tree and then submit it as a patch -- but 
what the hey it probably would be useful to everyone, not just me.

Ok, so presumably I will eventually find all the places that leak memory 
and somehow figure out a way to also free that memory (hopefully not 
having to spend too much time learning the tcc internals to fix it).  Fair 
enough.

> > That's one potential source of bugs.
> >
> > Another is that I fear TCC may exhaust the kernel stack (which is
> > tiny at 8kb).
>
> On newer kernels it's 4k.  Yeah, that's a real issue.
>
> > I am working on that second issue by carefully auditing
> > the code for any large buffers put on the stack.  However there are no
> > guarantees that at runtime TCC doesn't end up recursing heavily as its
> > parsing C code.  I may have to figure out a way to grow the kernel stack
> > forcibly.
>
> Easier to figure out a way to swap stacks.  There was discussion about this on
> linux-kernel...  6 months ago?

Really?  I should look into this as being able to swap stacks to something 
larger for my application is really what I need.  It looks like whatever 
I come up with will be a rather large hack.. :/

> > Anyway, any insights or help with the memory leaks would be appreciated.
> > Do newer versions of TCC addess these leaks?
> >
> > PS: If anyone is interested I can post the code to my "embeddified"
> > version of tcc.
>
> Yes please.

Ok, it's not quite ready yet as it isn't fully stable.  I am trying to 
track down why it crashes after a few compilations.  Could be kernel stack 
being full, could be memory leaks, or something else.

I'll keep you posted and let you know when I have something ready for 
general release.  I also will let you know if/when I come up with a patch 
to mainline tcc for fixing the memory leaks.

-Calin