Re: [Texmacs-dev] [Announce] nogencc-0.6

[David Allouche]

On Saturday 04 May 2002 12:20, Joris van der Hoeven wrote:
> Did you try to compile it with gcc 2.96 (before I retry)?

I did not. I will take care of that issue when the code is stable. That 
is probably anytime soon now.

> >   Minor cleanup in array.h and array.cc
> >
> >   Refcounting fix in hashmap and rel_hashmap
>
> Please make the necessary changes in the code only.

Those were cleanups in the rewritten code. I let a few ugliness and a 
couple of refcounting leak slip through previous versions and I fixed 
them when fiddling with const and references.

> >   operator-> defined in basic.h are now const-correct.
>
> Please let us deal with the const problem after the change.

Again, this only affect code I have rewritten anyway. I really do not see 
the point in postponing that particular change.

> >   More regular naming of .cc files.
>
> Please keep the same names as before.
> We will do additional cleanup afterwards.

I did that to allow me to implement aggregate compilation easily and in a 
way consistent with gencc. Now wigdet and boxes .cc files are named after 
the code module they used to define.

I previously had to introduce an exception on the naming scheme 
(Boxes/basic.cc was renamed Boxes/basic_boxes.cc) to make separate 
compilation work; otherwise the object generated from Boxes/basic.cc 
overwrote the object generated from Types/basic.cc. The new naming scheme 
avoid that kind of name clash, is much more regular, and cause no change 
in code.

In future releases, I will include a script which renames all 
files back the original name, so diff would not be confused.


> > Aggregate compilation did not improve the compilation time. Disabling
> > automatic template instanciation does not seem to help either. In the
> > next version I will try a simpler reference conting system in the
> > hope of improving the compilation time.
>
> Bad news. Maybe the C++ template system is just too monstruous
> (as I think it is; too much wisdom kills the wisdom) and far from
> competitive with the inherently better gencc system :^)

Actually, benchmarking shows that aggregate compilation is faster without 
optimization, but can be slower with optimization.

Obviously, aggregate compilation helps save time on parsing (headers 
files are included only once) and creation of generic code 
(uninstanciated templates), thus the better performance w/o optimization. 
But big compilation units seem to slow down the optimizer. It might 
simply be an algorithmic problem (optimization algorithms which are 
slower than O(N) cause slowdown on big units), or it might be that big 
compilation units allow more non-local optimizations. In that case, it 
may even be useful to compile everything in one big monster compilation 
unit to get the best out of the optimizer.


> Did you try to compile with
>
> export CXXFLAGS="-O3 -fexpensive-optimizations -malign-loops=2
> -malign-jumps=2 -malign-functions=2 -fno-exceptions -fno-rtti
> -fno-implicit-templates"

I did not. I will use it for optim_level=high in the next release.

I will not disable automatic template instanciation. Once the 
.rpo files have been generated, that cause no increase in compilation 
time. Actually, it even decrease compilation time since only needed 
template methods are instanciated. And the .rpo files can be generated 
rather quickly by using a first pass of non-optimizing compilation, and 
they could even be included in the distribution, since they are platform 
independant and compatible across versions of g++2.


> I do not think that reference counting will affect the compilation time
> a lot.

My reimplementation of reference counting, resources and events used a 
lot of useless indirections. That had probably no impact on generated 
code since that used only one-line inline methods, but it did have a 
mesurable impact on compilation time.

By the way, I understand that inline is a hint, but I think it is a safe 
bet to consider that one-statement pass-thru inline functions (which 
essentially produce no code) ARE inlined when compiling with 
optimizations.


> If the compilation time does not remain prohibitory (I hope it won't),
> I also remind you that you should adapt the additial programs
> in TeXmacs-misc to the nogencc convention.

Thanks to remind me of that.


On Sunday 05 May 2002 00:37, Joris van der Hoeven wrote:
> I retried a compilation. This time I have hundreds of messages of the
> form
>
>       In file included from Basic/Data/upgradetm.hh:14,
>                        from ./Basic/Data/upgradetm.cc:13:
>       Basic/Types/path.hh:66:24: warning: no newline at end of file
>       Basic/Types/path.hh:66:24: warning: no newline at end of file
>
> You probably did not remove the Emacs comments in the right way
> at the end of the files.

I will have a look. Weirdly, I have no such error here.


> Compilation remains extremely slow; I have the impression
> that compiling one single file is about thrice as fast as
> compiling one aggregate in the official distribution
> (whence an overall slowdown by a factor between two and three :^( ).
> Certain particular files, like Typeset/Env/env_exec.cc are terrible...
> On the other hand, separate compilation also has advantages, of course.

I have benchmarked compilation here. I guess the slowdown you 
observed is due to the use of optimizations by default in my distribution.

The following results were obtained here with gcc-2.95.4, and for 
compilation an linking only, that is after generation of support files 
and a first pass of non-optimized compilation (for generating the .rpo 
files). The reported time is the "user" time as mesured with the "time" 
program.

optim=none -> no optimization flag.
optim=medium -> -O2
optim=high -> -O3 -fexpensive-optimizations -malign-loops=2
              -malign-jumps=2 -malign-functions=2

       version \ optim  | none        medium      high      
------------------------+-----------------------------------------------
TeXmacs-1.0.0.1         | 1m55.820s   13m7.290s   12m47.790s
nogencc-0.6 (SEPARATE)  | 3m22.500s   13m54.550s  13m59.570s
nogencc-0.6 (AGGREGATE) | 1m41.090s   13m52.910s  14m10.760s

And then delta relative to the official release

       version \ optim  | none      medium   high      
------------------------+-----------------------------------------------
TeXmacs-1.0.0.1         |   0          0       0
nogencc-0.6 (SEPARATE)  | + 75 %    +  6 %   +  9 %
nogencc-0.6 (AGGREGATE) | - 13 %    +  6 %   + 11 %

I am currently working on a version which actually compiles faster than 
the official one in all cases but unoptimized separate compilation.

> Also, it is important to allow static compilation: for the official
> TeXmacs binary distributions, I prefer static binaries, which are only
> slightly larger in size and much more stable from the user point of
> view. Standard distributions which support TeXmacs (like Debian and
> Mandrake Cooker) build their own dynamic versions anyway.

I know static compilation is important, but linker flags are arcane. I 
will fix it when other issues are resolved.
-- 

                                  -- David --