Re: [lmi] [PATCH] Trivial patch to avoid building unnecessary wxWidgets parts

[Vadim Zeitlin]

On Fri, 06 Mar 2015 21:27:36 +0000 Greg Chicares <address@hidden> wrote:

GC> On 2015-03-06 18:19, Vadim Zeitlin wrote:
GC> > On Fri, 06 Mar 2015 18:14:18 +0000 Greg Chicares <address@hidden> wrote:
GC> > 
GC> > GC> I don't seem to have recorded how long it took previously, but...
GC> > 
GC> >  Me neither exactly because it was too long to redo it again just to
GC> > benchmark it. It takes less than 10 minutes now (inside my 2 CPU VM, I
GC> > should probably give it 4 of them...) which is definitely a gain.
GC> 
GC> Please say exactly what you're doing that takes less than ten minutes.
GC> Are you actually running the same command I gave:
GC> 
GC> /lmi/src/lmi[1]$time make wx_version=3.1.0 
wx-3.1.0-md5=3c049ed2f470d81627ea5d513a06aa15 $coefficiency -f install_wx.make 
>../log 2>&1

 Yes, I do use the same command with two minor exceptions: I don't use the
output redirection and I don't use "time" neither but rely on zsd
REPORTTIME option which shows me exactly the same report as time does at
the end if the command takes longer than 10 (my REPORTTIME value) seconds
to run:

make $coefficiency -f install_wx.make wx_version=3.1.0  -s  57.05s user 146.03s 
system 36% cpu 9:13.59 total


 Because the line above seemed so suspicious to me (see below), I also just
redid the same command with "time" in front but the results are almost the
same, so at least I'm reassured that I can continue relying on REPORTTIME
in the future (I strongly suspect it and built-in time command reuse the
same implementation anyhow):

make $coefficiency -f install_wx.make wx_version=3.1.0  -s  62.21s user 159.18s 
system 39% cpu 9:25.09 total


GC> (which does something like 'make clean' first), and using a MinGW
GC> gcc-3.4.5 toolchain? Does your "2 CPU VM" have two cores with that
GC> function as four due to hyperthreading, one that functions as two
GC> for the same reason, or two cores with hyperthreading turned off?

 But this is where the problem was, sorry for confusing you. I completely
forgot (this must be the phrase I use the most in my correspondence...)
that I had already increased the number of CPUs to 4, exactly because
building wx inside the VM was so slow. So the numbers above are with 2
(emulated) CPUs with 2 cores each, so 4 logical CPUs in all.

 FWIW VMware doesn't seem allow to configure hyperthreading, it probably
isn't very advantageous for the VMs.

GC> It's about half that speed in my VM:
GC>  541.79s user 821.04s system 165% cpu 13:41.95 total

 What I find strange here is that you have 165% CPU utilization while I
have 36%. It clearly must be due to the difference between KVM and VMware
but I have no idea why is it so. FWIW your numbers seem more realistic as
the CPU usage shown by the task manager is visibly quite high here as well:
it starts at ~0 because configure is completely sequential (and probably IO
bound) and ends at ~50% because linking is also IO bound, but during the
entire compilation part the 4 CPUs remain pegged at 100%.

 So the conclusion I make is that KVM is better at reporting the time to
the programs running inside the VM but VMware seems to have less overhead
so the actual time is better for it.

 As a final comparison, building all (i.e. including all the parts excluded
in install_wx.make now) wxWidgets DLLs in release build using MSVC 12 takes
64 seconds. Again, I don't think even the latest g++ is as fast as MSVC is,
but I hope it could come closer to it than 3.4.5 does... And these numbers
also explain why I prefer to use MSVC so much, especially as the difference
is even greater in debug (non-optimized) builds.

GC> This is 32-bit xp, but at no point during these runs do I have less
GC> than a gigabyte of spare RAM available.

 Yes, my RAM usage peaks at ~1.8GB and I allocated 4GB to this VM, so this
is clearly not the limiting factor.

GC> In the VM, and also in native xp, I'm using a recent version of
GC> cygwin, perhaps a few months old; I know of no dramatic speedup
GC> they've achieved recently.

 And my Cygwin installation is of about the same age.

 Regards,
VZ