Re: benchmarks (was Re: Progress on a Classpath mauve suite?)

[Dalibor Topic]

David P Grove wrote:
> I think you need to understand the point of the benchmark suite.  The 
> whole goal is reproducible science, so if someone doesn't cite the exact 
> version of the benchmarks, then it isn't useful (in an academic sense). 

Sure. On the other hand, nothing in the free software world prevents 
people from citing the exact version of free software used. That's 
simply proper academic conduct, I belive, and not something that needs 
to be governed by licenses. I fully agree the goal is reproducible 
science, and that goal should be obvious enough for a scientist not to 
have to explicitely tie our hands behind their backs to make us do it, 
right? :)

> The license forces that plus proper academic credit (ie a citation) for 
> the benchmark suite, which personally I think is quite fair given how much 
> work was put into putting it together (much more than the typical academic 
> paper). 

I agree that academic credit is very important. Again, I belive that to 
be a part of proper academic conduct, rather than something that needs 
to be explicitely enforced in a software license. Lawyers can't fix bad 
science. :)

> I know some of the people who put together the benchmark suite fairly well 
> and it was a massive effort (think person-years, not person-weeks; good, 
> usable, portable, benchmark suites are a lot of work).  It would be a 
> shame if your politics prevented you from using it, but that's life I 
> suppose.

Given that the nice scientists who put together DaCapo are using (as far 
as I can tell) exclusively free software for their benchmarks, I find it 
a little strange that they went to such great lengths to create a 
software license for their work (which is combining other people's free 
software) that prohibits to 'alter, modify, improve, decompile, 
disassemble or otherwise reverse-engineer the obtained object code'.

If there is any LGPLd code in there, they may be violating the LGPL with 
that clause (3.1) as the LGPL says

"  6. As an exception to the Sections above, you may also combine or
link a "work that uses the Library" with the Library to produce a
work containing portions of the Library, and distribute that work
under terms of your choice, provided that the terms permit
modification of the work for the customer's own use and reverse
engineering for debugging such modifications."

unless the LGPLd code has some exception that allows them to do so. The 
aim of this clause is to precisely avoid the case where people close off 
free software by tacking on too restrictive terms to it.

I wouldn't call that politics, different people have different goals, 
and we have to respect their wishes, even if it makes their software 
unuseable for others to build upon. Just like DaCapo authors have to 
respect the wishes of the authors of the works they chose to build their 
work upon.

cheers,
dalibor topic