Re: Java compilation: I don't like Ant

[Jason Wessel]

I think this is another case for batching compiles, or adding a feature to
"collect" a set of files.

Java has the added complexity that more than one class can come out of one
java file (although I personally think it is poor practice to do that).

If you had a tool that computed the Java dependency graph, you are half way
there.  Then if you add batching compilation I think you get what you are
looking for.  I will point out that I have a working implementation of it,
but have been pondering a different implementation (more later). Let us
assume the following basic dependencies.

a.class: ab.java
b.class: ab.java
c.class: c.java
d.class: d.java
e.class: e.java

ab.java is used to derive two different classes.  Attached are the two files
I used for this example. The makefile uses the enhancement I added to make,
called "globalvar".  It allows assignment of a variable from the command
section of make rule.  I use this to collect all the files that need to be
compiled while the rules are being evaluated.  The "all:" rule has a
dependency to "batch_build" which is evaluated after all the .class
dependencies.  The batch_build will pass all the java files it found
dependencies to the compilation routine.

The makefile follows:

CollectRule = $(globalvar to_compile,$(sort $(to_compile) $<))
CompileRule = @echo compiling $(to_compile) ; ./java.pl $(to_compile)
SOURCES = ab.java c.java d.java e.java
OBJECTS = a.class b.class c.class d.class e.class

all: $(OBJECTS) batch_build
 @echo Build complete!

batch_build:
 $(if $(to_compile),$(CompileRule),)

#--Generated by a dependency program from SOURCES--
#Dependencies to other java files
e.class: c.java ab.java
#Dependencies with their correct compile rule
a.class: ab.java
 $(CollectRule)
b.class: ab.java
 $(CollectRule)
c.class: c.java
 $(CollectRule)
d.class: d.java
 $(CollectRule)
e.class: e.java
 $(CollectRule)


I have been using a program to generate make files and then use my special
version of gmake for the last year, and it works quite well.  We actually
use it more for sending multiple files to the Microsoft C++ compiler since
it takes advantage of this similarly to the java compiler.


---------
Another approach that I have been pondering is to create the concept of a
collection dependency.  Basically, all the targets of the collection
dependency would be saved and passed to the collection target rule.  For a
moment assume that the pipe symbol could be used to represent this new type
of dependency which is associated with each target.  Here is what some might
look like:

target | collection_target_rule : dependencies...
%.obj | cpp_comp_rule : %.cpp
%_1.class | java_comp_rule1 : %_1.java

Later you have the collection_target_rule which would allow you to receive
all the targets or all the source dependencies.  The collection target rules
only get executed if one of the collection rules need to be processed.  The
idea being we just do all the collections together.

We need the concept of the sources and targets that were collected from the
collection dependencies above.  Therefore we need to introduce two new
variables to represent them.  $|< will represent the sources that were
collected, and $|@ would represent the targets that were collected, for any
rule that evaluated that it needed to be built.  We would represent the
collection_target_rule with out anything preceding the pipe symbol, so as to
keep the collection rules clearly apart from standard rules. Here is what it
might look like:

| collection_target_rule : depdancies_to_other_rules

| java_comp_rule1 :
    echo Building $|@
    java.pl $|<

| cpp_comp_rule :
    echo Building $|@
    cl.exe $|<


If we go back to the original make file and now have a 1 to 1 correspondence
with the java files to classes.  We can make a very simple Make file which
would be extremely efficient.  It follows bellow.

SOURCES = a.java b.java c.java d.java e.java

## Assume 1 to 1 mapping from .java to .class files
all: $(patsubst %.java,%.class,$(SOURCES))
        @echo Build Complete

## This rule collects the files to be compiled
%.class|java_comp:%.java

## This rule is a batch compiles
|java_comp:
        java.pl $|<

#--Generated by a dependency program--
a.class: a.java b.java
b.class: a.java b.java
c.class: c.java
d.class: d.java
e.class: e.java c.java a.java b.java

Any comments?  I was thinking about writing the code to implement the above
example.


Jason.

----- Original Message -----
From: "David Copeland" <address@hidden>
To: <address@hidden>
Cc: <address@hidden>
Sent: Thursday, April 18, 2002 7:15 AM
Subject: Re: Java compilation: I don't like Ant


> Thanks for the reply.  I guess your characterization of makes
> target/dependant method is really what I meant by "facilitate C
> development".
>
> With Java, there's two main issues:
>
> 1. The Java compiler can compile multiple files at one time much much much
> more quickly than multiple invocations (compare to gcc, where the
difference
> in "gcc foo.c bar.c" and "gcc foo.c; fcc bar.c" is not much).
>
> 2. The Java compiler figures out *some* dependancies (you tell it the root
> of your source tree and the root of our object file tree and it searches
the
> source tree for files to compile that it didn't find in the output tree or
> classpath [libpath]).
>
> Because of this, while you *can* compile Java as you would C (e.g.) (and
> I've made a Makefile that does this), it can be at best slow, and at worst
> problematic (since the compiler will compile files that you didn't
explictly
> tell it to, if it finds a source file for a class used by a class it's
> compiling).
>
> Now, ant doesn't necessarily solve these problems, either.  It's solution
is
> to send the entire source tree to javac each time, and let javac decide
what
> needs to recompile.  Since compiling tons of classes this way isn't much
> slower than compiling one class (really!), this is somewhat effective.
>
> So, coming back to how this could be solved in make, I suppose what
happens
> is that you would need to be able to say that multiple targets are
dependant
> on mulitple files, and that all dependants are given to a command that
will
> generate all the targets, e.g.
>
> $(CLASS_FILES) : $(SOURCE_FILES)
>     $(JAVAC) $(SOURCE_FILES)
>
> but still have make figure out which classes need to be recompilied, based
> on what files are out of date.  Basically, what the class files and target
> files would look like would be:
>
> $(CLASS_OUTPUT_DIR)/com/java/package/structure/here/Foobar.class
> $(SOURCE_DIR)/com/java/package/structure/here/Foobar.java
>
> and that the list of $(TARGET_FILES) would be a bunch like above, but with
> files in various directories, based on their package.
>
> From the list of source files (of the form above) and the CLASS_OUTPUT_DIR
> (root of where all .class files will go; -d option to javac), it would be
> cool to figure out the list of those source files needing recompilation
> (based on mod time, and perhaps some generated dependancies a la gcc -M),
> and then send ALL those files to one invocation of javac.
>
> Now, of course, you could just cheat and do
>
> compile_java :
>     $(ANT) buildclasses
>
> and if all you ever wanted to do was compile your whole source tree, that
> would work.  But, ant is really bad at dynamically changing values of
> variables at runtime, so if you wanted to compile only a subset of your
> source tree, possibly outputting the classes to a different location for
> that build (and this occurs frequently), ant is light years behind make in
> terms of flexibility.
>
> Which brings me back to the main issue, which is "should make be modified
to
> deal with Java?"  Certainly it _could_ be, and perhaps even modified in a
> nice general way.
>
> Dave
>
>
>
>
>
>
>
> >From: "Paul D. Smith" <address@hidden>
> >Reply-To: "Paul D. Smith" <address@hidden>
> >To: address@hidden
> >CC: address@hidden
> >Subject: Re: Java compilation: I don't like Ant
> >Date: Wed, 17 Apr 2002 18:04:04 -0400
> >
> >%% "David Copeland" <address@hidden> writes:
> >
> >   dc> So, for the make developers/maintainers out there: is there an
> >   dc> answer in adding features to make to facilitate Java development
> >   dc> the way it facilitates C development?  Is this a case where a
> >   dc> separate tool really is needed?
> >
> >Hm.  First, realize I've never written a line of Java in my life so I
> >really have only a vague concept of what it takes to "facilitate Java
> >development".
> >
> >Second, I don't think there's anything in make that's really dedicated
> >to "facilitating C development".  Rather, make is designed to work in a
> >"serial" environment where only one target file is generated from zero,
> >one, or more prerequisites.  That fits C, but it also fits a very
> >significant majority of other language environments (as well as lots of
> >other things besides programming).
> >
> >My understanding of Java is that they want to take a bunch of input
> >files and generate a bunch of output files, and that the mapping between
> >the input and output is not simple (that is, can't be calculated using
> >string manipulations on the file names).
> >
> >One possible new feature that I've always wanted which might help Java
> >folks is the ability to tell make that a single invocation of an
> >_explicit_ rule can generate multiple target files.  Right now (as a GNU
> >make extension) you can tell make that a single invocation of an
> >implicit (pattern) rule can generate multiple targets, but not explicit
> >rules.  Doing the latter would require some new syntax, which is one
> >reason I've not really made a serious run at this feature.
> >
> >--
>
>---------------------------------------------------------------------------
----
> >  Paul D. Smith <address@hidden>          Find some GNU make tips at:
> >  http://www.gnu.org
> >http://www.paulandlesley.org/gmake/
> >  "Please remain calm...I may be mad, but I am a professional." --Mad
> >Scientist
>
>
>
>
> _________________________________________________________________
> Send and receive Hotmail on your mobile device: http://mobile.msn.com
>
>
> _______________________________________________
> Help-make mailing list
> address@hidden
> http://mail.gnu.org/mailman/listinfo/help-make
>

java.pl
Description: Perl program

makefile
Description: Binary data