[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Partial linking with _RELOCATABLES - Proposed enhancement

From: Marc Alff
Subject: Partial linking with _RELOCATABLES - Proposed enhancement
Date: Sun, 19 Mar 2006 11:11:58 -0700
User-agent: Thunderbird 1.5 (X11/20051201)

Hi all

This email describe a proposed enhancement to Automake, to support partial linking (also known as "ld -r").

This is the discussion thread; a proposed patch is available on address@hidden

All the errors are mine and sorry for the long babble,
but I think it's important to explain why this enhancement can help is some cases.

============= Part I : What is partial linking ? =============

Building code typically consist of a chain like this :

*.c, *.cpp --> *.o --> lib*.a, lib*.so or binaries

Partial linking allows to do this :

part1.o, part2.o ... partn.o --> glued.o

A typical "partial link" or "relocatable (-r)"  command is :
ld -r -o glued.o part1.o part2.o ... partn.o

As a result, a build chain can be longer :

*.c, *.cpp --> *.o --> * glued.o --> * super-glued.o --> lib*.a, lib*.so or binaries

The difference between "glued.o" and "libglued.a" is that a "relocatable" object contains all the code in one block, with internal link dependencies already resolved,
where *.a is just an aggregation of individual objects in an archive.
In some cases, that DOES matter (see below).

=============== Part II : Motivation ===============

1) Organizational

Consider a project that delivers only 1 binary at the end,
but which consist of so many files, components and configuration options that it's highly desirable
to organize the code base in a lot or directories, sub directories etc.
For example, the linux kernel :)
In this case, linking a kernel image from a huge list of *.o is very impractical; instead it's better to link small parts together first, and link bigger parts from "convenience" relocatables.

The main concern is that, how the source code is organized into directories
with recursive makefiles is an implementation choice,
it should **not** impact what the final deliverable looks like (maybe I still want ONE *.a or *.so or binary at the end,
not expose a collection of *.a or *.so to my users).

2) Technical limitations due to volume of code

Sometimes you have no choice : linking all the *.o at once makes the command line too big, or flat out kills the linker.
There is also a performance aspect to it in extreme cases
(Historical note: I have first seen this technique used 15 years ago on a project that would take 1 full day to build on a dedicated machine, using ld -r was just the only way to get a result at all).

3) Subtle differences between static libraries, dynamic libraries and relocatables

Consider a program where main.o contains all the "core" logic, and is capable or calling plug-ins. Various plug-ins plugin-1.o, plugin-2.o, ... all implement the same plug-in interface.

The problem is that main.o does not contain any call to (read : does not reference any symbol present in) plugin-*.o,
since that code is **independent** of the plug-ins implementation.

3-a) Using static libraries (libplugin-1.a)

Link the binary with main.o -llibplugin-1.a ... nothing happens

To make things work, you have to "register" the plug-ins somewhere, to force a reference to them.

In C, this can be done like this :

$ cat all_plugin.c

#ifdef HAVE_PLUGIN_1
#include "plugin-1.h"

#ifdef HAVE_PLUGIN_2
#include "plugin-2.h"


struct plugininfo * plugin_tab [] = {

#ifdef HAVE_PLUGIN_1
& plugin_1_info,

#ifdef HAVE_PLUGIN_1
& plugin_2_info,


} ;

It works if you know at build time what to use, and if the list of available
plug-ins is fairly controlled and known, which might be an invalid assumption.

3-b) Using dynamic libraries

This time, plug-ins are loaded dynamically using dlopen().
This works great if the choice of which plug-in to use is done at runtime (instead of build time),
but you need a configuration file somewhere to list what dll to dlopen().

3-c) Using relocatables and C++

Each plugin contains a line like this :

static PluginInfo myInfo(...) ;

The C++ class PluginInfo is implemented in main.o, and happen to do something like :

  all_plugin.register(this) ;

What happens is magic between __main and _main due to the static myInfo().

The final binary is linked with :
main.o all-plugin.o more-plugin-if-you-like.o

Note that using
- main.o
- main.o all-plugin.a more-plugin-if-you-like.a
would **not** produce the same result (at least, last time i tried).

The interesting point is :
- no logic ever in the core (main.o) implementation ever reference explicitly plugin-x.o - there is no configuration file either to maintain with a list of dll to dlopen().

From the end-user point of you : I need X : I put X.o in my link command; I don't : then I don't.
That's it, KISS. No configuration to mess with.

This is a remote case, but I happen to depend on it heavily for a project of mine, where the "plug-in" stuff happen to be generated code that varies all the time.

I think this enhancement has value just because of point 1) alone,
and can help in case of projects with a fairly sizable code base.

============ Part III : Proposed enhancement ============

In short, can be written like this

noinst_RELOCATABLES = glued.o
glued_o_SOURCES = part-1.cpp part-2.cpp

glued.o is a **notation**, not the actual name of a file :

..._RELOCATABLES = glued.o super-glued.obj
glued_o_... = ...
super-glued_obj_... = ...

will work both on platforms that use .o or .obj for objects,
and in both cases the generated makefile will contain names like :
glued.$(OBJEXT) : ...
super-glued.$(OBJEXT) : ...

You can add these if you like :
glued_o_LDR = /my/own/ldr/program
glued_o_LDRFLAGS = -please -make -section blah -start -at -address blah
glued_o_LIBADD = more-part-X.$(OBJEXT) more-part-Y.$(OBJEXT)

The generated Makefile uses LIBTOOL to actually do the linking work,
so it should be portable : $LIBTOOL --mode=link xxx

============ Part IV : Conclusion ============

This enhancement has been implemented already, since I happen to need it and depends on it,
and it has been tested on some platforms already.

The point here is to propose it for discussion, and possibly inclusion in the "official" Automake code base.

Thanks all for your comments,
and thanks to Ralf Wildenhues for early comments and suggestions that encouraged this babble :)

Marc Alff.

reply via email to

[Prev in Thread] Current Thread [Next in Thread]