Re: Compiled load path issues

[Andy Wingo]

Hello ludovic,

On Tue 20 Oct 2009 10:27, address@hidden (Ludovic Courtès) writes:

> Andy Wingo <address@hidden> writes:
>
>> On Sun 18 Oct 2009 17:36, address@hidden (Ludovic Courtès) writes:
>
> [...]
>
>>> Andy: can you comment?  What was the idea behind
>>> ‘%load-compiled-path’?
>>
>> The idea is that given that the compiled files are
>> architecture-dependent,
>
> In theory, we could interpret the ‘.go’ cookie and byte-swap things if
> needed...

In theory yes. In practice we map things read-only so they can be cached
and not copied, and we'd have to instrument individual VM ops with
checks based on the current objcode, flags for the objcode to say their
format, etc. I really think it's too complicated. If this is really the
way we want to go, we should give up on having endian-specific
bytecode -- which is a bigger task, not to mention the tail wagging the
dog.

>> that they should go in $libdir instead of $datadir.  We can add
>> $libdir, but I don't think it's a good idea -- not only for reasons of
>> excessive stat, but because I don't think we should be putting
>> binaries in with installed source.
>
> By now people may have started to update their packages to run
> “guile-tools compile” and install ‘.go’ files, so we really need to get
> this issue settled.
>
> I’m in favor of ‘.go’ alongside ‘.scm’: that’s what happens with
> .elc/.el and .pyc/.py and it had been the plan from 1.9.0 until
> recently.

For python, pyc files are in $libdir, for exactly this reason. Plus, you
might have some source files that you want to compile with multiple
versions of Guile. I don't think we should be encouraging this.

>>> Besides, ‘scm_search_path ()’ was changed incompatibly compared to 1.8
>>> in 22f4ee48822db5e30df3abf9a11b6066f2bab9d3.  I’m wary about such
>>> incompatibilities and would like it if we could (1) list them, and
>>> (2) avoid them unless we really really can’t think of any other way.  In
>>> this particular case, do you have an idea on how to avoid it?
>>
>> I don't really know. I'm sure it could be worked around somehow, but
>> it's not very fun work.
>
> It’s not, but there’s a fair amount of not very fun work in this vain to
> be done by 2.0.  :-)
>
> I think we must pay close attention to backwards compatibility, at least
> to honor long time promises
> (http://lists.gnu.org/archive/html/guile-devel/2003-02/msg00074.html).

So for the list, we did have a chat about this on IRC. We both agree
that we should not needlessly introduce incompatibilities, especially on
the C level. This is especially a problem because e.g. gnucash,
configuring as it is with guile.m4 and guile-config, will simply pick up
the new version of Guile when it's installed -- which is like upgrading
when you didn't choose to. Gnucash should only have to be concerned with
Guile when it chooses to.

For that reason we also think that Guile should be parallel-installable,
at least on the library level. That means that we should have the
version in the library name, and the version in the include path; so
pkg-config --cflags guile-2.0 will say e.g. -I/usr/include/guile-2.0,
and that pkg-config --libs guile-2.0 will be e.g. -lguile-2-0, or
something. A more detailed manifesto is here:

  http://www106.pair.com/rhp/parallel.html

  Removing barriers to new version adoption

  The big benefit of parallel installation is that you remove the reason
  why people are reluctant to upgrade to a new version of Foo, because
  upgrading to Foo 5 has no effect on users of Foo 4. This means that
  the packages in [the distro] can be upgraded by their upstream
  maintainers, one at a time. It means that GNOME packages can upgrade
  to Foo 5 one at a time. It means that if I use a text editor dependent
  on Foo 4, but want my package to use Foo 5, I can do that since I can
  install both versions of Foo at the same time.

  In short, parallel install nukes the chicken and egg problem, and it
  saves everyone a bunch of time and energy.

  (A side effect: suddenly you have far more freedom to break backward
  compatibility; your new incompatible version is in fact a different
  library, not the same library. So if you need to clean up a big nest
  of cruft, no big deal. No one is forced to upgrade until they have
  time to deal with the breakage.)

That last paragraph is key for me. Sure, we need to make
well-thought-out changes -- but our current policy of very extended
C-level compatibility is very, very limiting, and a big energy drain. If
we think we need to change a function interface, well, we just change
it, and document the change as well -- perhaps even with a Coccinelle[0]
patch. So whenever Lilypond realizes that Guile 2.2 gives them native
Scheme compilation, but changes a couple of functions, they can weigh
their choice, and if they decide to upgrade, they know what to do.

[0] http://lwn.net/Articles/315686/

Anyway, comments welcome. Ludovic let me know if this actually
represents what you think, or if I'm just putting bytes in your mouth :)

Cheers,

Andy
-- 
http://wingolog.org/