Re: Relocatable packages?

[David Bateman]

Benjamin Lindner wrote:
> In the process of putting the pieces for the mingw32 build together
> I stumbled over the fact that apparently octave itself is relocatable
> (provided you keep the directory tree structure) since paths are
> relative to OCTAVE_HOME, but the packages are not.
> You must at least call "pkg rebuild" when moving octave
>
> I feel that this is an unneccesary restriction, since 
> 1) you need write access to the octave tree - this way you'll never be 
> able to run octave e.g. from a cd/dvd, which would be a really cool 
> feature IMO (something like a "Octave Live")
> 2) the need for an "initialization" of the installation/copy is a 
> hindrance in portability, e.g. having octave on a portable drive or 
> USB stick with changing drive letters/mount points in win32. This
> works for octave alone, but breaks as soon as you include the first
> package.
>
> What I'd like to have ultimately is a binary package that does not
> essentially have to be installed, but can be simply downloaded and
> unzipped and voila!
>   
Yes this wouldn't be a bad idea.. The limitation we have is that the
directory to the package is stored in the package structure in an
absolute manner. One idea of this is that the packages themselves might
be individually relocatable rather than under a particular directory.
However, the "pkg install" and "pkg rebuild" commands enforce that the
packages are relative to a particular base address. So in fact at the
moment we have the worse of both world...

So in the end is it better to allow packages to be placed anywhere
individually or is it better to enforce that they are all under a
particular directory and make the package paths in the database relative?

> A "portable octave" would also be a cool feature to have on windows
> (packaging it completely with a compiler and msys environment and an 
> editor and forge-packages and ... and all in an "atomic" package that
> can be moved/copied without the need for installations ...
> just getting visionary now)
>
> So I started and took a look into pkg.m, but I was at totally lost.
> The file is huge and packed with intermeshed functions and - frankly
> said - I didn't get an idea where to start.
>   
Yes, its kind of grown. I try cutting it back a bit from time to time,
but its has rather complex functionality and constraints and so its more
a matter of cleaning the functionality than shrinking the size of pkg..

> Apparently there are two database files in binary format, which 
> contain info on installed/loaded packages. 
> And apparently the installation paths are hardcoded into it.
>   
Yes, that is right..
> My idea was to have an additional flag telling if the global packages 
> (it doesn't make sense for local packages, does it?) are located 
> at the default location and if so, then the paths to the packages 
> are only specifyed relative to OCTAVE_HOME (not absolute) and are 
> reconstructed at runtime when addpath() is executed.
> So my question is:
> a) is this in principle possible with the current package manager?
> b) if so, I'd be grateful for hints where to hook it in, since I'm a bit
> lost by the complexity of the pkg-code. 
>   
With the different API version numbers of each new release of Octave, we
essentially have to redo all packages for each new version of Octave. It
will be like this till at least 3.0. So we don't really need to think
about backward compatibility with binary builds of packages. If we
decide to go this way, then it probably just makes sense to make all
packages relative to the directory containing the the octave_packages
file and be done with it. This would simplify the issue of implementing
this significantly.

My only concern would be whether the distributions want the capability
to install the packages anywhere. If so then we'll have to be a lot
smarter to implement this..

As for where are the hooks that need to be modified, basically its lots
of places.. The first to modify is the "pkg:install" function. The
desc.dir variable needs to be made relative. However, and absolute copy
of it needs to be used later in pkg:install, and so there will be some
issues that will need resolving. A support function

function p1 = packagepath (prefix, p0)
  p1 = fullfile (prefix, p0);
endfunction

might help here.. Basic you then need to search all of the code then for
".dir" and decide what to do at each.. The pkg:rebuild function will
need similar treatment to pkg:install..

> Is there some documentation on the details of the package manager 
> interna? I looked at the manual chapter on packages, but it doesn't
> help in this case
>   
Internals to pkg, there is no documentation. The code is still
continuously evolving, and so there is no point documenting..  I'd be
happy enough trying to implement this, once the needs for the Octave
packaging in the distributions is clarified..

D.

-- 
David Bateman                                address@hidden
Motorola Labs - Paris                        +33 1 69 35 48 04 (Ph) 
Parc Les Algorithmes, Commune de St Aubin    +33 6 72 01 06 33 (Mob) 
91193 Gif-Sur-Yvette FRANCE                  +33 1 69 35 77 01 (Fax) 

The information contained in this communication has been classified as: 

[x] General Business Information 
[ ] Motorola Internal Use Only 
[ ] Motorola Confidential Proprietary