[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Package system

From: Paul Kienzle
Subject: Re: Package system
Date: Sun, 23 Oct 2005 10:44:21 -0400

On Oct 23, 2005, at 4:59 AM, Søren Hauberg wrote:

Thanks for the info on octave-forge.

lør, 22 10 2005 kl. 16:59 -0400, skrev Paul Kienzle:
- main/audio creates a binary that should be installed in the helper
binary directory.  I put them into octave_config_info
One problem with this is that not all users have write access to this directory.
Would it be possible to install this binary somewhere else, or would we
loose functionality then?

The binary is called via popen from aurecord.m.  It doesn't matter where
it is so long as octave puts that directory on the executable path
prior to calling it.  How about putting it in .octave/bin/arch if the
package is being installed for a single user, and have octave add that
directory to the PATH environment variable on startup. This is a security
risk, but then so is installing any package you find on the net :-(

- main/audio/data has a sample wav file which I install into the data
path for the package.  In this case, I simply use the package path
itself and find from the script using file_in_loadpath('file.wav').
I see you simply put the .wav file in the same directory as the .m files, so the easy solution is simply put the .wav file in the 'inst' directory. However it might be a good idea to provide some standard way to provide sample data with a package.

There is more than one instance in octave-forge:

$ find octave-forge -name "data" -print

- main/comm/doc has texinfo documentation which should be built and
installed in the usual texinfo places.  Pure documentation packages
such as doc/coda are also possible.
Yesterday, I submitted a patch to the bugs list (not a very good patch, but still) that makes
'help -i function' look for a file called doc.info in the directory
where 'function' lives, and if this file exist use it. If such
functionality gets integrated we would have a standard way of providing
documentation with packages.

You don't want to store docs in the libexec directory, which is where
the oct-file lives. Instead you will want to store them with the m-files,
the data and other shared information for the package.

If we can look up the package containing a particular function and
the share directory for the package then we have the tools to
implement this.  Note that we will want this functionality anyway
for an automated 'see also' to find related commands based on the
contents of INDEX.

Can we use INDEX to gather texinfo strings from all functions into
the same info file?  Many packages (particularly those which are for
both matlab and octave) will have ascii help.  Can we automatically
wrap those for info with a raw text command?

Some packages may have html, doc or pdf help rather than texinfo help.

- main/control is an example of a single function package intended to
extend an existing package. Does it need its own directory? Or can we
request that it be installed in the same directory as the control
package, while updating the list of functions reported by "help
Hmmm, currently the answer is no. I need packages to go into separate directories in order
to make uninstallation easy. I'd really like not to keep track of each
file provided by a package, as I think this makes the system more
simple. It is however possible that it becomes to simple...

Many people only have one or two scripts to contribute.  For now we
can use something like octave-forge to gather them into larger chunks.
This will be more convenient for the user anyway.

- extra/engine has libraries and headers that are needed by other
programs.  They should be installed in /usr/include and /usr/lib or
/usr/local/include and /usr/local/lib.

- extra/mex and main/fixed have libraries and headers that need to be
available for compiling other oct-files.  They should be installed
along side the octave libraries and headers.
The problem here is that not all users have the privileges to do this. We could of course
check this and tell the user if she doesn't have the needed privileges.
This should be possible in the current setup, by providing
'pre_install.m', 'post_install.m', and 'on_uninstall.m'. These could
also handle the installation of the files. However, this really isn't a
very pretty solution (it's really a hack). Personaly, I don't think the
package system should take extra care to make packages like this
possible as I'm guessing there won't be many packages like this.

These can be installed on a per-user basis.  With minor tweaks
to mkoctfile, other packages wouldn't have to know where these
are installed.  E.g.,


- extra/graceplot introduces alternatives. The package system should
support this in some way so that the user can toggle at run time
between graceplot and gnuplot. extra/nan, which replaces some standard
octave commands with alternatives which ignore NaN, could also be
handled with alternatives.
Hmmm, I guess that 'pkg load' (or whatever name we choose) could add the
package to the beginning of the path, thereby overriding other
functions. The user can then choose which functionality to use by the
order in which packages gets loaded. Not very user-friendly, but it
should work.

See how it is done in extra/graceplot and in Debian.  I don't know
the details.

As to the gnuplot/graceplot issue, would it be possible to provide
gnuplot as a package as well (removing gnuplot from core octave)? That
way the user could choose which system to use by loading the right
package. (This reminds me that we should have a 'pkg unload' (or
whatever naming we use))

With a package system John is free to move a number of things out
of the octave tree such as statistics and control systems and into
a less strictly controlled environment like octave-forge where domain
experts can have commit rights.

- configure.base contains configure information useful to a number of
packages.  Much of this is available in modern versions of octave,
through system commands such as octave-config and through the octave
command octave_config_info.  Other things such as the flags to the cp
command, and the details of txi->info conversion should be part of the
packaging system.
Couldn't this file be maintained by octave-forge and then copied to the packages that need it? As to the txi->info issue, I think we need a standard way of providing extra
documentation (see above).

The package system itself can have a configure script to discover
the frequently needed tools.  In particular, if you want to encourage
texinfo tools.  Octave already discovers these and the package system
will be part of octave, so it really is no trouble.

The thing to do is go through configure.base and Makeconf.base
and decide how they can be made available to the package makefile
conveniently through things such as mkoctfile and octave_config_info.
I believe most of them will already be covered.

- anticipate translated help files for multiple languages as part of
the package.
I don't really know how this is going to be implemented. What is the
needs of the translation project?

For each function there will be function.pt, function.es, function.fr,
etc. containing the translated help into language xx.  These files have
a one-line header giving the md5 sum and cvs revision number of the
original text, but other than that they are the usual help extracted

The main problem is that the translation is done by a third party
for each language under a separate source control system.  This is
to avoid having the package developer wait for all the translators to
finish prior to doing a release.

It would be convenient of the package system could query the translation
site to see if that particular package has been translated for a
particular language.  That way packages can be smaller and users don't
have download or install languages they are not interested in.

Alternatively, as part of the package build process, the current
snapshot of the translated text could be added to the package.  If each
translator rebuilds the package when they are finished with their
language and updates the central package repository.

A third alternative is to have pure translation packages which the
user would have to find and install along with the package.

Synchronizing translations with package versions is something I haven't
thought about.

- I didn't see where you created the PKG_ADD file by extracting PKG_ADD
lines from the individual m-files.
I don't, because I really don't know what that means :-)
What is PKG_ADD?

Things like dispatch and mark_as_command which change the way the
interpreter interprets function calls are put in a PKG_ADD file
in each installed directory.  When the user sets the LOADPATH
variable, octave scans the directories added to the path and
executes all the PKG_ADD files it finds.

Some more general comments:

- package version should be available at run time.  Currently I have
OCTAVE_FORGE_VERSION.m for all the packages.  Maybe a version
subcommand on the pkg command is the best way to handle this?
Guess you're right. It was mentioned that matlab has a 'ver' command, which could be implemented on top of 'pkg list'. Perhaps this would give the needed infomation.

Yes.  I discovered this too.

- we should be able to build and test the packages before installing
Yes! Octave-forge has scripts to handle this, right? Could these be integrated with octave?

I think the testing routines have enough real world usage now that
we can incorporate them into octave, but the failure tests need
some work.

My point above is that we will need 'pkg build' and 'pkg test'

- we should note whenever a installed command shadows an existing
Good idea. Shouldn't be to hard to implement.

It gets hard when a script file contains multiple functions.  Then
you need to look for aliases in the PKG_ADD file.  This is new
syntax in the current CVS tree, and I don't remember the details.

- in octave forge most tests are embedded in the files themselves and
discovered with admin/mktests.sh.  The tests should be installed along
with the functions so that the user can see at run time what tests the
function is supposed to pass. Also some tests are actually demo blocks
which are part of the documentation of the function.  Some tests are
embedded in C++ files.  These will need to be extracted before
installing.  In retrospect tests should perhaps be in separate .test
files in both cases, and maybe separate .demo files for the demos.
So what you're suggesting is that octave provides two function
*) test('somefile.test')
*) demo('somefile.demo')
And that packages provide the .test and .demo files?

Octave forge looks for a file named 'fn' or 'fn.m' or 'fn.cc' on the
path.  If it finds one then it extracts the tests from there.  This
should be changed to look for 'fn.test' in addition.

Type 'help test' on octave-forge for details of the test syntax.

In addition to 'test' blocks there are also 'demo' blocks which
are extracted and displayed to the user using the example command
and run with the demo command.  I find I don't use the example
command, however it would be convenient if any examples could
be automatically extracted and included with the documentation for
the function.

- for windows systems which may not have the compilers installed, we
may want to support binary packages.  This would be advised for OS X
systems as well, where tracking down and installing third party
libraries prior to installing the package would be a pain.
I choose not to call 'make install', but instead move files from the 'src' directory to the 'inst' directory, to make creation of binary packages easy. While
not implemented, it should be very easy to create binary packages.
Simply stop the installation before installing any files.
So it's not implemented, but it's on the todo :-)

E.g., with a 'pkg build' command.

- the packaging system should be available and useful for matlab as
well. That way authors of packages such as wavelab will have less work
to do to support octave.  For example, if you automatically generate
the contents.m from INDEX and admin/make_index the contents file will
be easier to maintain and it is a win for them.  Similarly, we should
Do the Right Thing with a usual matlab tarball which has contents.m
rather than COPYING DESCRIPTION and INDEX files.
In general, I think we should provide scripts for creating different kinds of packages.
It seems like we need .rpm, .dep, and perhaps also matlab package
creator scripts. I don't know the first thing about any of these
formats, but we should have this functionality. Perhaps a 'Package
Creator' package should be available?

The flip side is that it would be nice to be able to take a usual
matlab package file (whatever that is) and install it in octave,
similar to something like alien.

Once again, thanks for the input

reply via email to

[Prev in Thread] Current Thread [Next in Thread]