Re: [gnulib-tool-py] The root of the problem

[Bruno Haible]

Hi Dmitriy,

> Here is the scheme how I imagine it in the most simple case:
> 
> 1. User gives a list of modules which he wants to operate with.
> 2. For each module in $USER_MODULES:
>   1). Check whether the name of the module contains in any module from
> $LIST_MODULES or $LIST_DEPENDENCIES (this allows to avoid
> GLModuleSystem.find before adding, so we gain a serious speedup). If not,
> we use GLModuleSystem.find on it and add GLModule instance to $LIST_MODULES.
>   2). Get dependencies of the created instance. Then for each dependency
> (which is a string, like in a 1).) check if such string contains
> $LIST_MODULES or $LIST_DEPENDENCIES. If not, we use GLModuleSystem.find on
> it and add GLModule instance to $LIST_DEPENDENCIES.
>   4) Go to the next module in list of $USER_MODULES.

I think what you are describing here is the translated implementation
of func_modules_transitive_closure? Your algorithm is the same for the
basics (there's not many ways to program a "transitive closure" algorithm
<http://en.wikipedia.org/wiki/Transitive_closure>), but in the case of
gnulib-tool we want to parameterize its execution through the options
  --with[out]-tests
  --with[out]-c++-tests
  --with[out]-longrunning-tests
  --with[out]-privileged-tests
  --with[out]-unportable-tests
  --with-all-tests
  --avoid

> I've thought that only two errors can happen here:
> a) User module or dependency module does not exist.
> b) User module or dependency module is broken.
>
> These errors are easy to handle

As discussed a couple of days ago, let's only throw exceptions for the
moment, and not catch/handle them. That comes later, depending on real
use situations.

> In this case we don't ever need a cache, we just need two separated lists
> to store modules added by user and modules added as dependencies.

The cache is meant to make the creation of GLModule objects fast.

In particular, the code for --import and --create-testdir will request
the configure.ac part of each module, then the Makefile.am part of each
module, then the file list of each module, etc. Without a cache,
gnulib-tool would start to parse the modules/* files each time anew.
The idea of the cache is that each module description file is read into
memory only once, and dissected into sections only once.

If you can achieve this goal without a cache, sure, do it.

> To add modulename to $LIST_MODULES and its dependencies to
> $LIST_DEPENDENCIES, we use GLModules.addModule(modulename).
> To remove modulename from $LIST_MODULES and its dependencies from
> $LIST_DEPENDENCIES, we use GLModuleSystem.removeModule(modulename).
> Of course $LIST_MODULES and $LIST_DEPENDENCIES will be not global
> variables, but attributes of the GLModuleSystem. I just used such names to
> make things more clear.

WARNING! There is not a single list of modules and a single list of
dependencies. Look for example at func_modules_transitive_closure_separately
and how --single-configure works. There is a list of specified modules,
a list of included non-tests modules for the lib/ directory, a list of
non-tests modules for the tests/ directory, a list of included tests
modules for the tests directory, and so on.

Do *not* design things so as to assume that there is a global variable
such as "the list of all modules". Instead, we need a data structure
(possibly a class? possibly only a combination of a 'dict' and a list?) that
represents a list of modules. We will need at least 3, maybe more, instances
of this data structure.

Bruno