[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Re-write of documentation tree build process
From: |
Han-Wen Nienhuys |
Subject: |
Re: Re-write of documentation tree build process |
Date: |
Fri, 15 Dec 2006 17:34:33 +0100 |
User-agent: |
Thunderbird 1.5.0.8 (X11/20061107) |
John Mandereau escreveu:
> Hello all,
>
> Here's a (long-awaited) patch against the stable/2.10 branch.
>
> 'make web' can now generate two documentation trees in
> <src-dir>/out-www:
>
> - one in online-root/, where .png and .html extensions are stripped in
> HTML files, for hosting on a web site with content negociation.
>
> - the other in /offline-root, for browsing the docs on a hard disk.
>
> Only one of these trees or both can be generated, depending on the
> FINAL_TARGETS variable:
>
> make FINAL_TARGETS=offline web # same as 'make web'
> make FINAL_TARGETS=online web
>
> only builds one tree, and
>
> make FINAL_TARGETS="offline online" web
>
> builds both.
>
> If this can be applied and a possible link to the French tutorial in
> index.html.in is added, Jan and Han-Wen may want to release 2.10.3 or
> 2.10.2-2, as the current docball is broken because of .html stripping
> (maybe the 2.11 docball is broken too).
Hello John,
thanks for your patch. Before it's applied, I think it should be
cleaned up a bit first.
First of all: do you think you could shove the entire
local-WWW-post
target into a python script? In that case, we don't need any awkward
quoting, and can junk most of the command line decoding, as we could
do
import mirrortree
mirrortree.miror_tree ('out-www/offline-webroot', {'ignore-dirs':
'foo.*(ext|') , 'lose-dirs': 'more-regexes'})
(or use keyword arguments to pass options.)
> diff --git a/stepmake/bin/add-html-footer.py b/stepmake/bin/add-html-footer.py
> index 08ff5ff..9a28237 100644
> --- a/stepmake/bin/add-html-footer.py
> +++ b/stepmake/bin/add-html-footer.py
> @@ -13,7 +13,7 @@ import getopt
> index_url=''
> top_url=''
> changelog_file=''
> -content_negotiation = False
> +targets = []
I think that this script could be cleaned up a lot if didn't try to do
two targets in one go. Can you look into doing something like
find online-root -name '*.html' | xargs $(footify)
find offline-root -name '*.html' | xargs $(footify)
if that's unpractical, it's probably better to do one target a time,
so you can remove the
for t in targets:
if t == 'online'
elif t == 'offline'
logic.
> +def help ():
> + sys.stdout.write (r"""Usage: build-trees [OPTIONS]... TARGETS
> +Build trees for different targets by hardlinking input trees.
> +
> +Options:
> + --input-root=DIR use DIR as input tree root (default is current
> directory,
> + multiple roots may be specified with a comma
> separator)
> + --target-pattern=STRING use STRING as target root directory name pattern
> + --lose-dir=PATTERN lose directories whose name matches PATTERN in
> copies
> + (write its content to parent)
is this to remove out-www from the paths? I'd call it
--strip-component or similar. The difference between --exclude-dir is
not clear.
> + --process-dir=PATTERN only process files in directories whose name
> matches PATTERN
"only process directories matching PATTERN"
> + --common-files=PATTERN filters files commmon to all trees
> + (they are hardlinked instead of being copied).
> + --specific-files=PATTERN filters files specific to different trees
> + (regular files are only hardlinked to first
> target).
> + --exclude-dir=PATTERN don't recurse into directories whose name matches
> PATTERN
> + --exclude-files=PATTERN exclude files whose name matches PATTERN
> + --dump-sfl=FILE dump specific files list to FILE
"sfl " -> no abbreviations please.
> + -h, --help print this help
> +
> +PATTERN should be a Python regular expression.
> +Common and specific files which are symlinks are always copied to all
> targets.
> +""")
> + sys.exit (0)
> +
> +(options, targets) = getopt.getopt(sys.argv[1:], 'h', [
> + 'input-root=', 'lose-dir=', 'common-files=', 'help', 'specific-files=',
> + 'exclude-dir=', 'exclude-files=', 'dump-sfl=', 'process-dir=',
> 'target-pattern='])
Nowadays, optparse is the standard module for handling option
parsing. Can you change to that? See lilypond-book.py
(get_option_parser) for an example how to use.
> +input_roots = []
> +target_pattern = ''
> +
> +common_f = '.*'
> +specific_f = '.*'
> +excluded_d = ''
> +excluded_f = ''
> +process_d = '.*'
> +lost_d_names = ''
> +sfl_dump = ''
> +
> +#file_list_re = re.compile (r'(?:^| )(?:[^ ]*|".*?")(?: |$)')
> +
> +for opt in options:
> + o = opt[0]
> + a = opt[1]
> + if o == '--input-root':
> + input_roots = re.split (',', a)
> + elif o == '--lose-dir':
> + lost_d_names = a
> + elif o == '--common-files':
> + common_f = a
> + elif o == '--specific-files':
> + specific_f = a
> + elif o == '-h' or o == '--help':
> + help ()
> + elif o == '--exclude-dir':
> + excluded_d = a
> + elif o == '--exclude-files':
> + excluded_f = a
> + elif o == '--process-dir':
> + process_d = a
> + elif o == '--target-pattern':
> + target_pattern = a
> + elif o == '--dump-sfl':
> + sfl_dump = a
> + else:
> + raise 'unknown opt ', o
> +
> +if input_roots == []:
> + input_roots = ['.']
> +
> +for s in ['common_f', 'specific_f', 'excluded_d',
> + 'excluded_f', 'process_d', 'lost_d_names']:
> + exec s + '_re = re.compile (' + s + ')'
Don't use exec. If you must, try
locals()[s + "_re"] = ..
what's _f and _d? Write _dir or _file if you mean that.
> +strip_lost_d_names_re = re.compile ('/(?:' + lost_d_names + ')')
> +slash_re = re.compile ('/')
> +
> +
> +if '%s' in target_pattern:
> + target_dirs = map (lambda s: target_pattern % s, targets)
usually, list comprehensions are shorter and clearer,
[p % s for p in targets]
> +for dir in target_dirs:
> + os.mkdir (dir)
map(os.mkdir, target_dirs)
if you like short.
> +path_strip = lambda bigpath, root: bigpath[:len (root)]
what's this?
> +
> +def new_link_path (link, dir, r):
> + l = slash_re.split (link)
> + d = slash_re.split (dir)
> + i = 0
> + while l[i] == '..':
> + if r.match (d[i]):
> + del l[i]
> + else:
> + i += 1
> + return reduce (lambda s1, s2: s1 + '/' + s2,
> + filter (lambda x: not r.match (x), l))
If you want to do it functionally, use zip to fold (l,d) together,
then filter on the d component, and unzip the resulting list.
> + sfl = open (sfl_dump, 'w')
> +
> +for d in input_roots:
> + for in_dir, dirs, files in os.walk(d):
> + i = 0
> + while i < len(dirs):
> + if excluded_d_re.search (dirs[i]):
> + del dirs[i]
> + else:
> + i += 1
> + out_dir = strip_lost_d_names_re.sub ('', in_dir)
> + if not lost_d_names_re.match (os.path.basename (in_dir)):
> + for t in target_dirs:
> + os.mkdir (os.path.join (t, out_dir))
> + if not process_d_re.search (in_dir):
> + continue
> + for f in filter (lambda s: not excluded_f_re.match (s),
> files):
preferred:
for f in files:
if excluded_f_re.match (f):
continue
> + in_f = os.path.join (in_dir, f)
> + if os.path.islink (in_f): # all symlinks are assumed to be
> relative and to point to files in the input trees
> + link_path = new_link_path (os.readlink (in_f), in_dir,
> lost_d_names_re)
> + for t in target_dirs:
> + os.symlink (link_path, os.path.join (t, out_dir, f))
> + elif specific_f_re.match (f):
> + os.link (in_f, os.path.join (target_dirs[0], out_dir, f)) #
> only hardlink specific file in first target
> + if sfl_dump:
> + sfl.write (os.path.join (out_dir, f) + '\n')
> + elif common_f_re.match (f):
> + for t in target_dirs:
> + os.link (in_f, os.path.join (t, out_dir, f))
> +if sfl_dump:
> + sfl.close()
should be unnecessary.
--
Han-Wen Nienhuys - address@hidden - http://www.xs4all.nl/~hanwen
LilyPond Software Design
-- Code for Music Notation
http://www.lilypond-design.com
- Re-write of documentation tree build process, John Mandereau, 2006/12/14
- Re: Re-write of documentation tree build process,
Han-Wen Nienhuys <=
- Re: Re-write of documentation tree build process, John Mandereau, 2006/12/17
- Re: Re-write of documentation tree build process, John Mandereau, 2006/12/20
- Re: Re-write of documentation tree build process, Han-Wen Nienhuys, 2006/12/20
- Re: Re-write of documentation tree build process, John Mandereau, 2006/12/21
- Re: Re-write of documentation tree build process, Juergen Reuter, 2006/12/21
- Re: Re-write of documentation tree build process, Jan Nieuwenhuizen, 2006/12/21
- Re: Re-write of documentation tree build process, John Mandereau, 2006/12/23
- Re: Re-write of documentation tree build process, Han-Wen Nienhuys, 2006/12/23