Re: [ft-devel] Integrating Docwriter into FreeType

[Cosimo Lupo]

It's not a good idea to have the `make refdoc` target call pip to install missing requirements in the user's python lib.

I don't want those packages to pollute my environment, if I just want to build the freetype documentation.

You should either exit with error if the requirements are not met, and maybe tell the user to run pip install -r requirements.txt

for themselves; 

or you could bootstrap a new python virtual environment, e.g. like we do in the noto-source repository:

https://github.com/googlei18n/noto-source/blob/c9e564be34ea2efe72f958016d4d8d74fad860b6/build#L20-L47

this way the only hard requirement besides python is pip itself (that's the responsibility of the user or the distro to set up).

All the rest is self-contained and doesn't affect the host environment.

This would also avoid issues like the one Werner reported where there is a conflict between the version of Pygments installed globally (via the distro package manager) and the one required by the docwriter.

On Fri, Jul 27, 2018 at 11:18 AM Werner LEMBERG <address@hidden> wrote:

> I have started integrating docwriter in the savannah repository. It
> is available in the 'GSoC-2018-nikhil' branch.

Thanks!

> I have added the required commands to make target 'refdoc', and
> converted the header files to markdown.  To generate the static
> documentation site, please run `make refdoc` at the top-level.  The
> docs should be generated in 'docs/reference/site/' folder. TOC and
> index files are 'ft2-toc/index.html' and 'ft2-index/index.html'.

I tried the following on a freshly cloned FreeType repository (branch
`GSoC-2018-nikhil'):

  make devel
  make refdoc
    Installing requirements...
    python -m pip install --user -r \
      .../freetype2/src/tools/docwriter/requirements.txt
  /usr/bin/python: No module named pip
  .../freetype2/builds/freetype.mk:296: recipe for target 'refdoc' failed
  make: *** [refdoc] Error 1

This is with python 2.7.13, as installed on openSuSE 42.3.

I get the same error if I do `autogen.sh & configure & make refdoc'.

So, the first step should be to check whether `pip' is installed,
aborting otherwise (with a proper error).  Note that we currently
don't do any configure checks for `make refdoc', which means that
checks should be included into `make refdoc' itself.

After installation of the openSuSE `pip' bundle, I tried again (note
that some Python packages are already installed via openSuSE's package
manager):

--------

Installing requirements...
python -m pip install --user -r \
                        /home/wl/git/freetype/freetype2.nikhil/src/tools/docwriter/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): mistune==0.8.3 in /usr/lib/python2.7/site-packages (from -r /home/wl/git/freetype/freetype2.nikhil/src/tools/docwriter/requirements.txt (line 1))
Collecting mkdocs<=0.17.5,>=0.17.3 (from -r /home/wl/git/freetype/freetype2.nikhil/src/tools/docwriter/requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/f9/49/cf13bb1e0d079624ce145bbe80e7a11a7cd7ace16149c3f28a43c8a72a43/mkdocs-0.17.5-py2.py3-none-any.whl (1.2MB)
    100% |################################| 1.2MB 278kB/s
Collecting mkdocs-material<=2.9.1,>=2.9.0 (from -r /home/wl/git/freetype/freetype2.nikhil/src/tools/docwriter/requirements.txt (line 3))
  Downloading https://files.pythonhosted.org/packages/38/3c/f22621b0d5b349e7ee6b711255ebdf02425ebd775267a0e959947eb40325/mkdocs_material-2.9.1-py2.py3-none-any.whl (570kB)
    100% |################################| 573kB 476kB/s
Collecting pymdown-extensions<=4.11,>=4.10.2 (from -r /home/wl/git/freetype/freetype2.nikhil/src/tools/docwriter/requirements.txt (line 4))
  Downloading https://files.pythonhosted.org/packages/7d/0b/8a645733fc6ad20c1217808d2bf26eb80996901d173e623f886b75f62239/pymdown_extensions-4.11-py2.py3-none-any.whl (205kB)
    100% |################################| 208kB 1.4MB/s
Requirement already satisfied (use --upgrade to upgrade): PyYAML>=3.10 in /usr/lib64/python2.7/site-packages (from -r /home/wl/git/freetype/freetype2.nikhil/src/tools/docwriter/requirements.txt (line 5))
Collecting Markdown>=2.3.1 (from mkdocs<=0.17.5,>=0.17.3->-r /home/wl/git/freetype/freetype2.nikhil/src/tools/docwriter/requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/6d/7d/488b90f470b96531a3f5788cf12a93332f543dbab13c423a5e7ce96a0493/Markdown-2.6.11-py2.py3-none-any.whl (78kB)
    100% |################################| 81kB 687kB/s
Requirement already satisfied (use --upgrade to upgrade): click>=3.3 in /usr/lib/python2.7/site-packages/click-6.7-py2.7.egg (from mkdocs<=0.17.5,>=0.17.3->-r /home/wl/git/freetype/freetype2.nikhil/src/tools/docwriter/requirements.txt (line 2))
Collecting tornado<5.0,>=4.1 (from mkdocs<=0.17.5,>=0.17.3->-r /home/wl/git/freetype/freetype2.nikhil/src/tools/docwriter/requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/e3/7b/e29ab3d51c8df66922fea216e2bddfcb6430fb29620e5165b16a216e0d3c/tornado-4.5.3.tar.gz (484kB)
    100% |################################| 487kB 543kB/s
Collecting Jinja2>=2.7.1 (from mkdocs<=0.17.5,>=0.17.3->-r /home/wl/git/freetype/freetype2.nikhil/src/tools/docwriter/requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/7f/ff/ae64bacdfc95f27a016a7bed8e8686763ba4d277a78ca76f32659220a731/Jinja2-2.10-py2.py3-none-any.whl (126kB)
    100% |################################| 126kB 250kB/s
Collecting livereload>=2.5.1 (from mkdocs<=0.17.5,>=0.17.3->-r /home/wl/git/freetype/freetype2.nikhil/src/tools/docwriter/requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/dd/b4/213daced3ff1b4e02a1f700748e20e9a7481f5bfef57d11ae9babfd4aa2f/livereload-2.5.2-py2.py3-none-any.whl
Collecting Pygments>=2.2 (from mkdocs-material<=2.9.1,>=2.9.0->-r /home/wl/git/freetype/freetype2.nikhil/src/tools/docwriter/requirements.txt (line 3))
  Downloading https://files.pythonhosted.org/packages/02/ee/b6e02dc6529e82b75bb06823ff7d005b141037cb1416b10c6f00fc419dca/Pygments-2.2.0-py2.py3-none-any.whl (841kB)
    100% |################################| 843kB 235kB/s
Collecting singledispatch (from tornado<5.0,>=4.1->mkdocs<=0.17.5,>=0.17.3->-r /home/wl/git/freetype/freetype2.nikhil/src/tools/docwriter/requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/c5/10/369f50bcd4621b263927b0a1519987a04383d4a98fb10438042ad410cf88/singledispatch-3.4.0.3-py2.py3-none-any.whl
Collecting certifi (from tornado<5.0,>=4.1->mkdocs<=0.17.5,>=0.17.3->-r /home/wl/git/freetype/freetype2.nikhil/src/tools/docwriter/requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/7c/e6/92ad559b7192d846975fc916b65f667c7b8c3a32bea7372340bfe9a15fa5/certifi-2018.4.16-py2.py3-none-any.whl (150kB)
    100% |################################| 151kB 236kB/s
Collecting backports-abc>=0.4 (from tornado<5.0,>=4.1->mkdocs<=0.17.5,>=0.17.3->-r /home/wl/git/freetype/freetype2.nikhil/src/tools/docwriter/requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/7d/56/6f3ac1b816d0cd8994e83d0c4e55bc64567532f7dc543378bd87f81cebc7/backports_abc-0.5-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23 (from Jinja2>=2.7.1->mkdocs<=0.17.5,>=0.17.3->-r /home/wl/git/freetype/freetype2.nikhil/src/tools/docwriter/requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/4d/de/32d741db316d8fdb7680822dd37001ef7a448255de9699ab4bfcbdf4172b/MarkupSafe-1.0.tar.gz
Requirement already satisfied (use --upgrade to upgrade): six in /usr/lib/python2.7/site-packages (from livereload>=2.5.1->mkdocs<=0.17.5,>=0.17.3->-r /home/wl/git/freetype/freetype2.nikhil/src/tools/docwriter/requirements.txt (line 2))
Installing collected packages: Markdown, singledispatch, certifi, backports-abc, tornado, MarkupSafe, Jinja2, livereload, mkdocs, Pygments, pymdown-extensions, mkdocs-material
  Running setup.py install for tornado
  Running setup.py install for MarkupSafe
Successfully installed Jinja2-2.10 Markdown-2.6.11 MarkupSafe-1.0 Pygments-2.0 backports-abc-0.5 certifi-2018.4.16 livereload-2.5.2 mkdocs-0.17.5 mkdocs-material-2.9.1 pymdown-extensions-4.11 singledispatch-3.4.0.3 tornado-4.5.3
You are using pip version 7.1.2, however version 18.0 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.                                           
Running docwriter...
python -B /home/wl/git/freetype/freetype2.nikhil/src/tools/docwriter/docwriter.py \
                  --prefix=ft2                          \
                  --title=FreeType-2.9.1           \
                  --output=/home/wl/git/freetype/freetype2.nikhil/docs/reference                   \
                  /home/wl/git/freetype/freetype2.nikhil/include/freetype/*.h                     \
                  /home/wl/git/freetype/freetype2.nikhil/include/freetype/config/*.h              \
                  /home/wl/git/freetype/freetype2.nikhil/include/freetype/cache/*.h
INFO    -  Cleaning output directory
INFO    -  Building markdown documentation to directory: /home/wl/git/freetype/freetype2.nikhil/docs/reference/markdown
WARNING -  Undefined cross reference 'FTC_Property_Set'.
WARNING -  Undefined cross reference 'FTC_Property_Get'.
Building static site...
cd /home/wl/git/freetype/freetype2.nikhil/docs/reference && mkdocs build
Traceback (most recent call last):
  File "/home/wl/.local/bin/mkdocs", line 11, in <module>
    sys.exit(cli())
  File "/usr/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/wl/.local/lib/python2.7/site-packages/mkdocs/__main__.py", line 155, in build_command
    site_dir=site_dir
  File "/home/wl/.local/lib/python2.7/site-packages/mkdocs/config/base.py", line 181, in load_config
    errors, warnings = cfg.validate()
  File "/home/wl/.local/lib/python2.7/site-packages/mkdocs/config/base.py", line 105, in validate
    post_failed, post_warnings = self._post_validate()
  File "/home/wl/.local/lib/python2.7/site-packages/mkdocs/config/base.py", line 85, in _post_validate
    config_option.post_validation(self, key_name=key)
  File "/home/wl/.local/lib/python2.7/site-packages/mkdocs/config/config_options.py", line 433, in post_validation
    config[key_name] = theme.Theme(**theme_config)
  File "/home/wl/.local/lib/python2.7/site-packages/mkdocs/theme.py", line 47, in __init__
    self._load_theme_config(name)
  File "/home/wl/.local/lib/python2.7/site-packages/mkdocs/theme.py", line 77, in _load_theme_config
    theme_dir = utils.get_theme_dir(name)
  File "/home/wl/.local/lib/python2.7/site-packages/mkdocs/utils/__init__.py", line 373, in get_theme_dir
    return os.path.dirname(os.path.abspath(theme.load().__file__))
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2354, in load
    self.require(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2371, in require
    items = working_set.resolve(reqs, env, installer)
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 844, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.VersionConflict: (Pygments 2.0 (/usr/lib/python2.7/site-packages/Pygments-2.0-py2.7.egg), Requirement.parse('Pygments>=2.2'))
/home/wl/git/freetype/freetype2.nikhil/builds/freetype.mk:296: recipe for target 'refdoc' failed
make: *** [refdoc] Error 1

--------

Please advise.


    Werner

_______________________________________________
Freetype-devel mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/freetype-devel

--

Cosimo Lupo