Re: [ft-devel] Integrating Docwriter into FreeType

[Cosimo Lupo]

I can confirm that running `make refdoc` from a within a fresh virtual environment (after having removed the --user) option, works for me.

I'm running python 2.7.14 on Debian.

I don't get the error that Werner gets, because I'm installing all the dependencies from scratch in the new virtual environment, instead of relying on a mixture of pip-installed packages and others provided by the distro package manager.

On Fri, Jul 27, 2018 at 12:17 PM Cosimo Lupo <address@hidden> wrote:

Besides, the `--user` option of pip install is incompatible with a virtual environment, you should remove that in any case, because it will prevent users like me, who would prefer to install the dependencies in a temporary virtual environment, from being able to run `make refdoc` from within an activated python virtual environment.

I know that without --user it would fails because then pip would attempt to install to the global python environment where it doesn't have write access (without sudo, which is even worse, don't use that ever), but that's exactly the point I'm making before, that you shouldn't force a user to permanently modify their global or user's environment just in order to build the documentation.

On Fri, Jul 27, 2018 at 12:14 PM Cosimo Lupo <address@hidden> wrote:

It's not a good idea to have the `make refdoc` target call pip to install missing requirements in the user's python lib.

I don't want those packages to pollute my environment, if I just want to build the freetype documentation.

You should either exit with error if the requirements are not met, and maybe tell the user to run pip install -r requirements.txt

for themselves; 

or you could bootstrap a new python virtual environment, e.g. like we do in the noto-source repository:

https://github.com/googlei18n/noto-source/blob/c9e564be34ea2efe72f958016d4d8d74fad860b6/build#L20-L47

this way the only hard requirement besides python is pip itself (that's the responsibility of the user or the distro to set up).

All the rest is self-contained and doesn't affect the host environment.

This would also avoid issues like the one Werner reported where there is a conflict between the version of Pygments installed globally (via the distro package manager) and the one required by the docwriter.

On Fri, Jul 27, 2018 at 11:18 AM Werner LEMBERG <address@hidden> wrote:

> I have started integrating docwriter in the savannah repository. It
> is available in the 'GSoC-2018-nikhil' branch.

Thanks!

> I have added the required commands to make target 'refdoc', and
> converted the header files to markdown.  To generate the static
> documentation site, please run `make refdoc` at the top-level.  The
> docs should be generated in 'docs/reference/site/' folder. TOC and
> index files are 'ft2-toc/index.html' and 'ft2-index/index.html'.

I tried the following on a freshly cloned FreeType repository (branch
`GSoC-2018-nikhil'):

  make devel
  make refdoc
    Installing requirements...
    python -m pip install --user -r \
      .../freetype2/src/tools/docwriter/requirements.txt
  /usr/bin/python: No module named pip
  .../freetype2/builds/freetype.mk:296: recipe for target 'refdoc' failed
  make: *** [refdoc] Error 1

This is with python 2.7.13, as installed on openSuSE 42.3.

I get the same error if I do `autogen.sh & configure & make refdoc'.

So, the first step should be to check whether `pip' is installed,
aborting otherwise (with a proper error).  Note that we currently
don't do any configure checks for `make refdoc', which means that
checks should be included into `make refdoc' itself.

After installation of the openSuSE `pip' bundle, I tried again (note
that some Python packages are already installed via openSuSE's package
manager):

--------

Installing requirements...
python -m pip install --user -r \
                        /home/wl/git/freetype/freetype2.nikhil/src/tools/docwriter/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): mistune==0.8.3 in /usr/lib/python2.7/site-packages (from -r /home/wl/git/freetype/freetype2.nikhil/src/tools/docwriter/requirements.txt (line 1))
Collecting mkdocs<=0.17.5,>=0.17.3 (from -r /home/wl/git/freetype/freetype2.nikhil/src/tools/docwriter/requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/f9/49/cf13bb1e0d079624ce145bbe80e7a11a7cd7ace16149c3f28a43c8a72a43/mkdocs-0.17.5-py2.py3-none-any.whl (1.2MB)
    100% |################################| 1.2MB 278kB/s
Collecting mkdocs-material<=2.9.1,>=2.9.0 (from -r /home/wl/git/freetype/freetype2.nikhil/src/tools/docwriter/requirements.txt (line 3))
  Downloading https://files.pythonhosted.org/packages/38/3c/f22621b0d5b349e7ee6b711255ebdf02425ebd775267a0e959947eb40325/mkdocs_material-2.9.1-py2.py3-none-any.whl (570kB)
    100% |################################| 573kB 476kB/s
Collecting pymdown-extensions<=4.11,>=4.10.2 (from -r /home/wl/git/freetype/freetype2.nikhil/src/tools/docwriter/requirements.txt (line 4))
  Downloading https://files.pythonhosted.org/packages/7d/0b/8a645733fc6ad20c1217808d2bf26eb80996901d173e623f886b75f62239/pymdown_extensions-4.11-py2.py3-none-any.whl (205kB)
    100% |################################| 208kB 1.4MB/s
Requirement already satisfied (use --upgrade to upgrade): PyYAML>=3.10 in /usr/lib64/python2.7/site-packages (from -r /home/wl/git/freetype/freetype2.nikhil/src/tools/docwriter/requirements.txt (line 5))
Collecting Markdown>=2.3.1 (from mkdocs<=0.17.5,>=0.17.3->-r /home/wl/git/freetype/freetype2.nikhil/src/tools/docwriter/requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/6d/7d/488b90f470b96531a3f5788cf12a93332f543dbab13c423a5e7ce96a0493/Markdown-2.6.11-py2.py3-none-any.whl (78kB)
    100% |################################| 81kB 687kB/s
Requirement already satisfied (use --upgrade to upgrade): click>=3.3 in /usr/lib/python2.7/site-packages/click-6.7-py2.7.egg (from mkdocs<=0.17.5,>=0.17.3->-r /home/wl/git/freetype/freetype2.nikhil/src/tools/docwriter/requirements.txt (line 2))
Collecting tornado<5.0,>=4.1 (from mkdocs<=0.17.5,>=0.17.3->-r /home/wl/git/freetype/freetype2.nikhil/src/tools/docwriter/requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/e3/7b/e29ab3d51c8df66922fea216e2bddfcb6430fb29620e5165b16a216e0d3c/tornado-4.5.3.tar.gz (484kB)
    100% |################################| 487kB 543kB/s
Collecting Jinja2>=2.7.1 (from mkdocs<=0.17.5,>=0.17.3->-r /home/wl/git/freetype/freetype2.nikhil/src/tools/docwriter/requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/7f/ff/ae64bacdfc95f27a016a7bed8e8686763ba4d277a78ca76f32659220a731/Jinja2-2.10-py2.py3-none-any.whl (126kB)
    100% |################################| 126kB 250kB/s
Collecting livereload>=2.5.1 (from mkdocs<=0.17.5,>=0.17.3->-r /home/wl/git/freetype/freetype2.nikhil/src/tools/docwriter/requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/dd/b4/213daced3ff1b4e02a1f700748e20e9a7481f5bfef57d11ae9babfd4aa2f/livereload-2.5.2-py2.py3-none-any.whl
Collecting Pygments>=2.2 (from mkdocs-material<=2.9.1,>=2.9.0->-r /home/wl/git/freetype/freetype2.nikhil/src/tools/docwriter/requirements.txt (line 3))
  Downloading https://files.pythonhosted.org/packages/02/ee/b6e02dc6529e82b75bb06823ff7d005b141037cb1416b10c6f00fc419dca/Pygments-2.2.0-py2.py3-none-any.whl (841kB)
    100% |################################| 843kB 235kB/s
Collecting singledispatch (from tornado<5.0,>=4.1->mkdocs<=0.17.5,>=0.17.3->-r /home/wl/git/freetype/freetype2.nikhil/src/tools/docwriter/requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/c5/10/369f50bcd4621b263927b0a1519987a04383d4a98fb10438042ad410cf88/singledispatch-3.4.0.3-py2.py3-none-any.whl
Collecting certifi (from tornado<5.0,>=4.1->mkdocs<=0.17.5,>=0.17.3->-r /home/wl/git/freetype/freetype2.nikhil/src/tools/docwriter/requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/7c/e6/92ad559b7192d846975fc916b65f667c7b8c3a32bea7372340bfe9a15fa5/certifi-2018.4.16-py2.py3-none-any.whl (150kB)
    100% |################################| 151kB 236kB/s
Collecting backports-abc>=0.4 (from tornado<5.0,>=4.1->mkdocs<=0.17.5,>=0.17.3->-r /home/wl/git/freetype/freetype2.nikhil/src/tools/docwriter/requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/7d/56/6f3ac1b816d0cd8994e83d0c4e55bc64567532f7dc543378bd87f81cebc7/backports_abc-0.5-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23 (from Jinja2>=2.7.1->mkdocs<=0.17.5,>=0.17.3->-r /home/wl/git/freetype/freetype2.nikhil/src/tools/docwriter/requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/4d/de/32d741db316d8fdb7680822dd37001ef7a448255de9699ab4bfcbdf4172b/MarkupSafe-1.0.tar.gz
Requirement already satisfied (use --upgrade to upgrade): six in /usr/lib/python2.7/site-packages (from livereload>=2.5.1->mkdocs<=0.17.5,>=0.17.3->-r /home/wl/git/freetype/freetype2.nikhil/src/tools/docwriter/requirements.txt (line 2))
Installing collected packages: Markdown, singledispatch, certifi, backports-abc, tornado, MarkupSafe, Jinja2, livereload, mkdocs, Pygments, pymdown-extensions, mkdocs-material
  Running setup.py install for tornado
  Running setup.py install for MarkupSafe
Successfully installed Jinja2-2.10 Markdown-2.6.11 MarkupSafe-1.0 Pygments-2.0 backports-abc-0.5 certifi-2018.4.16 livereload-2.5.2 mkdocs-0.17.5 mkdocs-material-2.9.1 pymdown-extensions-4.11 singledispatch-3.4.0.3 tornado-4.5.3
You are using pip version 7.1.2, however version 18.0 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.                                           
Running docwriter...
python -B /home/wl/git/freetype/freetype2.nikhil/src/tools/docwriter/docwriter.py \
                  --prefix=ft2                          \
                  --title=FreeType-2.9.1           \
                  --output=/home/wl/git/freetype/freetype2.nikhil/docs/reference                   \
                  /home/wl/git/freetype/freetype2.nikhil/include/freetype/*.h                     \
                  /home/wl/git/freetype/freetype2.nikhil/include/freetype/config/*.h              \
                  /home/wl/git/freetype/freetype2.nikhil/include/freetype/cache/*.h
INFO    -  Cleaning output directory
INFO    -  Building markdown documentation to directory: /home/wl/git/freetype/freetype2.nikhil/docs/reference/markdown
WARNING -  Undefined cross reference 'FTC_Property_Set'.
WARNING -  Undefined cross reference 'FTC_Property_Get'.
Building static site...
cd /home/wl/git/freetype/freetype2.nikhil/docs/reference && mkdocs build
Traceback (most recent call last):
  File "/home/wl/.local/bin/mkdocs", line 11, in <module>
    sys.exit(cli())
  File "/usr/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/wl/.local/lib/python2.7/site-packages/mkdocs/__main__.py", line 155, in build_command
    site_dir=site_dir
  File "/home/wl/.local/lib/python2.7/site-packages/mkdocs/config/base.py", line 181, in load_config
    errors, warnings = cfg.validate()
  File "/home/wl/.local/lib/python2.7/site-packages/mkdocs/config/base.py", line 105, in validate
    post_failed, post_warnings = self._post_validate()
  File "/home/wl/.local/lib/python2.7/site-packages/mkdocs/config/base.py", line 85, in _post_validate
    config_option.post_validation(self, key_name=key)
  File "/home/wl/.local/lib/python2.7/site-packages/mkdocs/config/config_options.py", line 433, in post_validation
    config[key_name] = theme.Theme(**theme_config)
  File "/home/wl/.local/lib/python2.7/site-packages/mkdocs/theme.py", line 47, in __init__
    self._load_theme_config(name)
  File "/home/wl/.local/lib/python2.7/site-packages/mkdocs/theme.py", line 77, in _load_theme_config
    theme_dir = utils.get_theme_dir(name)
  File "/home/wl/.local/lib/python2.7/site-packages/mkdocs/utils/__init__.py", line 373, in get_theme_dir
    return os.path.dirname(os.path.abspath(theme.load().__file__))
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2354, in load
    self.require(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2371, in require
    items = working_set.resolve(reqs, env, installer)
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 844, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.VersionConflict: (Pygments 2.0 (/usr/lib/python2.7/site-packages/Pygments-2.0-py2.7.egg), Requirement.parse('Pygments>=2.2'))
/home/wl/git/freetype/freetype2.nikhil/builds/freetype.mk:296: recipe for target 'refdoc' failed
make: *** [refdoc] Error 1

--------

Please advise.


    Werner

_______________________________________________
Freetype-devel mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/freetype-devel

--

Cosimo Lupo

--

Cosimo Lupo

--

Cosimo Lupo