Re: [lmi] Updating install_wx.make to get wxWidgets from git

[Greg Chicares]

On 2018-03-24 14:54, Vadim Zeitlin wrote:
> 
>  I'd like to finally make the long-promised patch implementing getting
> wxWidgets from Git repository, instead of using GitHub snapshots, which
> don't include submodules, but I realized that I'm not sure what would be
> the best way to do it. The question is whether we should optimize for the
> "once only" case or for repeated invocations of install_wx.make (with
> either the same or different wx_commit_sha values)?

A few weeks ago, I thought I might try to update wx manually myself
for my new "buster" chroot. (If that turned out to be simple, then
the makefile would just follow naturally.) Although I got distracted
by other things before I could make any real progress, I did realize
that we must answer this important question first. I even came up
with an answer, but it led to other questions...

TL;DR: 'git clone' once and only once; something like:
  https://www.wxwidgets.org/develop/code-repository/
git clone --recurse-submodules https://github.com/wxWidgets/wxWidgets.git

And it would be best to clone into a 'cache_for_lmi' directory.
For cygwin, that's "C:\cache_for_lmi". For my chroots, I maintain
  /srv/cache_for_lmi/
in my "host", and to create a new chroot I copy its contents, e.g., to
  /srv/chroot/lmi-buster/cache_for_lmi/
That's where we already keep downloadable files like these:
  $cache_dir/downloads/wxWidgets*.zip
  $cache_dir/downloads/libxml2-2.9.4.tar.gz
and also a bare git repository of proprietary product parameters:
  $cache_dir/blessed/proprietary/

Caching's main benefit is robustness. It's not uncommon for us to move
to a new machine, a new cygwin installation, or a new chroot; far too
often, downloads fail because of server outages (sf.net especially)
or corporate firewall blocking. Saving time and bandwidth are nice
benefits too, of course.

Where should this initial wx clone go? Easy: in its own subdirectory:
  $cache_dir/appropriate_name_for_wx_git_clone

Then I started wondering whether we want this repository to be bare.
Usually, when I start researching the git documentation, I get lost,
and walk away in puzzlement when some other pressing issue demands my
attention; that's what happened here. Do we want 'git clone --bare'?
Or some "push --mirror" command as this article
  https://help.github.com/articles/duplicating-a-repository/
recommends?

It might help to explain how we use our proprietary bare repositories.
Kim and I both have our own bare copies and our own working copies.
(Our bare repositories serve the function of a central online repo,
which doesn't exist, as all the contents are proprietary.) One of us
changes a working copy and emails a bundle to the other for acceptance
testing; when accepted, we both push to the "blessed" bare repository.
Then, when we need to migrate the system, we copy that up-to-date bare
repository to $cache_dir/blessed, and copy the entire contents of
$cache_dir to the new machine--maybe a few times a year at most. Thus,
we have three or four copies:

 (0) bare, in /srv/cache_for_lmi
This is outside the scope of 'install_wx.make', and indeed beyond its
reach, because it's not even visible to any chroot. The analog for Kim
would be an intranet subdirectory if the intranet worked correctly, but
in reality an external file-sharing service that actually does work.

 (1) bare, in /src/chroot/buster/cache_for_lmi or C:\cache_for_lmi
 (2) bare, in /src/chroot/buster/opt/lmi/blessed or C:\opt\lmi\blessed
 (3) working copy in .../opt/lmi/proprietary
  
For wx, I'm guessing we'd want something like (1) and (3). We need (3)
because wx can be built only in a working copy. I don't see why we'd
necessarily want anything like (2): it exists only to be pushed to,
but we don't anticipate that we'd push anything to wx. And we'd want
something like (1), I suppose, so we can 'git clone wx' only once per
lifetime; and $cache_dir seems to be the best place for our clone,
because that's where we cache downloads and pulling to the clone is
analogous downloading files.

Oh, and while we're doing this for wxWidgets, we should probably do
exactly the same thing for wxPdfDoc.

> In the latter case
> (which is mine), it would make sense to do "git clone" once, if the
> directory doesn't exist, or just "git clean" + "git fetch" if it does
> already exist to significantly speed up the second and subsequent
> executions, but at the price of making the first run slower as the full
> clone takes more time than a shallow clone retrieving just the given commit
> (although it's difficult to say by how much exactly, without doing several
> time-consuming tests).

I think we all want to do a full, deep clone of wx, and do it only
once per lifetime; then pull periodically. This is probably what
you do, and it's probably best for us not to deviate. We'd also
gain the ability to explore the entire wx history, experimentally
apply local patches, etc.