lmi
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [lmi] Sharing git repositories


From: Greg Chicares
Subject: Re: [lmi] Sharing git repositories
Date: Mon, 9 Mar 2020 23:21:51 +0000
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0

On 2020-03-04 23:24, Vadim Zeitlin wrote:
> On Wed, 4 Mar 2020 22:31:52 +0000 Greg Chicares <address@hidden> wrote:
> 
> GC> On 2020-03-04 00:39, Vadim Zeitlin wrote:
> GC> > On Wed, 4 Mar 2020 00:17:06 +0000 Greg Chicares <address@hidden> wrote:
> GC> [...]
> GC> > GC> Then I discovered
> GC> > GC>   git config --global core.sharedrepository true
> GC> > GC> which seems like a great idea...but is it?
> GC> > 
> GC> >  It definitely should be used for shared bare repositories. This option
> GC> > doesn't affect the working copy files.
> GC> 
> GC> Okay, then how exactly do I use it?
> 
>  This depends on what is the actual goal here and I must admit I'm not
> totally certain about this. Just to explain my point of view/frame of
> reference: typically, Git is used with several repositories. There can be
> just about any relationship between them, but it's pretty common to have
> some central/shared repository (and quite common to have several such
> repositories) and a per-user repository (and maybe several of those too).
> In the canonical case, there is a bare shared repository on some server and
> different users clone it to create their own user-specific repositories.
> Then they can push their changes to the shared repositories and pull the
> changes done by the other users (or by themselves on a different machine!).
> This is how we use Savannah, of course, and also how we use GitHub: both of
> them just use Unix servers hosting (an enormous number of, so I expect it's
> a bit different from hosting a few dozens repositories as I do on my own
> server) bare Git repositories.

We have several use cases, which are the cartesian product of:

(A) our single-user machines
(B) a shared corporate server

crossed with

(0) The public lmi repository.
(1) Cached repositories of wx and wxpdfdoc.
(2) A proprietary repository of product parameters, containing
 - *.policy, *.database, etc. files
 - C++ code to generate those files
 - corresponding *.rates files representing mortality tables
 - system-test data: testdecks and results for proprietary products

For (0), the "golden" repository is on savannah.nongnu.org, with a
mirror on github.com; similarly, for (1), a "golden" repository
exists on github.com . Presumably these are hosted as bare repos,
which are not directly accessible to us as such, so the questions
here don't arise with respect to them. I.e., I can't do
  git config --global core.sharedrepository true
on savannah, and there's no reason why I'd ever want to.

A{0,1,2} and B{0,1} seem to work well enough already (although there's
always presumably room for improvement). The most interesting case is B2,
or perhaps {A,B}2, because (2) is not "canonical" in the sense you use.

Here are more details concerning (2). Kim and I both maintain copies of
it on our local machines. We each have a "blessed" repo:

  /opt/lmi/blessed/proprietary[0]$git rev-parse --is-bare-repository
  true

and a working copy:

  /opt/lmi/proprietary[0]$git rev-parse --is-bare-repository
  false

We make changes to our working copies, sharing all changes as git bundles.
When we both decide that a bundle is "golden", we both push it to our
respective bare repositories at the same time. There is no central server
today. We have a corporate redhat server that we hope can replace Kim's
msw machine, but it can't really be a "central" server in the usual sense:
I can access it only through a VPN, and apparently only on a corporate
laptop, so my personal debian machine can't push to or pull from it.

Even case B1 is "interesting", as these aren't bare repositories:

/srv/cache_for_lmi/vcs/wxWidgets[0]$git rev-parse --is-bare-repository
false
/srv/cache_for_lmi/vcs/wxpdfdoc[0]$git rev-parse --is-bare-repository
false

Those are the result of deleting any cached copies and then running
the normal scripts, e.g., in 'install_wx.sh':

  git clone "$coefficiency" --recurse-submodules "$remote_host_url" $repo_name

Should that be a bare repository instead?

>  But I'd really like to know what are you trying to achieve here, i.e. what
> would be your desired ideal setup and workflow?

That's a bigger question. Ideally, Kim would be able to get rid of the
cygwin installation on her msw laptop, and use this redhat server as a
complete replacement:

 - server is much faster than company laptops
 - no cygwin maintenance (issues with firewall and msw-10 compatibility)
 - no version conflicts between cygwin and debian: we'd both use the
   same exact version of debian (in a chroot)

And ideally I'd be able to use it too, although I'd still do most of
my work on my own computer because that's much more convenient.

Each of us potentially needs a local worktree. Of these ideas:
 - separate, personal chroots, intended for single users only
   (and maybe another for lmi's 'master' branch)
 - separate directories in a single multi-user chroot
which seems better? (Usually we'd just use a designated version of wx
and wxpdfdoc, but we'd want to upgrade those in a personal environment
for testing before updating their versions in lmi 'master'.)

Given that it's a dedicated server, we might as well set up 'cron'
jobs to keep a 'master' build current--e.g.,
 - 'apt-get dist-upgrade' once a week
 - build lmi daily

> GC> Here's a post that seems thoughtful:
> GC> 
> GC> 
> https://stackoverflow.com/questions/3242282/how-to-configure-an-existing-git-repo-to-be-shared-by-a-unix-group/29646155#29646155
> 
>  The post seems correct, but why would you want to configure an _existing_
> repository for sharing instead of creating a new one? Normal course of
> actions is to just use
> 
>       $ git init --bare --shared /path/to/new/repository
> 
> and then either fetch the existing repository from it or git-push with
> --mirror option into it.

Then is the command we use to grab a wx repository (if not already cached):
  git clone "$coefficiency" --recurse-submodules "$remote_host_url" $repo_name
not the normal way? and if not, should we change it?

> GC>   chgrp -R <group-name> .                   # Change files and 
> directories' group
> GC>   chmod -R g+w .                            # Change permissions
> GC>   chmod g-w objects/pack/*                  # Git pack files should be 
> immutable
> GC>   find -type d -exec chmod g+s {} +         # New files get directory's 
> group id
[...]
> GC> I think I understand the four chgrp and chmod commands above.
> GC> But what does
> GC>   git config core.sharedRepository group
> GC> do that those four commands don't already do?
> 
>  If nothing else, it sets the sticky bit to ensure that all the new files
> will be created with the correct (i.e. same) group too. It may have other
> effects, but I'm not aware of them to be honest.

Exactly what do you mean by "sticky bit" here? In the context of the
{rwxXst} bits for chmod, I think you mean setgid (s) rather than
restricted-delete (t, the classic "sticky" bit). But wouldn't this line:

> GC>   find -type d -exec chmod g+s {} +         # New files get directory's 
> group id

above take care of the setgid (s) thing? Is it just that
  git config core.sharedRepository group
takes care of it in a clearer way?

> GC> The author suggests an almost-identical series of commands
> GC> for non-bare repositories, specifically including
> GC>   git config core.sharedRepository group
> GC> Were you implicitly advising against that above:
> GC> >  It definitely should be used for shared bare repositories.
> GC> or were you just emphasizing the "bare" case?
> 
>  Yes, the typical use case is to share a bare repository. Non bare
> repositories can be shared as well, but this can be confusing and
> definitely wastes disk space on the unnecessary working copy, so why would
> you do it?

Okay, it sounds like we should change this line in 'install_wx.sh':

  git clone "$coefficiency" --recurse-submodules "$remote_host_url" $repo_name

but what would we change it to? E.g.:
  git init --bare --shared /path/to/new/repository && git fetch
or
  git init --bare --shared /path/to/new/repository && git push --mirror
or something else?

> GC> I think all our non-bare repositories are created by running
> GC> some command like git-clone, and I could handle each OAOO thus:
> GC>   s/git clone/git clone --config core.sharedRepository=true/
> GC> or so I imagine. Would that alone be sufficient, so that the
> GC> chmod commands above would be unnecessary?
> 
>  I'm still not sure _what_ should it be sufficient for, but this approach
> seems weird to me and generally "Git" and "weird" can result in a lot of
> confusion (although sometimes you can do wonderful things too). To repeat
> in different words what I wrote above, only a single, ideally bare, Git
> repository should be shared. Your personal repositories don't need to set
> core.sharedRepository at all.

That ideal presumes that a central repository exists. But none exists
for our "proprietary" product repository.

For wx and wxpdfdoc, central repositories do exist, and we maintain
a cached version of each locally (outside any chroot) from which we
clone it into a chroot. Does this sentence:

> [...] Your personal repositories don't need to set
> core.sharedRepository at all.

apply to those cached versions?


reply via email to

[Prev in Thread] Current Thread [Next in Thread]