Re: [Savannah-hackers-public] Savannah git policy - non-fast-forward and branches

[Bob Proulx]

Hi Assaf,

Assaf Gordon wrote:
> Regarding recent discussions in other mailing lists (such as the
> recent sv-users) regarding  git policy:
> 
> I'd like to try and understand our policy regarding git (and then
> I'd put a wiki page for it).
> 
> First,
> "non-fast-forward" - this is enabled for all repositories default,
> correct?  (or technically, "denynonfastforward" is 'true') ?

Disabled.  It is disabled for all repositories by default.

I think there is a once a month crontask that scans and sets that
too.  I think.  Not sure.  But pretty sure I have seen that happen
automatically on a low frequence once a month action.  They are
definitely created with denynonfastforward=true by default.

> The reason being that if non-fast-forward commits are allowed, then
> code could be removed or deleted from savannah?

If someone can upload to a repository then with a non-fastforward
commit they could truncate the master branch to nothing.  Immediately
the files would be floating in the repository and available for
recovery.  However cron comes along and runs 'git gc' periodically on
all of the repositories.  That garbage collection would free up the
space and the files would then be reaped and lost.

Unfortunately there are incidences where someone maintaining released
software with a free(dom) software license then decides they want to
take the software commercial.  They then decide they would like to
prevent others from accessing the free(dom) licensed version.  They
attempt to remove the software from the world by zeroing out the
repository.  The denynonfastforward=true configuration is one possible
configuration that prevents this from happening and protects the
freedom of others to be able to continue to access the last free
version of the the software in the repository.

> And also this would cause troubles for people who already cloned the
> repository?  Are there other reasons?

Yes.  Anyone who has cloned the repository previously and if the
upstream "origin" has changed in an incompatible way such as a
non-fastforward commit then have a repository that has been chopped
off from it.  They will need to force the new version from upstream
into their downstream repository.  My experience is that most people
don't know how to do this and simply rm -rf the old and then git clone
again as the simplest brute force solution.

> Second,
> non-master branches seemed to be frowned upon in savannah, or at
> least in GNU projects, is this correct?

I am unaware of any issues with non-master branches.  References?  As
far as I am concerned they are perfectly fine.  However lots of people
use git and everyone has different ideas about how things should be
done.  Look at all of the release branch strategies written about on
the web for example.  If this were being talked about elsewhere there
would be twenty different release strategies proposed within minutes.
There isn't a canonical answer.

However I feel strongly that released branches should not be rewound
with non-fastforward commits.  Once released code should remain
released.  Instead if a specific commit needs to be removed that
specific commit should be reverted.  Basically a reversed commit that
removes it.  The released branch should keep moving forward and should
not back up.

I have previously maintained automated continuous integration build
systems.  Therefore I am sensitive to the problems there.  An
automated build usually moves forward with no trouble.  But rewinds
usually require manual attention to save.  Projects that rewind master
often cause a lot of thrash to CI build systems.

> And by proxy, transient branches that are used for temporary
> development features are not welcomed (because they will be deleted
> later, potentially leading to deleted code) ?

I don't think that way.  I think "next" and "-pu" (proposed updates)
branches used as staging areas for alpha features are fine.  (People
call those all kinds of different things.  What's in a name?)  Stage
there, let people look, then pull from there into master for
releases.  All fine.

Sure sometimes the next branches need to be rewound.  But anyone using
a next branch is working on an unreleased branch in a development area
specifically designated as might be rewound.  The difference is the
difference between a published newspaper and some temporary copy being
passed between the editors internally.  Headlines often change at the
last moment.

And then there are topic branches.  Those don't usually get rewound.
Those usually get deleted entirely once they are no longer useful.

> For example, I've longed wondered why there are couple of
> 'fiemap-copy' branches in GNU Coreutils which seems to be
> abandoned.

I have no idea of the specifics of those.  Looks like a "topic
branch".  Would need to ask the authors.

In my private work areas I often go off on a tangent with some idea
and work on a topic branch.  Sometimes the idea works and those get
pulled into the main trunk.  Sometimes it doesn't work immediately and
I get distracted.  I don't want to throw away that work in that code.
But it also isn't ready for release into the main trunk either.  I
often leave things in the topic branch for a long time.  Later if the
need for that comes up again I can look at what is there and pick back
up working on it again.  Then later if it works out that topic branch
can be rebased against the main trunk master branch and pulled into
it.

In working like this it is judgement of the developer as to how much
is useful to keep and how much is lint that should be cleaned up.
This is just as same spare parts on a work bench.  Too much junk and
some of it must go or the clutter is just too messy to get new work
done.  But a completely clean workbench is a sign that no actual work
is done there. :-)

> Third,
> Combining #1 and #2, means that 'non-fast-forward' commits to
> branches is not allowed (despite being technically possible with a
> hook, as Bob wrote in sv-users) ?

Unfortunately denynonfastforward=true blocks all branches.  I think
that is unfortunate.  I think this is only necessary on the master
branch.

> Fourth,
> There are few exceptions to allowing non-fast-forwards (as happened
> recently).
> These are:
> 1. If the entire source code is available elsewhere (as in
> 'libreJS', where all the code was available under BZR)
> 2. If the project is new, and a first commit was made in error
> (e.g. 'https://savannah.gnu.org/support/?108709') ?
> 3. other cases?

Most often when people are doing conversions.  In the old repository
there is often a lot of cruft that should not be there.  People check
in .o files.  People check in the compiler they used to compile it.
People check in large image files that should have been a tiny small
test image.  People do the darnedest things!  When they are doing a
conversion is a second chance to get it right.

For example Emacs did a conversion and created a 13G repository and
uploaded it.  Immediately everyone's dog wanted to clone a new copy of
it.  Our tuning and resources allowed that to overwhelm vcs and it
brought down Savannah for a while.  I reduced the number of
simultaneous downloads possible to keep below our resource
constraints.  To fix the repository I had Eric compact the repository
and upload it again.  That created a 250M repository on that pass.
(Probably an umpteenth pass for Eric.  We went through several
conversions and uploads before it was reasonable.)  That was a huge
savings all around and well worth it.

> and if we enable non-fast-forward commits, is this temporary for the
> next 24-hours or so until the situation is fixed, or left for good?

I think it should only be temporary.  A new project is registered.
They upload something.  It is messed up.  Five minutes before it
didn't exist.  Five minutes later it does exist but is in an unhappy
state.  Might as well fix things.  There isn't a user base.  There
isn't any base for it at all.

But time goes by.  Say it is five years later.  It will have collected
a following.  It may have been included in downstream software
distributions.  People may have local patches working against it in
their sandbox but haven't contributed them back yet.  At that point I
think it is too late.  It shouldn't be rewound.

The line is somewhere in between.  A grey area.  A judgement call.

And then there are the inevitable cases where someone has committed
files that they should not have committed.  Reasons are listed in the
hosting requirements document.

> Any other important notes regarding this?

Note that the hosting requirements state

  https://savannah.gnu.org/register/requirements.php

  In order to preserve history and complete transparency, we will not
  remove projects with substantive content.

I wrote up some ideas in the wiki about fixing munged commits in the
section "Renaming a branch" here.

  https://savannah.gnu.org/maintenance/Git/

Basically I have asked people to upload to new-master so that I could
compare and review and then if it seems reasonable can do the task.
If there is something that I don't like, such as seeing that it
removed content, then I could discuss it with them and potentially
reject the action.  It seems to me that it is easier to have them
upload something to review than to play twenty questions trying to
figure out what they are wanting to do before they do it.

> As for technicalities,
> When someone asks to enable non-fast-fowrards on the 'master' branch
> (assuming with a valid reason) -
> Does this entail anything beyond logging-in to 'vcs.sv.gnu.org' and
> changing the 'config' file ?

Nope.  That is all.  I use emacs and edit the config files directly.
They are simply text files.  A nicety that isn't available to
non-sv-hackers and therefore people often suggest other random things
that work with github or whatever.  However I think there is a monthly
crontask that comes by and rewrites those files too.

Note also that git is rapidly evolving.  People also often suggest
features that only work with bleeding edge versions.  That is fine.
It means that while 'git merge-base --is-ancestor A B' may be the new
way to do things that 'test "$A" = "$(git merge-base A B)"' is the
portable previously way to do the same thing.  And so forth.  Just an
example.  But be careful that features work in the LTS release we are
actually using.

Bob