Re: [Savannah-hackers-public] FSF public IP addresses are changing between December 20 and January 7th

[Bob Proulx]

Whew!  It's done!  \o/

Bob Proulx wrote:
> Tomorrow the plan is something like:

Things went mostly according to the plan today.  Only one big snag.
There was an early symptom that git shallow clones (--depth 1) were
failing.  I thought it was using the incorrect buggy version of git
and worked on other problems I thought were more urgent first.  That
was a mistake as the root cause of the problem was preventing write
access and should have been very urgent.  Sorry.  It's been a long two
days.  But the migration is mostly complete now.  There will be a
slowly reducing tale of things discovered that need twiddling.

The root cause of the big snag today was that the nfs mountd on the
old vcs backend was not connecting to the internal0 database server
for account data via libnss.  It was the libnss-mysql part that was
failing.  Confusing because it was working when testing it while
trying to figure out why write access was being denied but apparently
not working at the instant that the nfs mountd was initially started
and this was a sticky state.  Therefore user group ids were not
getting mapped on the backend.  Therefore write access was not being
allowed due to not matching unix file system group permissions.
Restarting the nfs mountd on the backend got this working again.
Unmounting the NFS mount on the client and remounting it finished it.

However I avoided locking myself out of the server today.  I count
that as a win!  And improved the sshd config to hopefully avoid it in
the future.  Time will tell.

Also this adds IPv6 addresses to all of the servers.  Previously they
were IPv4 only.  Now they are fully dual stack.  The are accessible
from IPv6 only networks now.

I also rolled the servers from the hand build iptables rules to using
Shorewall compiled rules.  Others probably like the hand built rules
better but I prefer the safety net.  This saved me a few times because
when making changes to the rules there is a "safe-restart" feature
that saves the old state, tries the new state, if you do not respond
in the affirmative within 60 seconds then shorewall restores the
previous presumably working iptables state.  This is what you need to
know for the new process.  If you don't say 'y' it returns to the
previous iptables state saved before the safe-restart.

  ..edit /etc/shorewall/rules...save...
  shorewall safe-restart   # IPv4
  ...good [y/n] y
  shorewall6 safe-restart  # it is dual stack so IPv6 is separate
  ...good [y/n] y

I also reduced the database access to just two active users now.  One
is a read-only nss-user and the other is the savannahscripts user with
write access.  The others are still in the database but attached to
inaccessible IP addresses.  I'll clean them all out later.  Things had
grown to 46 user-host access pairs which is quite a few.

There is still one regression suite failure that was working prior to
the migration but is failing now after the migration.  The
audio-video-ssh test is failing.  It was definitely working before.

  
https://git.savannah.gnu.org/cgit/administration/savannah-tests.git/tree/test/audio-video-ssh

Assaf do you remember the reason for this access?  You were the
original committer of this test for it.

I need to update the ChangeLog all around.  I will be using
etckeeper's automated git log to remind myself of what was changed.

Bob