[Savannah-hackers-public] Savannah Upgrades Today 2022-10-05

[Bob Proulx]

Savannah Hackers,

This last week I have been working on the systems getting things ready
for the next step of upgrades.  Today I upgraded the database server
from MySQL on Trisquel 8 to MariaDB on Trisuel 9.  And the new VM is
on a better host to move forward upon.  Yay!

That went so well, or so I thought, that I decided to also push the
Savannah web UI frontend from Trisuel 9 on frontend2 over to frontend1
which has been upgraded to Trisuel 10.  Ineiev had already made all of
the Savannah code changes to allow running the web server UI frontend
on the latest Apache and PHP.  It looked good to me to switch today.
This was the previous frontend server when it was Trisquel 8.  It has
been subsequently upgraded to Trisquel 9.  And on to Trisquel 10.  DNS
was switched with a short TTL just in case.  Everything looks good to
me on the new upgraded Trisquel 10 server!  Yay!

And then came in a problem report by IRC that git push was failing due
to permission problems!  Drat!  Just when I had thought things had
gone so well.  Sigh.

I didn't notice because my test for it was flawed as it was still
passing.  I could git clone but when I tried to reproduce the problem
with a git push I experienced the same issue.  The root cause is that
for the purpose of permissions I had a UID but not a list of GIDs
resulting me my git clone being able to read files that I could not
write.  I am going to need to enhance the regression test in order to
alert on this problem in the future.  If it had been correct I would
have known about this problem immediately instead of after a problem
was reported.  It looks like the issue was limited to write action and
not read action.  As the majority of activity is cloning I will hope
that not too many pushes were affected.  Hopefully.

The problem was that the back end storage server had been reconfigured
for the new SQL DB address but the running rpc.mountd was still
hanging onto the previous address.  The rpc.mountd is what associates
users with groups.  It looks up the user and applies groups giving
them write permission.  Meanwhile the uid is set through PAM each time
so did immediately get the new IP address for the new server.  That
allowed git clone over ssh to read the files.  But without the groups
there was no write permission.  This was not something I had expected.
Though in hindsight it does make sense.  The deamon had resolved the
address once and then forever caches it never reading it again.
Restarting it solved this problem.

And that's all for now...

Bob