[Savannah-cvs] [265] BackEndSetup: new page

[assafgordon]

Revision: 265
          
http://svn.sv.gnu.org/viewvc/?view=rev&root=administration&revision=265
Author:   agn
Date:     2016-10-20 03:02:56 +0000 (Thu, 20 Oct 2016)
Log Message:
-----------
BackEndSetup: new page

Modified Paths:
--------------
    trunk/sviki/SavannahHosts.mdwn

Added Paths:
-----------
    trunk/sviki/BackEndSetup.mdwn

Added: trunk/sviki/BackEndSetup.mdwn
===================================================================
--- trunk/sviki/BackEndSetup.mdwn                               (rev 0)
+++ trunk/sviki/BackEndSetup.mdwn       2016-10-20 03:02:56 UTC (rev 265)
@@ -0,0 +1,143 @@
+Savannah Frontend Setup
+=======================
+
+This page describes the configuration needed for savannah's
+new backend server and perl/python code (e.g. the scripts
+that run on `vcs0` which create new repositories for the projects
+based on database updates from the frontend).
+
+Further reading about savannah's server:
+
+* [[SavannahArchitecture]] - overview of the current ('old') setup
+  (i.e. vcs, mgt, frontend, internal, vcs, download).
+* [[SavannahServices]] - services/daemons available on savannah's hosts.
+* [[SavannahInternals]] - Savannah's inner-workings.
+* [[SavannahHosts]] - Configuration of the new servers
+  (i.e. mgt0, vcs0, frontend0, internal0, download0).
+
+
+
+General information
+-------------------
+
+Long ago, savannah's used a single server. This lead to
+the frontend code (php) and backend scripts (perl/python)
+to reside in the same repository, and rely on similar
+configuration files.
+
+Despite using the same repository, the frontend
+code does not use the Perl/python backend scripts, and the backend
+side (e.g. `vcs0`) does not use the frontend's php code.
+See [[FrontEndSetup]] for complimentary details about the frontend.
+
+The instructions below were tested on `vcs0`.
+
+`vcs0` needs to run `sv_groups` to create new repositories for new projects
+(or existing projects which enabled new repositories using the "Select 
Features"
+menu option on the savannah web site).
+See [[SavannahInternals]]For more details about the scripts and cronjobs
+(although they describe the setup on the current/old VMs, we aim to replicate
+a similar setup on the new VMs).
+
+
+As root, create the required directories.
+`/etc/savane` is temporarily created with write-access
+(for the 'make install' step as non-root user), and will be made
+read-only after the installation is complete.
+
+    mkdir -p /opt/savannah /etc/savane/content
+    chgrp staff /opt/savannah /etc/savane /etc/savane/content
+    chmod g+ws /opt/savannah /etc/savane /etc/savane/content
+
+As non-root (e.g. agn), clone the savage repository.
+The `nextgen` branch holds the latest updates for the code on the new machines:
+
+    cd /opt/savannah
+    git clone -b nextgen git://git.savannah.gnu.org/administration/savane.git
+    cd savane/
+    ./bootstrap
+    ./configure --prefix=/opt/savannah/usr --sysconfdir=/etc
+    make
+    make install
+
+The resulting files:
+
+    $ ls -l /opt/savannah/usr/
+    total 8
+    drwxrwsr-x 2 agn staff 4096 Oct 19 21:23 bin
+    drwxrwsr-x 3 agn staff 4096 Oct 19 21:09 share
+
+And savannah's common backend scripts:
+
+    $ ls /opt/savannah/usr/bin
+    [...]
+    sv_groups
+    sv_aliases
+    sv_homedirs
+    [...]
+
+Unfortunately the Savane Perl modules are
+still installed in '/usr/local' (the 'prefix' is likely not passed
+to the perl part during configure/make).
+That's not too bad, as it saves the need for custom PERL5LIB setting.
+
+    /usr/local/lib/perl/5.18.2/auto/Savane/.packlist
+    /usr/local/share/perl/5.18.2/Savane.pm
+    /usr/local/share/perl/5.18.2/Savane/Groups.pm
+    /usr/local/share/perl/5.18.2/Savane/Cvs.pm
+    /usr/local/share/perl/5.18.2/Savane/DB.pm
+    /usr/local/share/perl/5.18.2/Savane/Hg.pm
+    /usr/local/share/perl/5.18.2/Savane/User.pm
+    /usr/local/share/perl/5.18.2/Savane/Mail.pm
+    /usr/local/share/perl/5.18.2/Savane/Git.pm
+    [...]
+
+
+Manually prepare configuration files (passwords copied from frontend0).
+The `.pl` file is used by perl scripts (e.g. `sv_groups`).
+The `.ini` file is used by python scripts (e.g. `sv_tmpgpg`).
+
+    cd /etc/savane/
+    chown root:staff savane.*
+    chmod 0640 savane.*
+
+    # remove 'content' - it's only used by the frontend's php
+    rm -r content
+
+    # ls -l /etc/savane/
+    total 16
+    -rw-r----- 1 root staff 8563 Oct 19 21:29 savane.conf.pl
+    -rw-r----- 1 root staff   82 Oct 19 22:36 savane.ini
+    -rw-r----- 1 root staff    9 Oct 19 21:29 savane.pass
+
+    # and lock the directory
+    chmod g-w /etc/savane
+
+
+The script `sv_groups` writes log messages to hard-coded path,
+created it with 'staff' permissions:
+
+    touch /var/log/sv_database2system.log
+    chown root:staff /var/log/sv_database2system.log
+
+FIXME: `sv_groups` also uses a lock file named `groups-users.lock` -
+not sure where it is stored.
+
+Initial testing of 'sv_groups':
+As NON-ROOT user (e.g. agn), run:
+
+    cd /opt/savannah/usr/bin
+    ./sv_groups --only-git --debug
+
+This will show demonstrate how 'sv_groups' scans the groups
+(=projects) in the savane database, and for each one, checks if the
+repositories directories exist.
+
+FIXME: Check with new project or modified repositories type (e.g. git,hg/svn).
+
+FIXME: The log file `/var/log/sv_database2system.log` emits the
+following message, is this relevant / should we create this user on
+vcs0?
+
+    [sv_groups] Wed Oct 19 21:54:40 2016 ---- groupadd -g 77955 anoncvs 
(required by the savane backend)
+

Modified: trunk/sviki/SavannahHosts.mdwn
===================================================================
--- trunk/sviki/SavannahHosts.mdwn      2016-10-20 02:35:17 UTC (rev 264)
+++ trunk/sviki/SavannahHosts.mdwn      2016-10-20 03:02:56 UTC (rev 265)
@@ -624,6 +624,8 @@
 
     cpan -i Mail::Send   # used by perl backend script 'sv_groups'
 
+See [[BackEndSetup]] for details about setting up the savannah's custom
+perl/python backend scripts.
 
 FIXME: Consider the noacl,nocto,rsize=32768,wsize=32768 options.