[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Savannah-cvs] [615] update: mailing lists are configured from frontend
From: |
ineiev |
Subject: |
[Savannah-cvs] [615] update: mailing lists are configured from frontend directly |
Date: |
Mon, 15 May 2023 13:15:48 -0400 (EDT) |
Revision: 615
http://svn.savannah.gnu.org/viewvc/?view=rev&root=administration&revision=615
Author: ineiev
Date: 2023-05-15 13:15:46 -0400 (Mon, 15 May 2023)
Log Message:
-----------
update: mailing lists are configured from frontend directly
Modified Paths:
--------------
trunk/sviki/MailingListsInternals.mdwn
trunk/sviki/SavannahUpdateDelays.mdwn
Modified: trunk/sviki/MailingListsInternals.mdwn
===================================================================
--- trunk/sviki/MailingListsInternals.mdwn 2023-05-13 07:11:49 UTC (rev
614)
+++ trunk/sviki/MailingListsInternals.mdwn 2023-05-15 17:15:46 UTC (rev
615)
@@ -1,5 +1,4 @@
-Mailing Lists Management in Savannah
-====================================
+# Mailing Lists Management in Savannah
All gnu and nongnu mailing lists are processed on `lists.gnu.org`
using the Mailman package. This machine is managed by FSF admins.
@@ -14,9 +13,8 @@
* [[ListServer]] - manually running mailing-list related commands.
+## Overview
-Overview
---------
Savannah provides mailing lists to hosted groups.
Group admins can create as many mailing lists as needed through
@@ -26,22 +24,15 @@
1. A group admin on Savannah creates a new mailing list on Savannah
website.
-2. The PHP frontend code updates the mailing list information in the
- `mail_group_list` table in the `savane` MySQL database.
-3. A cronjob on 'mgt0` reads the `mail_group_list` table and determines
- which operations are needed (e.g. list creation, password reset).
-4. The cronjob runs local scripts (on mgt0) for each operation.
-5. Each script proxies the commands over SSH to `lists.gnu.org`
-6. On `lists.gnu.org`, the script `scannah_wrapper.pl` processes the commands
- and runs the appropriate Mailman scripts locally on `lists.gnu.org`.
-7. The cronjob script (back on `mgt0`) sends email notification to the
- group admins (e.g. notification about updated passwords).
+2. The PHP frontend code runs `savannah_wrapper.pl` on list@lists.gnu.org
+ via ssh; `savannah_wrapper.pl` runs appropriate Mailman scripts.
+3. The PHP frontend code updates the mailing list information in the
+ `mail_group_list` table in the `savane` MySQL database and sends
+ a notification with new mailing list password when needed.
+## Database
-Database
---------
-
The `mail_group_list` table contains the following fields:
mysql> select * from mail_group_list where list_name = 'sed-devel' \G
@@ -70,254 +61,80 @@
The meaning of the `status` field:
- Status 0: list is deleted (ie, does not exist).
- Status 1: list is marked for creation.
- Status 2: list is marked for reconfiguration.
- Status 5: list has been created (ie, it exists).
+ Status 0: obsolete.
+ Status 1: obsolete.
+ Status 2: obsolete.
+ Status 5: the only one used, so actually means nothing.
-The `password` field means:
+The `password` field means: nothing.
- NULL: default value
- '1': A password reset was requested for this mailing list.
+## savannah_wrapper.pl
-The password field should not contain any real passwords,
-though the database does contain values besides NULL and '1'
-(TODO: find out how/why):
+This script processes requests from frontend and translates them
+in Mailman script invocations. The requests are read from STDIN
+as a set of lines in form
- mysql> select count(*) from mail_group_list where password is not NULL and
password != '1';
- +----------+
- | count(*) |
- +----------+
- | 1674 |
- +----------+
- 1 row in set (0.00 sec)
+ VAR=VALUE
+The response is written to STDOUT; normal output looks like
+a similar set of
+ VAR=VALUE
-PHP web frontend code
----------------------
+It includes the version of the script and the variables
+set in the process. Errors, if any, are reported in lines
+starting with 'Error', typically without the '=' sign.
-Mailing list administration is implemented in
-[./frontend/php/mail/admin/index.php](https://git.savannah.gnu.org/cgit/administration/savane.git/tree/frontend/php/mail/admin/index.php).
+The COMMAND variable is required, it may have values:
-The PHP code updates *only* the `mail_group_list` table in the MySQL
-database, and queues future actions by changing the `status` and
-`password` fields in the table.
+ version: Report version and generated password and quit.
+ newlist: Create a list. The parameters used are
+ VISIBILITY, LIST_FULL_NAME, DESCRIPTION, ADMIN_MAIL, PASSWORD
+ (all are required).
+ config: Make list public or private, change its description.
+ Used parameters are VISIBILITY and DESCRIPTION (both optional).
+ change_pw: Request password reset. The parameters used are
+ LIST_NAME and PASSWORD (both required).
+ rmlist: Remove the list. The parameter used is LIST_NAME (required).
-No other actions are directly performed by the PHP frontend code.
+The parameters used by commands are
-The PHP code contains these additional comments:
+ DRY_RUN: Don't apply changes; the only valid value is 'yes'.
+ ADMIN_MAIL: Email of the administrator.
+ LIST_NAME: List name, e.g. pretest-users.
+ LIST_FULL_NAME: List name with domain, e.g. pretest-users@lists.nongnu.org.
+ DESCRIPTION: A line describing the list
+ PASSWORD: New password; when empty (which is the normal usage),
+ the script generates a new password.
+ VISIBILITY: Whether the list is 'public' or 'private'.
- This frontend PHP script sets status to:
- 0 if user deletes a list before the backend ever actually created it
- 1 if user adds a list
- 2 if user reconfigures an _existing_ list (ie, status was 5)
+## Manual walk-through example - changing description
- The backend sv_mailman.pl script sets status to:
- 0 when a list is actually deleted
- 5 when a list is actually created
-
- When we create an alias, which mean someone was able, according to
- group type restriction, to add to the group a list that was already
- inside the database, we add the list inside the database with a status
- of 5, so sv_mailman does not try to recreate it.
-
- In the worst case, if two persons create the same list at the same time,
- the field password will not contain real password, it will contain
- '1' when the backend is supposed to reset it.
-
-
-Backend cronjob
----------------
-
-The mailing list cronjob runs on `mgt0` (in the old setup
-before the 2017 migration to newer VM, the script was on the `oldinternal` -
-outdated wiki pages might still refer to it there).
-
-In `mgt0:/etc/cron.d/savane`:
-
- PATH=...:/opt/savannah/bin:...
- # New list creation
- */5 * * * * root sv_mailman --cron
-
-The script is `mgt0:/opt/savannah/bin/sv_mailman`.
-
-The above script is the installed version, originating from
-`mgt0:/opt/savannah/savane/backend/mail/sv_mailman.in` (and also in
-git
-<https://git.savannah.gnu.org/cgit/administration/savane.git/tree/backend/mail/sv_mailman.in>).
-If you want to modify the script, NEVER update the installed version
-in `/opt/savannah/bin`. ALAWAYS push the updated version to
-the Savannah repository, update the source version in
-`/opt/savannah/savane/backend/mail/sv_mailman.in`, then run `make
-install`.
-
-The script reads the `mail_group_list` MySQL table
-and performs the required actions based on the `status` and `password`
-fields.
-
-The script runs the following local scripts (on `mgt0`):
-
-- `/usr/sbin/newlist` - creating new mailing list
-- `/usr/sbin/config_list` - configuring a list (currently a no-op)
-- `/usr/sbin/rmlist` - removing a list
-- `/usr/lib/mailman/bin/change_pw` - resetting a mailing list password.
-
-Sadly, the paths are hard-coded in `sv_mailman` (there's even a comment
-about how bad they are).
-
-On `mgt0`, symlinks were created to the actual scripts:
-
- agn@mgt0:~$ ls -lhog /usr/sbin/{newlist,config_list,rmlist}
/usr/lib/mailman/bin/change_pw
- lrwxrwxrwx 1 28 Apr 17 22:44 /usr/lib/mailman/bin/change_pw ->
/opt/savannah/sbin/change_pw
- lrwxrwxrwx 1 30 Apr 17 22:43 /usr/sbin/config_list ->
/opt/savannah/sbin/config_list
- lrwxrwxrwx 1 26 Apr 17 22:43 /usr/sbin/newlist ->
/opt/savannah/sbin/newlist
- lrwxrwxrwx 1 25 Apr 17 22:43 /usr/sbin/rmlist -> /opt/savannah/sbin/rmlist
-
-FIXME: remove the hard-coded PATHs from `sv_mailman`, and install an
-updated version.
-
-NOTE: These scripts are proxies to `lists.gnu.org` - they do not
-perform any local action on `mgt0`. Their names are likely left over
-from the time the entire Savannah system (including Mailman)
-was running on the same local machine.
-
-SSH Proxing to lists.gnu.org
-----------------------------
-
-The `newlist`/`rmlist`/`change_pw` scripts on `mgt0`
-proxy the needed commands to `lists.gnu.org`,
-where the Mailman package is installed.
-
-1. On `mgt0`, the script `sv_mailman` runs as root from cron-jobs.
-2. The `sv_mailman` runs one of the action scripts (e.g. `newlist`).
- the script also generates random passwords if needed.
-3. The `newlist` script (on `mgt0`) connets with SSH to `list@lists.gnu.org`
- (this happens as user `root@mgt0`).
-4. The corresponding pubkey in `lists.gnu.org:/home/list/.ssh/authorized_keys`
- is defined as follows:
-
- command="./savannah_wrapper.pl" ssh-rsa AAAAB3NzaC1yc2...
-
-5. When the `newlist` script from `mgt0` connects with SSH to `lists.gnu.org`
- instead of getting a shell, the
`lists.gnu.org:/home/list/savannah_wrapper.pl`
- script is executed.
-6. This script parses limited input from STDIN, and if it contains
- recognized commands and parameters, it runs the corresponding program
- locally on `lists.gnu.org`.
-7. `sv_mainman` sends email notification to the administrators of the mailing
- lists, with the newly generated random passwords.
- (FIXME: emails are sent even if commands fail - there's no error
- checking).
-
-Example: running the following on `mgt0` as user `root` will result in
-changing the mailing-list password on `lists.gnu.org`:
-
- # ssh -T list@lists.gnu.org <<EOF
- COMMAND=change_pw
- LIST_NAME=bug-datamash
- PASSWORD=123456
- EOF
-
-The corresponding code in `lists.gnu.org:/home/list/savannah_wrapper.pl`
-(lots of code omitted for brevity):
-
- ...
- my %vars;
- while (<>) {
- chomp;
- my ($variable, $value) = split('=');
- $vars{lc($variable)} = $value;
- }
- ...
- if ($vars{'command'} eq 'change_pw') {
- (system('change_pw',
- '-l', $vars{'list_name'},
- '-p', $vars{'password'},
- '--quiet')
- == 0) or die "change_pw: $!";
- }
-
-TODO: document the "virtual host" hack (e.g. allowing gnu/nongnu mailing
lists).
-
-Logs
-----
-
-On `mgt0`, the log file is `mgt0:/var/log/sv_database2system.log`:
-
- [sv_mailman] Sun Apr 9 01:50:01 2017 - starting
- [sv_mailman] Sun Apr 9 01:50:02 2017 - List www-zh-cn-translators
<Fossilet> config_list.
- [sv_mailman] Sun Apr 9 01:50:02 2017 - List www-zh-cn-translators
<Fossilet> reconfigured.
- [sv_mailman] Sun Apr 9 01:50:02 2017 - List www-zh-cn-translators
password was reset.
- [sv_mailman] Sun Apr 9 01:50:02 2017 - Mail sent to foo@gnu.org,
bar@gnu.org, baz@gnu.org.
- [sv_mailman] Sun Apr 9 01:50:02 2017 - work finished
-
-On `lists.gnu.org`, the log file is
`lists.gnu.org:/home/list/savannah_wrapper.log`.
-It contains all the passed parameters (and thus passwords in clear text):
-
- Tue Apr 18 02:52:06 2017
- password = 12345
- list_name = pretest-users
- command = change_pw
-
-Manual walk-through example - changing password
------------------------------------------------
-
-When group admins request password reset for their mailing list
-on Savannah website, the PHP frontend code sets the corresponding
-`password` field in the `mail_group_list` table to `1`.
-Equivalent command:
-
- $ mysql savane
- mysql> UPDATE mail_group_list SET password='1' where
list_name='pretest-users' LIMIT 1;
-
-On `mgt0` the `sv_mailman` is run periodically from cron.
-Equivalent command:
-
- $ ssh root@mgt0
- # cd /opt/savannah/bin
- # ./sv_mailman --cron
- New pretest-users password: foobar12345
-
-The log file `mgt0:/var/log/sv_database2system.log` will show:
-
- [sv_mailman] Mon 17 Apr 2017 10:52:05 PM EDT - starting
- [sv_mailman] Mon 17 Apr 2017 10:52:06 PM EDT - List pretest-users password
was reset.
- [sv_mailman] Mon 17 Apr 2017 10:52:06 PM EDT - Mail sent to
assafgordon@gmail.com for list pretest-users.
- [sv_mailman] Mon 17 Apr 2017 10:52:06 PM EDT - work finished
-
-The `sv_mailman` generated a new random password and executed `change_pw`.
-Equivalent command (still on `mgt0`):
-
- $ ssh root@mgt0
- # cd /opt/savannah/sbin
- # ./change_pw -n pretest-users -p 123456
- New pretest-users password: 123456
-
-The `change_pw` script connects to `lists.gnu.org`
+The frontend code connects to `lists.gnu.org`
and transmits the commands as STDIN.
-Equivalent command (still on `mgt0`):
+Equivalent command (still on `frontend`):
- # ssh -T list@lists.gnu.org <<EOF
- COMMAND=change_pw
- LIST_NAME=pretest-users
- PASSWORD=123456
+ # sudo -u www-data ssh -T list@lists.gnu.org <<EOF
+ COMMAND=config
+ LIST_FULL_NAME=pretest-users@lists.nongnu.org
+ DESCRIPTION=Test mailing list
EOF
+ version=list@lists.gnu.org 0.1
+ command=config
+ list_full_name=pretest-users@lists.nongnu.org
+ description=Test mailing list
-
The script `savannah_wrapper.pl` on `lists.gnu.org`
-reads the commands from STDIN and execute the needed
+reads the commands from STDIN and executes the needed
programs.
+
Equivalent command (on `lists.gnu.org`):
$ ssh list@list.gnu.org
- lists:~$ which change_pw
- /home/list/mailman/bin/change_pw
- lists:~$ change_pw -l pretest-users -p 123456 --quiet
- New pretest-users password: 123456
+ lists:~$ echo description='Test mailing list' \
+ | config_list -i /dev/stdin pretest-users
-Mailing list creation - FOR REFERENCE ONLY
-------------------------------------------
+## Mailing list creation - FOR REFERENCE ONLY
**The Following section describes the original setup. It is kept for
historical purposes**
Modified: trunk/sviki/SavannahUpdateDelays.mdwn
===================================================================
--- trunk/sviki/SavannahUpdateDelays.mdwn 2023-05-13 07:11:49 UTC (rev
614)
+++ trunk/sviki/SavannahUpdateDelays.mdwn 2023-05-15 17:15:46 UTC (rev
615)
@@ -19,8 +19,8 @@
email addresses, notification settings, GnuPG and SSH keys
are in effect with no delay.
- Group home pages update within a few minutes.
-- Mailing list-related requests (creation, deletion, admin password
- reset) are passed to lists.gnu.org every 5 minutes.
+- Mailing list-related requests are processed on lists.gnu.org
+ immediately.
Copyright © 2023 Ineiev
[Prev in Thread] |
Current Thread |
[Next in Thread] |
- [Savannah-cvs] [615] update: mailing lists are configured from frontend directly,
ineiev <=