Re: [Monotone-devel] ikiwiki monotone support

[Brian May]

>>>>> "William" == William Uther <address@hidden> writes:

    >> If I type in
    >> 
    >> cd $path mtn sync
    >> 
    >> Will it call the hooks in $path/_MTN/montonerc?

    William> I believe so, yes.

Hmmm. Maybe I should check.

I can't help but be really sceptical, simply because "mtn sync", "mtn
pull", and "mtn merge" don't touch the workspace, and I would be
really surprised if they look at the _MTN directory, except to get the
default database.

In fact, this would suggest to me I might be correct:

address@hidden:~/tree/config$ strace -eopen mtn sync
open("/etc/ld.so.cache", O_RDONLY)      = 3
open("/usr/lib/libz.so.1", O_RDONLY)    = 3
open("/usr/lib/libstdc++.so.6", O_RDONLY) = 3
open("/lib/tls/libm.so.6", O_RDONLY)    = 3
open("/lib/libgcc_s.so.1", O_RDONLY)    = 3
open("/lib/tls/libc.so.6", O_RDONLY)    = 3
open("/usr/lib/locale/locale-archive", O_RDONLY|O_LARGEFILE) = 3
open("/usr/share/locale/locale.alias", O_RDONLY) = 3
open("/usr/share/locale/en_AU.UTF-8/LC_MESSAGES/monotone.mo", O_RDONLY) = -1 
ENOENT (No such file or directory)
open("/usr/share/locale/en_AU.utf8/LC_MESSAGES/monotone.mo", O_RDONLY) = -1 
ENOENT (No such file or directory)
open("/usr/share/locale/en_AU/LC_MESSAGES/monotone.mo", O_RDONLY) = -1 ENOENT 
(No such file or directory)
open("/usr/share/locale/en.UTF-8/LC_MESSAGES/monotone.mo", O_RDONLY) = -1 
ENOENT (No such file or directory)
open("/usr/share/locale/en.utf8/LC_MESSAGES/monotone.mo", O_RDONLY) = -1 ENOENT 
(No such file or directory)
open("/usr/share/locale/en/LC_MESSAGES/monotone.mo", O_RDONLY) = -1 ENOENT (No 
such file or directory)
open("/usr/lib/gconv/gconv-modules.cache", O_RDONLY) = 3
open("/dev/urandom", O_RDONLY|O_LARGEFILE) = 3
open("/home/bam/.monotone/monotonerc", O_RDONLY) = 4
open("_MTN/format", O_RDONLY|O_LARGEFILE) = 4
open("_MTN/options", O_RDONLY|O_LARGEFILE) = 4
open("/home/bam/monotone/config.mtn", O_RDWR|O_CREAT|O_LARGEFILE, 0644) = 4
open("/dev/urandom", O_RDONLY|O_LARGEFILE) = 5
open("/var/tmp/etilqs_DXVWqVZACfCKxQh", O_RDWR|O_CREAT|O_EXCL|O_LARGEFILE, 
0600) = 5
open("/home/bam/.monotone/keys", O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY) = 
5
open("/home/bam/.monotone/keys/address@hidden", O_RDONLY|O_LARGEFILE) = 5
open("/home/bam/monotone/config.mtn-journal", 
O_RDWR|O_CREAT|O_EXCL|O_LARGEFILE, 0644) = 5
open("/home/bam/monotone", O_RDONLY|O_LARGEFILE) = 6
mtn: connecting to /tmp/out.mtn
open("/usr/share/locale/en_AU.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT 
(No such file or directory)
open("/usr/share/locale/en_AU.utf8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT 
(No such file or directory)
open("/usr/share/locale/en_AU/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No 
such file or directory)
open("/usr/share/locale/en.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT 
(No such file or directory)
open("/usr/share/locale/en.utf8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No 
such file or directory)
open("/usr/share/locale/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such 
file or directory)
mtn: network error: failed to connect: No such file or directory
Process 17398 detached

although admittedly the test would be better if the other database
existed and contained new revisions.

    William> My original thought was that the
    William> note_netsync_revision_received hook would go in the
    William> monotonerc for the server mtn process.  That process has
    William> nothing to do with the workspace that contains
    William> $path/_MTN/montonerc.

Ok, so I probably should delete the mergerc stuff totally (as per my
other post), but keep the ikiwiki wrapper.

I think configuring the note_netsync_revision_received hook of the
server is outside the scope of ikiwiki, although documentation could
be improved to say this can be done, e.g. by supplying a default hook
that a user can add to /etc/monotone/hooks.lua

At the moment I have:

=== cut ===
netsync_branches = {}
function note_netsync_start(nonce)
   netsync_branches[nonce] = {}
end

function _note_netsync_cert_received(rev_id,key,name,value,nonce)
   if name == "branch" then
      if netsync_branches[nonce][value] == nil then
         netsync_branches[nonce][value] = 1
      else
         netsync_branches[nonce][value] = netsync_branches[nonce][value] + 1
      end
   end
end

function note_netsync_revision_received(new_id,revision,certs,nonce)
   for _, item in pairs(certs)
   do
      _note_netsync_cert_received(new_id,item.key,item.name,item.value,nonce)
   end
end

function note_netsync_cert_received(rev_id,key,name,value,nonce)
   _note_netsync_cert_received(rev_id,key,name,value,nonce)
end

function note_netsync_end(nonce)
   for item, amount in pairs(netsync_branches[nonce])
   do
        io.stderr:write("updating workspace for ",item,"...\n")
        rc=execute("daemon",
                "--pidfile=/var/log/monotone/" .. item ..".pid",
                "--output=/var/log/monotone/" .. item ..".log",
                "--",
                "sudo","-u","content",
                "/usr/local/bin/update",
                item)
        io.stderr:write("...done (",rc,")\n")
   end
   netsync_branches[nonce] = nil
end
=== cut ===

It should be easy to change to use the ikiwiki wrapper instead of my
shell-script executed via a restricted sudo command. You would need to
check the branch received though.

Only I am reluctant to change my version, because it works on a
website I have that uses static html files too. My version also would
appear to easier to understand too, although the update script (below)
is very specific to my computer.

I use daemon to ensure only one update process runs at a time, BTW. It
also means that the client can disconnect before the update occurs, so
no locking issues occur.

I also use sudo because all my web files are owned by "content" (and
not monotone or www-data - I felt this increases security).

the sudoers line:

=== cut ===
monotone ALL=(content) NOPASSWD:/usr/local/bin/update
=== cut ===

    >> Also, if this code is used, you probably only want to call the
    >> ikiwiki wrapper script on the last revision received, not for
    >> every revision received.

    William> Good point :)

For completeness, here is my current /usr/local/bin/update script from
above (note: I haven't tested the merge stuff yet). I would like to check
that it is never run as root (as this is almost always a big mistake), but 


=== cut ===
#!/bin/sh -ex

if [ "$UID" != 110 ]
then
        exec sudo -u content "$0" "$@"
        exit 1
fi

if [ -z "$1" ]
then
        echo "Incorrect parameters: " >&2
        echo "update <branch>" >&2
        exit 1
fi

export HOME="/home/content"

BRANCH="$1"

SOURCE="louie.microcomaustralia.com.au:4691"

if [ "$BRANCH" == "au.com.microcomaustralia.brian.microcom_website" ]
then
        WORKDIR="/home/microcomaustralia.com.au/www"
elif [ "$BRANCH" == "au.com.microcomaustralia.wiki" ]
then
        WORKDIR="/home/microcomaustralia.com.au/wiki.src"
else
        echo "Unknown brach $BRANCH - not updating working directory" >&2
fi

ALLDB="/home/content/monotone/default.mtn"
if [ -n "$ALLDB" ]
then
        mtn --db="$ALLDB" set database default-server "$SOURCE"
        mtn --db="$ALLDB" -k address@hidden pull "$SOURCE" "$BRANCH"
fi


if [ -n "$WORKDIR" ]
then
        PULLDB="/home/content/monotone/$BRANCH.mtn"
        mtn --db="$PULLDB" set database default-include-pattern "$BRANCH"
        mtn --db="$PULLDB" set database default-server "$SOURCE"
        mtn --db="$PULLDB" -k address@hidden pull "$SOURCE" "$BRANCH"

        cd "$WORKDIR"

        heads=`mtn --db="$PULLDB" automate heads "$BRANCH" | wc --lines`
        while [ "$heads" -gt 1 ]
        do
                export MTN_MERGE=fail
                if ! mtn merge
                then
                        break
                fi

                heads=`mtn --db="$PULLDB" automate heads "$BRANCH" | wc --lines`
        done

        diff=`mtn diff | wc --lines`
        if [ "$diff" -gt 3 ]
        then
                echo "Local changes made" >&2
                exit 0
        fi

        if [ "$BRANCH" == "au.com.microcomaustralia.wiki" ]
        then
                mtn update --db="$PULLDB" --branch="$BRANCH"
                ikiwiki --setup /home/microcomaustralia.com.au/wiki.setup
        else
                mtn --db="$PULLDB" update --branch="$BRANCH"
        fi
fi
=== cut ===

$ALLDB is for viewmtn, back when it only supported one database. I
guess I could change ikiwiki just to use one database or use multi-db
support with viewmtn, but I haven't made any decision.

I decided against using the diffutils non-interactive merge, because I
feel it is better to have ikiwiki use an older version then render a
page with conflict markers. It is a bit different when editing a page
inside ikiwiki, and you know immediately if conflicts occur.

Also I put it in a loop, on the pretence that if there are more then 2
heads, it will merge them all. Will this work? Or have I risked a
infinite loop somehow?

    William> It is called after an ikiwiki commit... and nowhere else.
    William> Which is weird.  I don't know how my setup could have
    William> worked.  Maybe something has changed there in Ikiwiki?

Did you check the wrapper generation stuff? I thought it called it
too, but I may have been confused.

Unfortunately my repository is on my other computer right now.

I still find it weird that a Perl program will generate and compile C
code....
-- 
Brian May <address@hidden>