[Audio-video] The proper way of updating web pages on audio-video.gnu.org

[Luca Saiu]

Hello.  I'm Luca Saiu; I organized this year's GNU Hackers Meeting.

Thanks to Mark Weaver's kind help I was eventually able to upload the
video files from this year's GHM to audio-video.  However we had a
little difficulty understanding the process, at least in the beginning.

I think I understand how the system works now.  If you think it's worth
the trouble (which is to say, if the switch to the new system is not
imminent) I'd volunteer to update the documentation, which doesn't
reflect the current way of working with audio-video.

I'm including here my conversation with Mark, with his permission.
Please tell me if I misunderstood something, and if there's some smarter
way of doing things.

When dealing with how to remove files in my exchange with Mark, the
rsync option I've called "--delete" should read instead
"--remove-source-files"; and now I can indeed confirm that the idea is
correct.

Thanks,

> --- Begin Message ---
>
>
>
> Subject: 
>
> Topics
>
>
>
>
> Topics:
>    Updating audio-video.gnu.org
>    Re: Updating audio-video.gnu.org
>    Re: Updating audio-video.gnu.org
>
> --- End Message ---
> --- Begin Message ---
>
>
>
> Subject: 
>
> Updating audio-video.gnu.org
>
>
>
>
> Date: 
>
> Tue, 03 Sep 2013 00:59:33 +0200
>
>
>
>
> Hello Mark.
>
> I've tried playing a little with audio-video.  I'm slightly scared,
> because the system seems fragile.
>
> As far as I can understand, committing a change on some HTML file in CVS
> does *not* automatically update the corresponding page: you have to use
> rsync to copy it -- presumably, but not necessarily, from an up-to-date CVS
> working directory.
>
> I used rsync to get a list of the files there to have an idea about how
> the server directories were structured:
>
> --8<---------------cut here---------------start------------->8---
> rsync -avz -e ssh --list-only --recursive address@hidden:/srv/audio-video/ | 
> tee FILE-LIST
> --8<---------------cut here---------------end--------------->8---
>
> The thing worked, and I could use rsync to update the main video page
> adding links to the GHM 2013 directory you created yesterday, and to the
> GHM 2011 page as well, which was already fine but not linked.  I also
> committed my (very simple) changes in CVS.
>
> How is one supposed to *delete* files from the remote machine?  I can
> use some rsync kludge with --delete, copying from the remote machine to
> mine, but this seems very error-prone, and quite dangerous.  So I guess
> it's best not to add any temporary file to audio-video.
>
> Trying to upload something to the ghm 2013 directory (I didn't dare to
> add an index.html before I was sure to get it right, in order not to
> break the directory listing) I got a "permission denied" error --
> presumably because the directory belongs to your user and mine doesn't
> have write permission on it:
>
> --8<---------------cut here---------------start------------->8---
> address@hidden ~/repos/audio-video]$ rsync boilerplate.html -avz -e ssh 
> address@hidden:/srv/audio-video/video/ghm2013/
> Enter passphrase for key '/home/luca/.ssh/id_rsa': 
> sending incremental file list
> boilerplate.html
> rsync: mkstemp "/srv/audio-video/video/ghm2013/.boilerplate.html.Wuvdj0" 
> failed: Permission denied (13)
>
> sent 3461 bytes  received 31 bytes  367.58 bytes/sec
> total size is 10997  speedup is 3.15
> rsync error: some files/attrs were not transferred (see previous errors) (code 
> 23) at main.c(1070) [sender=3.0.9]
> address@hidden ~/repos/audio-video]$ 
> --8<---------------cut here---------------end--------------->8---
>
> Am I doing something particularly silly?  Or maybe I should *first* use
> CVS to create or delete a file, and then rsync it?  Apparently no, I
> still get the same error -- see video/ghm2013/please-ignore-this-file .
>
> I don't quite get the logic, and I don't want to pollute the repository
> too much with my tests, when I might be overlooking something obvious.
> The documentation at http://www.gnu.org/server/standards/ seems to imply
> that web updates are automatic, and the Readme file in the repo doesn't
> deal with using rsync for web pages.
>
> Do you have any suggestions?  If not, I can very carefully prepare an
> HTML file for you to upload.  However this thing should really be
> improved or at least better documented, unless the switch to the new
> system is really imminent.
>
> Thanks,
>
> -- 
> Luca Saiu
> Home page:   http://ageinghacker.net
> GNU epsilon: http://www.gnu.org/software/epsilon
> Marionnet:   http://marionnet.org
>
>
> --- End Message ---
> --- Begin Message ---
>
>
>
> Subject: 
>
> Re: Updating audio-video.gnu.org
>
>
>
>
> Date: 
>
> Mon, 02 Sep 2013 23:45:18 -0400
>
>
>
>
> Hi Luca,
>
> > As far as I can understand, committing a change on some HTML file in CVS
> > does *not* automatically update the corresponding page: you have to use
> > rsync to copy it -- [...]
>
> Are you sure?  Did you check that video/index.html wasn't updated
> automatically after some delay?  Tests involving video/ghm2013 are not
> reliable, because they might have been affected by the bad permissions.
> It's also possible that the update process doesn't handle new
> directories automatically.
>
> > The thing worked, and I could use rsync to update the main video page
> > adding links to the GHM 2013 directory you created yesterday, and to the
> > GHM 2011 page as well, which was already fine but not linked.  I also
> > committed my (very simple) changes in CVS.
>
> Looks good to me, thanks!
>
> > How is one supposed to *delete* files from the remote machine?  I can
> > use some rsync kludge with --delete, copying from the remote machine to
> > mine, but this seems very error-prone, and quite dangerous.  [...]
>
> Good question.  Maybe someone on address@hidden knows?
>
> > Trying to upload something to the ghm 2013 directory (I didn't dare to
> > add an index.html before I was sure to get it right, in order not to
> > break the directory listing) I got a "permission denied" error --
> > presumably because the directory belongs to your user and mine doesn't
> > have write permission on it:
>
> Sorry about that.  I think it's fixed now.  Would you like to try again?
>
> > The documentation at http://www.gnu.org/server/standards/ seems to imply
> > that web updates are automatic, and the Readme file in the repo doesn't
> > deal with using rsync for web pages.
>
> Indeed, my expectation was that web updates would be automatic after
> some delay, but maybe this is not implemented on audio-video.org, or
> perhaps 'please-ignore-this-file' wasn't copied successfully because of
> the incorrect permissions in ghm2013.
>
> > Do you have any suggestions?  If not, I can very carefully prepare an
> > HTML file for you to upload.  However this thing should really be
> > improved or at least better documented, unless the switch to the new
> > system is really imminent.
>
> Agreed.  Even just a few more sentences in the "Readme" could have saved
> us both time that would have been better spent hacking Guile or Epsilon.
> We should try to spare future volunteers this trouble.
>
> Thanks for working on this :)
>
>       Mark
>
>
> --- End Message ---
> --- Begin Message ---
>
>
>
> Subject: 
>
> Re: Updating audio-video.gnu.org
>
>
>
>
> Date: 
>
> Tue, 03 Sep 2013 13:01:41 +0200
>
>
>
>
> Hello Mark.
>
> On 2013-09-03 at 05:45, Mark H Weaver wrote:
>
> >> As far as I can understand, committing a change on some HTML file in CVS
> >> does *not* automatically update the corresponding page: you have to use
> >> rsync to copy it -- [...]
> >
> > Are you sure?  Did you check that video/index.html wasn't updated
> > automatically after some delay?  Tests involving video/ghm2013 are not
> > reliable, because they might have been affected by the bad
> > permissions.
>
> I'm quite sure; I waited a long time, and my first changes were on
> video/index.html .
>
> Actually video/ghm2013 wasn't even in the CVS repository before I added
> it; I could only see the directory with rsync.  As far as I understand,
> audio-video is *only* managed via rsync, with the CVS repo being sort of
> a backup, quite dangerous because a user can forget about it and leave
> it out of sync.  I did verify that with my change in video/index.html :
> an HTML change can be performed via rsync before (or without ever)
> replicating it in CVS, and the effect is immediate.
>
> > It's also possible that the update process doesn't handle new
> > directories automatically.
>
> It's possible, but that wouldn't explain the anomaly.
>
> >> The thing worked, and I could use rsync to update the main video page
> >> adding links to the GHM 2013 directory you created yesterday, and to the
> >> GHM 2011 page as well, which was already fine but not linked.  I also
> >> committed my (very simple) changes in CVS.
> >
> > Looks good to me, thanks!
>
> Well, not really.  I just did the obvious minimal change, but the thing
> should be reorganized; many GHM talks don't belong in the category of
> "Video Recordings about our Philosophy" (do we really want a capital P,
> by the way?), and are more technical in nature.  However I won't
> restructure the page without discussing the change first.
>
> >> How is one supposed to *delete* files from the remote machine?  I can
> >> use some rsync kludge with --delete, copying from the remote machine to
> >> mine, but this seems very error-prone, and quite dangerous.  [...]
> >
> > Good question.  Maybe someone on address@hidden knows?
>
> I think this whole exchange of ours could be forwarded to the list.  If
> you agree I will, proposing to update the documentation based on the
> explanations I'll get.
>
> >> Trying to upload something to the ghm 2013 directory [...]  I got a
> >> "permission denied" error -- presumably because the directory belongs
> >> to your user and mine doesn't have write permission on it:
> >
> > Sorry about that.  I think it's fixed now.  Would you like to try again?
>
> Thanks.  I'll just try again using rsync to copy the same file to the
> same place:
>
> --8<---------------cut here---------------start------------->8---
> rsync boilerplate.html -avz -e ssh 
> address@hidden:/srv/audio-video/video/ghm2013/
> --8<---------------cut here---------------end--------------->8---
>
> It worked now, with immediate effect.  Ok, so that was just a trivial
> permission problem.  But notice that I'm ignoring CVS...
>
> >> The documentation at http://www.gnu.org/server/standards/ seems to imply
> >> that web updates are automatic, and the Readme file in the repo doesn't
> >> deal with using rsync for web pages.
> >
> > Indeed, my expectation was that web updates would be automatic after
> > some delay, but maybe this is not implemented on audio-video.org, or
> > perhaps 'please-ignore-this-file' wasn't copied successfully because of
> > the incorrect permissions in ghm2013.
>
> Ok, checking with a second file in the same directory, but I don't think
> it will work.
>
> --8<---------------cut here---------------start------------->8---
> address@hidden ~/repos/audio-video]$ echo Hello world > 
> video/ghm2013/please-ignore-this-as-well
> address@hidden ~/repos/audio-video]$ cvs add 
> video/ghm2013/please-ignore-this-as-well
> Enter passphrase for key '/home/luca/.ssh/id_rsa': 
> cvs add: scheduling file `video/ghm2013/please-ignore-this-as-well' for addition
> cvs add: use `cvs commit' to add this file permanently
> address@hidden ~/repos/audio-video]$ cvs commit -m 'Added another test file 
> (sorry for the noise)'
> cvs commit: Examining .
> cvs commit: Examining audio
> cvs commit: Examining docs
> cvs commit: Examining ghm2009
> cvs commit: Examining ghm2010
> cvs commit: Examining graphics
> cvs commit: Examining video
> cvs commit: Examining video/ghm2009
> cvs commit: Examining video/ghm2010
> cvs commit: Examining video/ghm2013
> Enter passphrase for key '/home/luca/.ssh/id_rsa': 
> ? GET-FILE-LIST
> ? FILE-LIST
> /web/audio-video/audio-video/video/ghm2013/please-ignore-this-as-well,v  <--  
> video/ghm2013/please-ignore-this-as-well
> initial revision: 1.1
> Triggering webpages update...
> --8<---------------cut here---------------end--------------->8---
>
> I will not use rsync for please-ignore-this-as-well; if it gets
> automatically created after some long delay, it will be appear in
> http://audio-video.gnu.org/video/ghm2013/
>
> >> However this thing should really be improved or at least better
> >> documented, unless the switch to the new system is really imminent.
> >
> > Agreed.  Even just a few more sentences in the "Readme" could have saved
> > us both time that would have been better spent hacking Guile or Epsilon.
> > We should try to spare future volunteers this trouble.
> >
> > Thanks for working on this :)
>
> Ok, I will.  It shouldn't be a very big deal: after confirming that the
> thing works the way I think I'll just have to update the Readme file,
> contribute an updated description for the Savannah project page, and
> prepare a small patch for the standards document.
>
> May I forward our conversation to the list?
>
> Thanks,
>
> -- 
> Luca Saiu
> Home page:   http://ageinghacker.net
> GNU epsilon: http://www.gnu.org/software/epsilon
> Marionnet:   http://marionnet.org
>
>
> --- End Message ---
-- 
Luca Saiu
Home page:   http://ageinghacker.net
GNU epsilon: http://www.gnu.org/software/epsilon
Marionnet:   http://marionnet.org