Re: [bug-gsrc] /srv/bzr/gsrc/trunk r764: revert to make-3.81 due to backwards incompatibility of 3.82

[Brian Gough]

Thanks for the explanation. Do you have an account on savannah? I can
add you to the project so you can commit directly.

At Mon, 17 Oct 2011 19:36:56 -0400,
Luke Shumaker wrote:
> 
> [1  <text/plain; US-ASCII (7bit)>]
> At Mon, 17 Oct 2011 14:30:21 +0100,
> Brian Gough wrote:
> > At Sun, 16 Oct 2011 23:49:27 -0400,
> > Luke Shumaker wrote:
> > > The attatched patch should make GSRC work just fine with make 3.82.
> > > 
> > > Now, instead of using
> > >  $ make http//example.com/foo.tar.gz
> > > it's
> > >  $ make download/http/example.com/foo.tar.gz
> > > 
> > > While it's a little more... verbose, I like it because it seems
> > > slightly less "magic"; protocols are foreign to make, directories aren't.
> > 
> > Hi Luke. Can you explain how this works?  I thought the problem was
> > that if a filename like 'patch-3.18.tar.gz' matched a target like
> > 'patch-%' it triggered the wrong rule, which seemed impossible to work
> > around.  I've tried "make -C gnu/patch" and I get an infinite loop
> > using make-3.81.
> > 
> > address@hidden:/home/gnu/gsrc$ make -C gnu/patch
> > make: Entering directory `/home/gnu/gsrc/gnu/patch'
> >  ==> Grabbing download/patch-2.6.1.tar.gz
> >      ==> Trying download/file/files/patch-2.6.1.tar.gz
> > <snip>
> 
> Make 3.82 will select the rule with the shortest stem (the %); in
> other words the rule that is most specific, or the longest. (Directory
> snipping makes latter definition easier to explain).
> 
> "patch-%" matches 6 non-stem characters, but "http/%" only matches 5,
> so make selects the former. To work around this, all we have to do is
> make the latter longer; for example by prefixing it with "download/".
> "download/http/%" will match 14 non-stem characters, so it will be
> selected.
> 
> This method is great, because it makes the downloads look more like
> ordinary files, which make deals with. Ideally, we'd integrate it a
> little better with the garchive system used by GAR, but that patch is
> a little less trivial.
> 
> Now, for the prefix to tack onto the urls, I just chose
> $(DOWNLOADDIR), which creates a collision between the URLS and
> "$(DOWNLOADDIR)/%", which selects from a number of URLS to fetch the
> file.
> 
> This is fine in make 3.82; "$(DOWNLOADDIR)/http/%" is longer than
> "$(DOWNLOADDIR)/%", so make did the right thing.
> However, I forgot that make 3.81 would just select the first pattern,
> "$(DOWNLOADDIR)/%", which throws it into an infinate loop (I'm
> actually surprised that make doesn't detect the loop, but my approach
> to learning make's safety mechanisms is that if you need them at all,
> you're doing it wrong).
> 
> Anyway, here's a another patch (instead of, not on top of the old one)
> that uses "cache/" instead of "download/".
> 
> ~ Luke Shumaker
> 
> [2 make382-fix-take2.patch <text/plain; US-ASCII (base64)>]
> === modified file 'gar.lib.mk'
> --- gar.lib.mk        2011-08-08 16:47:44 +0000
> +++ gar.lib.mk        2011-10-17 19:50:27 +0000
> @@ -52,11 +52,12 @@
>  SIGFILE_URLS = $(foreach DIR,$(FILE_SITES) $(SIGFILE_DIRS) 
> $(MASTER_DIRS),$(addprefix $(DIR),$(SIGFILES)))
>  PATCHFILE_URLS = $(foreach DIR,$(FILE_SITES) $(PATCHFILE_DIRS) 
> $(MASTER_DIRS),$(addprefix $(DIR),$(PATCHFILES)))
>  
> +ALL_URLS = $(DISTFILE_URLS) $(SIGFILE_URLS) $(PATCHFILE_URLS)
>  # FIXME: doesn't handle colons that are actually in the URL.
>  # Need to do some URI-encoding before we change the http:// to
>  # http// etc.
> -URLS = $(subst ://,//,$(DISTFILE_URLS) $(SIGFILE_URLS) $(PATCHFILE_URLS))
> -
> +URL_FILES = $(subst ://,/,$(ALL_URLS)) # This is where any encoding happens
> +URLS = $(addprefix $(DOWNLOADDIR)/,$(URL_FILES))
>  
>  # Download the file if and only if it doesn't have a preexisting
>  # checksum file.  Loop through available URLs and stop when you
> @@ -84,23 +85,23 @@
>  WGET_OPTS = -c --no-check-certificate --passive-ftp -U "GSRC/1.0"
>  
>  # download an http URL (colons omitted)
> -http//%: 
> +$(CACHEDIR)/http/%:
>       wget $(WGET_OPTS) -O $(DOWNLOADDIR)/$(notdir $*).partial http://$*
>       mv $(DOWNLOADDIR)/$(notdir $*).partial $(DOWNLOADDIR)/$(notdir $*)
>  
>  # download an https URL (colons omitted)
> -https//%: 
> +$(CACHEDIR)/https/%:
>       wget $(WGET_OPTS) -O $(DOWNLOADDIR)/$(notdir $*).partial https://$*
>       mv $(DOWNLOADDIR)/$(notdir $*).partial $(DOWNLOADDIR)/$(notdir $*)
>  
>  # download an ftp URL (colons omitted)
> -ftp//%: 
> +$(CACHEDIR)/ftp/%:
>       wget $(WGET_OPTS) -O $(DOWNLOADDIR)/$(notdir $*).partial ftp://$*
>       mv $(DOWNLOADDIR)/$(notdir $*).partial $(DOWNLOADDIR)/$(notdir $*)
>  
>  # link to a local copy of the file
>  # (absolute path)
> -file///%: 
> +$(CACHEDIR)/file//%: 
>       @if test -f /$*; then \
>               ln -sf "/$*" $(DOWNLOADDIR)/$(notdir $*); \
>       else \
> @@ -109,7 +110,7 @@
>  
>  # link to a local copy of the file
>  # (relative path)
> -file//%: 
> +$(CACHEDIR)/file/%: 
>       @if test -f $*; then \
>               ln -sf $(CURDIR)/$* $(DOWNLOADDIR)/$(notdir $*); \
>       else \
> @@ -118,26 +119,26 @@
>  
>  # Using Jeff Waugh's rsync rule.
>  # DOES NOT PRESERVE SYMLINKS!
> -rsync//%: 
> +$(CACHEDIR)/rsync/%: 
>       rsync -azvL --progress rsync://$* $(DOWNLOADDIR)/
>  
>  # Download a directory tree as a tarball.
>  RSYNC_OPTS ?= -az
>  RSYNC_PATH ?=
> -rsynctree//%:
> +$(CACHEDIR)/rsynctree/%:
>       mkdir -p $(DOWNLOADDIR)/rsync
>       rsync -v --progress $(RSYNC_OPTS) $(RSYNC_PATH) $(DOWNLOADDIR)/rsync
>       cd $(DOWNLOADDIR)/rsync && tar -czvf ../out *
>       mv $(DOWNLOADDIR)/out $(DOWNLOADDIR)/$*
>  
>  # Using Jeff Waugh's scp rule
> -scp//%:
> +$(CACHEDIR)/scp/%:
>       scp -C $* $(DOWNLOADDIR)/
>  
>  # Check out source from CVS.
>  CVS_CO_OPTS ?= -D$(GARVERSION) -P
>  CVS_MODULE ?= $(GARNAME)
> -cvs//%:
> +$(CACHEDIR)/cvs/%:
>       mkdir -p $(DOWNLOADDIR)/cvs
>       cd $(DOWNLOADDIR)/cvs && \
>               cvs -d$(CVS_ROOT) login && \
> @@ -147,7 +148,7 @@
>  # Check out source from Subversion.
>  SVN_REVISION ?= "{$(GARVERSION)}"
>  SVN_CO_OPTS ?= -r $(SVN_REVISION)
> -svnco//%:
> +$(CACHEDIR)/svnco/%:
>       mkdir -p $(DOWNLOADDIR)/svn
>       cd $(DOWNLOADDIR)/svn && \
>               svn co $(SVN_CO_OPTS) $(SVN_PATH) $(DISTNAME) && \
> @@ -155,7 +156,7 @@
>  
>  # Check out source from Darcs.
>  DARCS_GET_OPTS ?= --partial --to-match "date $(GARVERSION)"
> -darcs//%:
> +$(CACHEDIR)/darcs/%:
>       mkdir -p $(DOWNLOADDIR)/darcs
>       cd $(DOWNLOADDIR)/darcs && \
>               darcs get $(DARCS_GET_OPTS) $(DARCS_PATH) $(DISTNAME) && \
> @@ -163,7 +164,7 @@
>  
>  # Check out source from Git.
>  GIT_REVISION ?= v$(GARVERSION)
> -git//%:
> +$(CACHEDIR)/git/%:
>       mkdir -p $(DOWNLOADDIR)/git
>       cd $(DOWNLOADDIR)/git && \
>               git clone $(GIT_PATH) $(DISTNAME) && \
> @@ -173,7 +174,7 @@
>  # Check out source from Mercurial.
>  HG_REVISION ?= $(GARVERSION)
>  HG_CLONE_OPTS ?= -r "$(HG_REVISION)"
> -hg//%:
> +$(CACHEDIR)/hg/%:
>       mkdir -p $(DOWNLOADDIR)/hg
>       cd $(DOWNLOADDIR)/hg && \
>               hg clone $(HG_CLONE_OPTS) $(HG_PATH) $(DISTNAME) && \
> @@ -182,7 +183,7 @@
>  # Check out source from Bazaar.
>  BZR_REVISION ?= before:date:$(GARVERSION)
>  BZR_CHECKOUT_OPTS ?= -r "$(BZR_REVISION)" --lightweight
> -bzr//%:
> +$(CACHEDIR)/bzr/%:
>       mkdir -p $(DOWNLOADDIR)/bzr
>       cd $(DOWNLOADDIR)/bzr && \
>               bzr checkout $(BZR_CHECKOUT_OPTS) $(BZR_PATH) $(DISTNAME) && \
> 
> === modified file 'gar.mk'
> --- gar.mk    2011-04-13 10:52:47 +0000
> +++ gar.mk    2011-10-17 19:30:06 +0000
> @@ -29,6 +29,7 @@
>  TEMPDIR = $(GARDIR_ABSOLUTE)/gar.tmp
>  FILEDIR ?= files
>  DOWNLOADDIR ?= download
> +CACHEDIR ?= cache
>  COOKIEDIR ?= cookies
>  WORKDIR ?= work
>  LOGDIR ?= logs
>