autoconf problems with cyrus imapd 2.2.5

[Kevin]

Hi All-

I'm experiencing what I believe to be autoconf problems while trying to 
build cyrus imapd 2.2.5.  I'm not sure if the problems stem from my 
autoconf installation or from the imapd use of it or something else.

I'm using Gentoo Linux as my base distribution, but I've done a great 
deal of experimentation from the command line (doing ./configure, make, 
make install myself to try and figure out what's going wrong).

Some of this post relates to Gentoo-specific information, and I'm not 
trying to get information about that issue in this list, but I explain 
what I'm doing with Gentoo because not doing so leaves the whole post 
without much context.  Please feel free to just scan the stuff about 
Gentoo if you wish, but I'm pretty sure that the root problem is in 
some way related to autoconf which is why I'm posting in this list.

I'm trying to write a new ebuild script for cyrus-imapd-2.2.5 (which is 
the most current version of the stable source branch).  An ebuild script 
is a Gentoo method for automating much of the configuration and 
installation of a package from sources.  I'm basing this new script
on an already functional ebuild script for cyrus-imapd-2.2.3.  In many 
cases, simply renaming the ebuild script with the package's new version 
number (eg. mv cyrus-imapd-2.2.3.ebuild cyrus-imapd-2.2.5.ebuild) is 
enough to accomplish the goal of using it to install the newer package.  
Cyrus imapd 2.2.3 was apparently considered a beta level package and 
2.2.5 is considered stable, so I'd like to use it rather than 2.2.3.

In preparation for this project, I tested the 2.2.3 ebuild script by 
using it to install cyrus imapd 2.2.3 (just in case there were bugs in 
the 2.2.3 ebuild script itself).  There were no apparent problems 
during the emerge process (Gentoo install method that uses the ebuild 
script) itself (I haven't configured or run the server program), so I 
took that as reasonable proof that the 2.2.3 ebuild script was bug-free 
(at least enough so to serve my purposes).

After renaming and making some very slight modifications of the 2.2.3 
ebuild script (which does successfully install cyrus-imapd-2.2.3), 
trying to install 2.2.5 with my new script fails with:

================
Calculating dependencies ...done!
>>> emerge (1 of 1) net-mail/cyrus-imapd-2.2.5 to /
>>> md5 src_uri ;-) cyrus-imapd-2.2.5.tar.gz
>>> Unpacking source...
>>> Unpacking cyrus-imapd-2.2.5.tar.gz 
to /var/tmp/portage/cyrus-imapd-2.2.5/work
 * Applying cyrus-imapd-libwrap.patch...                                        
                
[ ok ]
 * Applying cyrus-imapd-2.2.5-db4.patch...                                      
                
[ ok ]
 * Recreating configure...                                                      
                
[ ok ]
sed: can't read configure: No such file or directory

!!! ERROR: net-mail/cyrus-imapd-2.2.5 failed.
!!! Function src_unpack, Line 71, Exitcode 2
!!! sed failed
================

The problem comes from the last line of the src_unpack() function in the 
ebuild script:

===================
src_unpack() {
  unpack ${A} && cd "${S}"

  # Add drac database support.
  if [ "`use drac`" ] ; then
          epatch "${S}/contrib/drac_auth.patch"
  fi

  # Add libwrap defines as we don't have a dynamicly linked library.
  if [ "`use tcpd`" ] ; then
          epatch "${FILESDIR}/${PN}-libwrap.patch"
  fi

  # DB4 detection and versioned symbols.
  epatch "${FILESDIR}/${P}-db4.patch"

  # Fix master(8)->cyrusmaster(8) manpage.
  for i in `grep -rl -e 'master\.8' -e 'master(8)' "${S}"` ; do
          sed -e 's:master\.8:cyrusmaster.8:g' \
                  -e 's:master(8):cyrusmaster(8):g' \
                  -i "${i}" || die "sed failed"
  done
  mv man/master.8 man/cyrusmaster.8
  sed -e "s:MASTER:CYRUSMASTER:g" \
          -e "s:Master:Cyrusmaster:g" \
          -e "s:master:cyrusmaster:g" \
          -i man/cyrusmaster.8 || die "sed failed"

  # Recreate configure.
  export WANT_AUTOCONF="2.5"
# sh   # added by me for troubleshooting
  rm -f configure config.h.in
  ebegin "Recreating configure"
  sh SMakefile &>/dev/null || die "SMakefile failed"
  eend $?

  # When linking with rpm, you need to link with more libraries.
  sed -e "s:lrpm:lrpm -lrpmio -lrpmdb:" -i configure || die "sed failed"
}
===================

This sed command expects to find a file named, "configure" and it 
doesn't.

The reason that it finds no configure file is that it wasn't recreated 
with the "sh SMakefile &>/dev/null || die "SMakefile failed" line.

To figure out why, I uncommented the "# sh   # added by me..." line, 
commented the "rm -f" line and the "sh SMakefile" line and did some 
experimenting.

First, I just tried doing what the ebuild script would do:
# rm -f configure config.h.in
# sh SMakefile

The SMakefile step fails with:
===============
sh-2.05b# sh SMakefile
aclocal -I cmulocal
autoheader
Can't locate object method "path" via package "Request" 
at /usr/share/autoconf/Autom4te/C4che.pm line 69, <GEN1> line 111.
autoheader-2.58: /usr/bin/autom4te failed with exit status: 1
autoconf
Can't locate object method "path" via package "Request" 
at /usr/share/autoconf/Autom4te/C4che.pm line 69, <GEN1> line 111.
===============

I'll admit to having an extremely limited understanding of autoconf and 
automake and so here's where I'm stuck.

I've been able to figure out that we're deleting the configure and 
config.h.in files because we're patching some of the scripts that go 
into making these files, and we thus need to regenerate them.  For 
example, if we try skipping the rm -f step and sh SMakefile step, then 
the emerge fails with complaints of not being able to find db_create in 
the ldb libraries.  When berkdb.m4 is patched and configure and 
config.h.in are regenerated, then the configure process searches for 
and finds the db_create_4001 functions in ldb---this I learned because 
after trying the above (sh-2.05b# sh SMakefile), I started another 
shell, did a 
# cd /var/tmp/portage/cyrus-imapd-2.2.5/work/cyrus-imapd-2.2.5/ and then
did a
# sh SMakefile
from there.  This succeeds, but makes a broken configure script---it 
includes stuff that was apparently brought in directly from 
configure.in.  For example, it looks like every line from here:
AH_TOP([
on down in configure.in is just copied into configure.

So with that in place, I went back to the first shell (invoked by the 
ebuild script) and exited from it, thus continuing the emerge process 
which then fails with:

=================
...
checking for net-snmp-config... no
checking for sprint_objid in -lsnmp... no
checking UCD SNMP libraries... no
./configure: line 8228: syntax error near unexpected token `newline'
./configure: line 8228: `AH_TOP('

!!! ERROR: net-mail/cyrus-imapd-2.2.5 failed.
!!! Function econf, Line 365, Exitcode 2
!!! econf failed

=================

I don't understand why
# sh SMakefile
invoked from a shell called by the ebuild script fails (undoubtedly in 
the same way and for the same reason that it fails when called directly 
in the ebuild script but the output is redirected to the bit bucket), 
but does not fail when invoked from a separate shell (perhaps it needs 
environment variables that are not present in the shell invoked by the 
ebuild script?).  Furthermore, why does running sh SMakefile from the 
other script make a broken configure script?

To try and understand why, I did pretty much the same thing as above but 
with the 2.2.3 ebuild script being used to install cyrus imapd 
2.2.3.  What I found was that the "sh SMakefile" step in the ebuild for 
2.2.3 _does_ succeed and generates a completely functional configure 
script which can then be used to emerge the package.  But when I shell 
out of the ebuild script, start another shell, cd to the working 
directory, and then do a # sh SMakefile, this process also makes a 
broken configure script (broken in the same was as for 2.2.5).

Can anyone tell me what's going on here and how to fix it?

I started reading the info pages for automake and autoconf and it looks 
like I would be doing that for quite a long time and possibly not 
solving the problem by doing so, so I'm hoping someone here can give me 
some pointers.

I've asked in the gentoo-dev mailing list for some thoughts on this 
problem, and someone mentioned that it might be a version problem 
(where a script should be created with a different version of autoconf 
than what I'm using), but I don't see much evidence of that.  None of 
these scripts have any WANT_AUTOCONF=... lines.

I've tried all of this completely outside the context of Gentoo and 
ebuilds, just doing by hand what the ebuild script would do if it ran 
(patching the berkdb.m4 file by hand), and the problems are still there 
and this is why I don't think it's a Gentoo problem but rather an 
autoconf problem.  autoconf has worked perfectly on this system for 
many months when building other packages.  I realize that the problem 
may also be with the cyrus imapd tarball itself, and that's my next 
avenue to explore, but I was hoping to get some ideas from folks in 
this list on why the autoconf programs are behaving this way first 
(failing inside the ebuild script but succeeding outside of it; 
succeeding outside the ebuild script, but creating a broken configure 
script).

Thanks in advance.

-Kevin