[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Pan-users] ssl problems, not working on ports 443 or 563
Re: [Pan-users] ssl problems, not working on ports 443 or 563
Tue, 3 Apr 2012 08:05:50 +0000 (UTC)
Pan/0.136 (I'm far too busy being delicious; GIT 72905f5 /st/portage/src/egit-src/pan2)
Joe Zeff posted on Mon, 02 Apr 2012 12:03:24 -0700 as excerpted:
> On 04/02/2012 11:46 AM, Duncan wrote:
>> On gentoo I build from sources, but I've not had dependency hell since
>> my Mandrake days, as there's usually at least a masked version of the
>> ebuild that I can unmask somewhere, that has all the deps listed.
> I know that apt-get, the Debian/Ubuntu package manager, will resolve
> deps and bring them in as needed, as I've seen it happen on my sister's
> Ubuntu box. And, yum, the Fedora equivalent, does the same. I'm not
> familiar with gentoo, so I have to ask: doesn't your package manager
> take care of such things, or are you talking about building packages
> that are not from the gentoo repos?
Umm... somewhere along the line, one or both of us got off track. I
/thought/ what I wrote was perfectly clear, but obviously not, at least
not to you, or your question either wouldn't be needed or would itself be
clear to me as well.
So I'll try backing up a bit and do a retrace. Let's see if that helps.
1) Bob is trying to build pan from current git sources (so presumably he
has a pan package and its deps are installed, but any new deps in the git
version are not). It builds fine without SSL support (thus the original
question, since he didn't know the ssl support wasn't built), but won't
build with it as his gnutls is too old (that's a new dep, since the ssl
support is new, no released version with it, yet).
2) So he needs a new gnutls to build the new pan with ssl support.
Again, the package he has would take care of deps as of the old gnutls,
but he needs a newer one for pan, with newer deps.
3) As it happens, one of those newer gnutls deps is nettle, and in gentoo
(what I use here) as well, the "stable" gnutls was too old for pan. They
hadn't stabilized the newer gnutls due to API changes related to its new
I suspect that we're all clear to that point.
4) This is where I think the confusion may have started so I'll include a
bit more detail this time around. I don't know what distro bob is on but
I'm presuming it's not the Gentoo ~arch (aka testing) that I run.
Whatever it is, tho, they seem to be in about the same position Gentoo
itself was a couple months ago when I went thru the gnutls upgrade in
ordered to satisfy pan's new deps, myself. Originally I left out a bit
of gentoo detail as I doubted it mattered to him, but eh... throw it in
this time and see if it clears things up.
5) On gentoo the in-tree available versions of gnutls and their stability
level has changed since March 16:
2.10.x (2.10.5 currently, might have been 2.10.4 or whatever then):
No nettle required, stable for quite some time. (This is what Dave S
says is in Ubuntu latest, also.)
2.12.x (2.12.18 added in gentoo on March 16): Requires nettle AND p11-
kit. In theory, it's still 2.x so should maintain API, but apparently
there were some issues, as gentoo, and now it seems bob's distro... and
dave shocket's (which he said is Ubuntu latest)... stuck at 2.10.x stable
for quite some time.
Since March 16's gentoo addition of 2.12.18, various archs appear to be
stabilizing on it, with about a third of them (including the big ones,
amd64 and x86) now stable on 2.12.18, while the rest are still stable on
3.0.x (3.0.16 also gentoo-added on March 16): 3.x major version. Gentoo
has this available in-tree, but it's not stable yet, only ~arch
(testing). Before March 16's 3.0.16, 3.x was in-gentoo-tree but I
believe hard-masked, not visible for installation even for ~arch users,
unless they queried for it specifically, and deliberately unmasked it.
>From what I can see both from gentoo, and now from this thread I see the
same in other distros, the 2.12 version is where a lot of the BIG changes
occurred, including nettle and p11-kit dependencies, and really SHOULD
have been 3.0. The 2.x series should have remained stable at 2.10.x,
without the nettle and p11-kit deps, which only should have been added
for the 3.x series.
Oh, well, upstreams decide, distros (and end users advanced enough to
care) have to cope with it.
6) Now, it should be mentioned that gentoo is both automated-from-source
(end-user builds, but using an automated dependency tracking and build-
script system) and rolling distribution (no real distro version upgrades
after installation, individual package upgrades are introduced and
unmasked to ~arch, then to stable, over time). Additionally, unlike a
lot of distros but as can be seen from #4 above, it's the norm for
multiple versions of a particular package to be in-tree at the same time,
with various users on various versions, depending on whether they're
~arch or stable (or even have unmasked hard-masked versions), their
upgrade schedule and whether they religiously deep-upgrade (as I do), or
only pull in dependency upgrades when some leaf package they've installed
ups its dependency to require it, or the old versions get pruned from the
7) On gentoo, ~arch (testing) is generally defined as upstream-stable,
but gentoo-package testing, tho in practice, there's a bit of variance on
that per individual package. So with some exceptions, unless upstream
defines it as stable, it's not even a candidate for ~arch and later
stable. Upstream development versions, betas and rcs are often available
in-tree, but they normally remain hard-masked, only visible to those who
deliberately unmask them for their own reasons. Additionally, in many
cases especially for major projects such as kde/gnome/xorg/mozilla,
upstream unstable/development versions are not added to the main tree,
but can be found in various overlays. Often, the overlays have live-vcs-
version builds available as well, for the REALLY adventurous. There's a
few of them (hard-masked, of course) in the main tree, but not many.
8) Dependency-wise, gentoo's ebuilds (the build-scripts for individual
packages) specify all the usual dependency information, which is then
automatically handled by the package manager. That's why I said I'd not
seen dependency hell since back when I was on Mandrake. That's I guess
the short and direct answer to your question, but it's not complete.
9) Here, I normally run ~arch, but in addition to the main tree, have the
kde, x11/xorg and mozilla (for firefox) overlays added, and routinely
unmask various packages before they're unmasked in-tree.
For example, due to gentoo's build-from source, gcc is a critical system
package, and it doesn't tend to be unmasked even to ~arch until it can
build nearly everything at that stability level (so ~arch for me). gcc
4.6 still remains hard-masked, for instance, due to the fact that gentoo's
still stable on grub-legacy (0.97 with a whole slew of patches), with
grub2 packages in-tree but hard-masked, and there's apparently some
blocking boot-bugs when grub-legacy is built with gcc-4.6.
But I unmasked gcc 4.6.2 a few months ago here and did a full rebuild.
Everything I'm running (I've unmasked grub2 here) builds and appears to
run fine with gcc 4.6.2.
Similarly, I unmasked and ran the kde 4.8 prereleases, betas 1 and 2, and
rcs 1 and 2. (FWIW, kde 4.8.0 was released at the end of January, with
4.8.1 a month later. 4.8.2 should be out this week. So the pre-releases
were late last fall and early winter.) They were never in-tree at all as
they're pre-releases and weren't stable candidates, but I'm running the
kde overlay, which had them available.
***>>> (to end)
10) Because I'm always running at least ~arch, which by definition is
gentoo testing level ebuilds for upstream stable, and am often
deliberately unmasking the even less mature hard-masked versions, not
even ~arch, while gentoo's ebuilds DO specify dependency information, at
the immaturity level I run them, that dependency information isn't always
Thus, a more complete answer to your dependency question is yes, gentoo's
package management deals with it in general, but at the immature package
testing level I routinely run, those dependencies aren't 100% guaranteed
to be current and upto date. So I do very occasionally see too-low-a-
version errors when I try to build something.
However, because I DO religiously run deep upgrades (I don't ever NOT do
deep upgrades and my own helper scripts all assume deep), in practice, I
very seldom see such errors, because generally I've long since done that
upgrade anyway. Instead, I much more often see errors because I'm trying
to run versions too new as I've unmasked something that's hard-masked
precisely for that reason, an incompatible API change or the like, and
either the deps are specified in the ebuild and I have to try to figure
out how to get everything playing on either the old or new version, or
they're not yet high-side version limited or patches applied to build
with the new versions (I unmasked the still-masked version, not yet
expected by the currently specified dependencies, after all), and I have
to go looking for patches or unmasking sometimes live-vcs versions of the
depending packages, in ordered to get everything working with the new
But that's not dependency hell, since (1) it's almost always due to a
deliberate unmask on my part, and (2) it's almost always only a single
level of dependency changes I need to figure out. Beyond that, generally
I'm already updated, and/or the existing dependency specs are correct, so
I don't get into the sort of recursive "but I can't build this because it
depends on that, and I can't build that because it depends on something
else, N levels deep" sort of situation that is normally referred to as
11) Another contrast between gentoo and most distributions that helps
with this sort of thing, is that since gentoo assumes build-from-source
at the user end, it really doesn't make sense at all to try to split
packages into run-time and development sub-packages. IOW, there's no
libXXXX-dev packages on gentoo, there's only the libXXXX package,
including both the binary library and the related headers, *.pc files,
etc, that are normally only required if you're building a package
depending on that library. Since in gentoo, EVERY package is assumed to
be built from source, (almost, servant-ware binary-only packages are an
exception) ANY package depending on that library would require the build-
time bits to build and install anyway, and why would someone only install
the library, if they weren't going to be using it as a dependency for
some other package. On a source-based distro, it really doesn't make
sense to split up packages into runtime and -dev sub-packages, so gentoo
doesn't do so.
That alone seriously diminishes dependency hell issues, since it roughly
halves the number of package dependencies required. No more having to
track down libXXX-dev, even when one *IS* doing manual builds outside the
package system, because if the library is installed, so are the bits of
it normally found in the -dev package on binary-based distros. =:^)
12) So much for the general gentoo case, now we're back to the specific
case at hand, pan from live-git-repo sources. =:^)
As mentioned above, only git-pan requires gnutls at all, and that only
when the --with-gnutls configure option is enabled in ordered to handle
secure connections. FWIW, gentoo does have a live-git pan ebuild
(pan-9999, in gentoo, -9999 versions are used for live-vcs packages)
available, but it doesn't have a gentoo developer following things
closely enough to keep up with deps like this. As a result, I've
customized my pan-9999, keeping it in my own personal overlay, where I
can manage the deps changes myself. I'd submit the changes back to
gentoo, with optional deps changes hooked up to gentoo USE flags as
necessary, and do occasionally do so (usually in relation to a release
version bump), but generally only test the side of the option I'm
interested in (in this case, --with-gnutls), not both sides, and
sometimes don't even know how to properly define the mapping for both
sides, so I'm afraid I'm not ready to be a proper proxy-maintainer of the
gentoo pan-9999 ebuild yet -- my fixes only help on the one side, and
require real gentoo developer help to get them setup for the other.
So of course, when git-pan's deps change, I'm making the changes to my
ebuild to reflect them, myself. The package management system obviously
can't do that by itself, tho as with most distros, there's various
maintainer tools available to help with the process.
But a lot of the time, I'm not even aware of minimum version requirement
changes, because as I said, I routinely do deep upgrades myself, and with
occasional exceptions like this gnutls thing, generally have well above
minimal versions long since installed. So usually, the only time I
actually bump the in-ebuild deps is when Heinrich or someone mentions
current deps on list, and I go back and compare what's in the ebuild
against them, or when I sync to changes made in the gentoo ebuild since I
last checked it, sometimes in preparation for submitting a gentoo pan
package version-bump request, etc.
Of course right now, with gnutls being such a new dep, when the new pan
version comes out this summer (assuming a continuation of the annual bump
pattern, on or about August 1), if I file the version-bump request with
gentoo, I'll probably have to say I don't know exactly how to specify the
new deps and how to hook them up to the relevant SSL USE flag (there's no
old one yet to pattern after). Meanwhile, I haven't modified my ebuild
at all for that, but instead, have a bashrc file setup that portage
(gentoo's main PM and the one I use) will source before reading the
ebuild, that adds the --with-gnutls option via an available user
extension syntax. ( EXTRA_ECONF='--with-gnutls' ) So the ebuild doesn't
specify gnutls dependencies at all, I use the extension to inject the
configure script option, and with that the configure script simply picks
up the gnutls (3.0.16) I've manually unmasked and upgraded to.
13) But of course, most of that, being gentoo specific, was irrelevant to
bob and dave, as dave's running ubuntu and bob's presumably not running
gentoo either. So I had skipped all that, apparently leaving you with
the impression that gentoo's ebuilds don't handle deps automatically,
even tho I specifically stated that I hadn't experienced dependency hell
since my mandrake era, before I switched to gentoo. I still don't quite
understand how you got from that specific statement that I had NOT had
dependency hell on gentoo, that it didn't handle the deps well, but since
I had skipped the details as irrelevant in that context, I guess it must
have been due to omitted information.
I trust that my usual information overload approach has corrected that
problem now? =:^)
14) Fully back to the thread topic, now. Based on data from multiple
distros now, it seems that gnutls 2.10.x has been where many stabilized,
because 2.12.x and above require nettle and p11-kit. Meanwhile, pan's
new secure-connections feature requires the --with-gnutls configure
option, which requires gnutls 2.12+ as earlier versions apparently didn't
work so well in Heinrich's testing.
So it is that users trying that feature in git-pan are having to build a
newer gnutls from sources, since their distros are still stabilized on
2.10, and end up running into both the nettle and p11-kit recursive
dependency issues when they do so. Those in turn may recurse into
further necessary updates, thus triggering "dependency hell", the seeming
unending series of manual updates required just to build what was
originally thought to be a single package update.
Luckily for me, most of the work was already done for me in gentoo and
because I run quite new versions of everything anyway; I only had a
couple unmasks to do, plus of course rebuilding a couple other packages
that depended on gnutls after the upgrade, due to the ABI issues. But
I'd say about 95% of the complexity was already handled, and what wasn't,
was a level of issue that I'm used to dealing with relatively routinely
anyway, in my normal course of early unmaskings, running kde betas, etc.
People on the more ordinary binary distros, however, will be far more
likely to have to deal with at least 20% of the issues manually (unless
they're running redhat 5 or something, the packaging system will still
deal with the majority of the problem, but it'll still look like
dependency hell, especially to the uninitiated), where I had to deal with
only perhaps 5%, and on average, they'll be less accustomed to dealing
with that level of dependency resolution issue in the first place. So
they'll have perhaps 4X the exposed problem, but only be perhaps a third
as well prepared to deal with it, resulting in an experience that's
likely 4*3=12 times more difficult for them to deal with!
But I've seen suitably determined people bring even a redhat 5 system up
to current kde, taking days of recursive dependency builds to even get
kdelibs to build, but getting it done. If a suitably determined person
can do that, then surely, a suitably determined person can do the much
less difficult as it's far closer to current, upgrade from pan 0.133+ to
git-pan, and with it from gnutls 2.10 to at minimum 2.12, along with its
recursive dependencies in turn. There, there's only the two of them,
nettle and p11-kit, the others are likely to be upto date, instead of the
several dozen just to build kdelibs, that the poor guy upgrading that
redhat system was dealing with!
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman
- Re: [Pan-users] ssl problems, not working on ports 443 or 563, (continued)
- Re: [Pan-users] ssl problems, not working on ports 443 or 563, Heinrich Mueller, 2012/04/01
- Re: [Pan-users] ssl problems, not working on ports 443 or 563, bob, 2012/04/01
- Re: [Pan-users] ssl problems, not working on ports 443 or 563, Duncan, 2012/04/02
- Re: [Pan-users] ssl problems, not working on ports 443 or 563, bob, 2012/04/02
- Re: [Pan-users] ssl problems, not working on ports 443 or 563, Duncan, 2012/04/02
- Re: [Pan-users] ssl problems, not working on ports 443 or 563, Joe Zeff, 2012/04/02
- Re: [Pan-users] ssl problems, not working on ports 443 or 563,
- Re: [Pan-users] ssl problems, not working on ports 443 or 563, Joe Zeff, 2012/04/03
- Re: [Pan-users] ssl problems, not working on ports 443 or 563, bob, 2012/04/05
- Re: [Pan-users] ssl problems, not working on ports 443 or 563, Duncan, 2012/04/06