Re: lynx-dev FORCE_SSL_PROMPT:NO

[Stef Caunter]

Thanks. Revision attached and below.

I get the impression that linux is "better at", (handles
differently?) this than the BSDs, and so have indicated
that a bit with the environment variable stuff.

I emphasized the environment variable since that is what
ultimately seemed most grievous on my BSD systems.

I hope I've dealt with it okay.

Also added advice to get IE's certs for updating.

--Stef

On Wed, 23 Jul 2003, Doug Kaufman wrote:

>
> A good discussion, but I believe that some of the statements are
> incorrect. Comments below.
>
> > Briefly, the procedure will involve confirming the default system location 
> > for
> > certificates, setting and exporting the environment variable SSL_CERT_DIR,
> > and hashing the certificates found in that directory using an openssl 
> > utility
> > to enable recognition.
>
> This should probably say "possibly putting a value for SSL_CERT_DIR in
> the environment". Note that SSL_CERT_DIR only needs to be used if the
> certs directory is different from the default compiled into the openssl
> library. "Exporting" is specific to certain shells (e.g. bash, ksh, sh)
> but doesn't apply to tcsh or csh.
>
> > It is assumed that openssl has been installed correctly, that SSL_CERT_DIR 
> > is
> > /usr/local/ssl/certs, and that lynx has been compiled --with-ssl. If the
> > default location for certs on your system is different you will have to
> > substitute that location for /usr/local/ssl/certs in the following 
> > instructions.
>
> Once again, SSL_CERT_DIR only needs to be set if a non-default location
> is used.
>
> > The source for openssl will be required in order to access the c_rehash 
> > utility
> > and the CA cert bundle.
>
> You don't need the source. When OpenSSL is installed, c_rehash is
> installed in a binary directory (default /usr/local/ssl/bin). There is
> no CA cert bundle distributed with OpenSSL. The OpenSSL team
> specifically decided NOT to do that. Getting a set of trusted
> certificates is left up to the installer.
>
> > So the next thing to do is to hash the cert using c_rehash in the default
> > location for your system (SSL_CERT_DIR, the oft-referred to
> > /usr/local/ssl/certs), and to set the environment variable so that openssl,
> > and lynx, can find the certs.
>
> Once again, SSL_CERT_DIR is for non-default installations. If you use
> the default no environment variable needs to be set.
>
> > INSTALLING OR UPDATING THE CA BUNDLE:
> >
> > Now would be a good time to check to see if you have the bundle of CA certs
> > in your /usr/local/ssl/certs, or to update them. Openssl and mod_ssl ship
> > with them.  They are in the certs directory of the openssl source tree.
>
> The certs in the OpenSSL certs directory are for testing purposes only
> and are not suitable for use in accessing secure sites. The ca-bundle
> from mod-ssl is somewhat old. It might be better to advise users to
> extract from a current version of Netscape or Internet Explorer rather
> than use this bundle. I believe that each extracts as a PKCS7 file and
> needs to be converted with something like:
> "openssl pkcs7 -inform DER -in site_name.crt -outform PEM -out site_name.pem 
> -print_certs -text"
>
> > Copy them to /usr/local/ssl/certs.
>
> The bundle should be copied to the default directory for bundles
> (usually /usr/local/ssl) and renamed to "cert.pem". The certs directory
> is for individual certificates.
>
> > We now have all of the certs we wish to trust in our certs directory.
> > Run the perl script c_rehash, which ships with the openssl source, and is
> > located in the tools directory of the openssl source tree.
> >
> > As root, run:
> >
> > ./c_rehash
> >
> > This is a perl script that runs openssl commands which creates the files
> > named after the hash values of the certs in the default directory for certs.
> > The output looks like this:
> >
> > Doing /usr/local/ssl/certs
> > vsignss.pem => f73e89fd.0
> > vsign3.pem => 7651b327.0
> > ...more output
> > <snip>
> >
> > All pem encoded certs in /usr/local/ssl/certs will now be recognized as long
> > as we perform the last step.
>
> This is only necessary if adding individual certificates. The
> alternative is to concatenate them to the bundle in /usr/local/ssl.
>
> > SETTING AND EXPORTING THE ENVIRONMENT VARIABLE SSL_CERT_DIR:
> >
> > Almost done! The last thing we _have_ to do is set the environment variable
> > SSL_CERT_DIR in our shell initialization .profile or .*shrc, or 
> > /etc/profile,
> > like so:
> >
> > SSL_CERT_DIR=/usr/local/ssl/certs
> > export SSL_CERT_DIR
> >
> > This environment variable _must_ be set, and it must be exported!
>
> This is necessary only if using a non-default location. "Exporting" is
> dependent on the shell being used.
>
> I would also add that the environment variable "SSL_CERT_FILE" applies
> to the cert-bundle if used outside of the default location compiled into
> OpenSSL. I am not sure that there is an easy way to determine the
> default locations if you didn't compile OpenSSL yourself. I have used
> "strings libcrypto.a", then searched for SSL_CERT_FILE. The default
> locations are usually located near that string. On the linux machine on
> my ISP, these are /usr/share/ssl and /usr/share/ssl/certs.
>
> Overall, I think this is a clear summary. I would recommend fixing the
> problems I mentioned before putting this in the lynx distribution.
>                             Doug
>
>
>
> --
> Doug Kaufman
> Internet: address@hidden
>
>
> ; To UNSUBSCRIBE: Send "unsubscribe lynx-dev" to address@hidden
>

        Lynx SSL support for certificates - README.sslcerts file

BACKGROUND:

The original README.ssl document for lynx stated:

        Note that the server... may not have a valid certificate. Lynx will not
        complain, as it does not yet support certificates...

Such lack of support is no longer the case.  Lynx now features excellent
certificate management through the openssl project.  There is almost no
online documentation available regarding how to use openssl's certificate
management with other programs, so this will accompany lynx and hopefully
encourage good practical security for unix clients.

Lynx relies on openssl to not only encrypt connections over https, but also to
determine whether it should even accept a certificate and establish a secure
connection with a remote host. Because of this reliance upon openssl by lynx,
most of this tutorial deals with how to use openssl to "install" both
commercial CA cert bundles as well as self-signed certs from trusted sources
and, most importantly, how to get them recognized by lynx.

While lynx on a correctly set up system will transparently accept valid
certificates, not all systems enjoy such functionality. Further, as noted
above, older versions of lynx do not perform any validity check on a
certificate.

There is also the common case of wanting to trust, use and install a
self-signed certificate from a known source and have it be trusted by the
browser.

Briefly, the procedure will involve confirming the default system location for
certificates, possibly putting a value for SSL_CERT_DIR in the environment,
and hashing the certificates found in the default locations using an openssl
utility to enable recognition.

THE CURRENT SITUATION:

Prior to lynx2.8.5dev9, lynx did not check at all for certificate validity.

Since lynx2.8.5.dev9, lynx has reported this error:

SSL error:unable to get local issuer certificate-Continue? (y)

whenever an https connection was initiated and the certificate could not be
found,  for whatever reason, by openssl, and therefore lynx.

This checking for a certificate is an enhancement to security, but rather
tediously generates errors at each https browser request.

The ability to turn off reporting of this error to the user was added to
lynx2.8.5dev16 as the FORCE_SSL_PROMPT setting in lynx.cfg as noted in the
CHANGELOG:

        This lets the user decide whether to ignore prompting for questionable
        aspects of an SSL connection.

While this is a convenient setting to employ when using lynx to script
https -dumps, it by definition ignores the issue of certificate validity
altogether. Those concerned with proper certificate management and
the maintenance of a store of updated CA certificates will be uncomfortable
with this relaxed security setting.

PRELIMINARY PROCEDURES:

It is assumed that openssl has been installed correctly, that the default
cert location is /usr/local/ssl/certs, and that lynx has been compiled
--with-ssl.

If the default location for certs on your system is different you will have
to substitute that location for /usr/local/ssl/certs in the following
instructions.

On some systems, SSL_CERT_DIR only needs to be set if a non-default location
is used. This appears to be true with linux. It seems to want to be set on
BSD type systems.

Note that when OpenSSL is installed, c_rehash is installed in a binary
directory (default /usr/local/ssl/bin).  You will need to know where it is
on your system. The commands whereis or find will be useful.

Note also that there is no CA cert bundle distributed with OpenSSL. The
OpenSSL team specifically decided NOT to do that. Getting a set
of trusted certificates is left up to the installer. It is a fairly trivial
procedure to pull them out of a recent version of Internet Explorer.

If you simply need to have commercially provided certificates trusted by lynx,
you can skip down a few lines to the INSTALLING OR UPDATING THE CA BUNDLE
section.

INSTALL A SELF-SIGNED CERTIFICATE:

When you would like to trust a self-signed (non-commercial) certificate you will
need to get hold of the actual file. If it's a cert local to your network you
can ask the sysadmin to make it available for download as a link on a webpage.

If such file is not human-readable it's probably DER formatted and will need to
be converted to PEM format to allow openssl to use it.

To convert DER formatted certificates into something openssl can deal with:

Save the cert as site_name.crt in a directory. In that directory, type:

openssl x509 -inform DER -in site_name.crt -outform PEM -out site_name.pem

You can now copy this individual cert into the directory for that, usually
/usr/local/ssl/certs.  The alternative is to concatenate them to the bundle
cert.pem in /usr/local/ssl (See INSTALLING OR UPDATING THE CA BUNDLE below).

The file will now be in an acceptable format to openssl, PEM encoded. However,
openssl, and by extension lynx, will not know about it until that cert is
present in a file named after the hash value of that cert, in the default
/usr/local/ssl/certs.

So the next thing to do is to hash the cert using c_rehash.

INSTALLING OR UPDATING THE CA BUNDLE:

Now would be a good time to check to see if you have the bundle of CA certs
/usr/local/ssl/cert.pem, or to update them.

It is best to extract the current bundle from a current version of Netscape or
Internet Explorer (export them all from IE and get it onto your system).
It extracts as a PKCS7 file and needs to be converted with something like:

openssl pkcs7 -inform DER -in bundle.crt -outform PEM -out cert.pem \
-print_certs -text

The resulting cert.pem file should be copied to the default directory for
bundles (usually /usr/local/ssl) and renamed to "cert.pem". It will also
process if it is in /usr/local/ssl/certs.

We now have all of the individual certs we wish to trust in our certs
directory, and the most recent bundle of CA certs as well.

Confirm that you have the script c_rehash; if it is not found, a copy is
usually located in the tools directory of the openssl source tree.

As root, run:

c_rehash

This is a perl script that runs openssl commands which creates the files
named after the hash values of the certs in the default directory for certs.
The output looks like this:

Doing /usr/local/ssl/certs
vsignss.pem => f73e89fd.0
vsign3.pem => 7651b327.0
...more output
<snip>

All pem encoded certs in /usr/local/ssl/certs will now be recognized.

SETTING AND EXPORTING THE ENVIRONMENT VARIABLE SSL_CERT_DIR:

We _may_ need to set the environment variable SSL_CERT_DIR in our shell
initialization .profile or .*shrc, or /etc/profile, like so:

SSL_CERT_DIR=/usr/local/ssl/certs
export SSL_CERT_DIR

If lynx is still not recognizing certs, this environment variable may need
to be set; if it needs to be set it must be exported!

Also note that the environment variable "SSL_CERT_FILE" applies to the
cert-bundle if used outside of the default location (/usr/local/ssl/cert.pem)
compiled into OpenSSL.

Make sure you have FORCE_SSL_PROMPT set to PROMPT in lynx.cfg like so:

FORCE_SSL_PROMPT:PROMPT

You will now connect without error to https servers with trusted certs, but
will still get this error for untrusted certs:

SSL error:self signed certificate-Continue? (y)

A quick check confirms that these procedures have the same effect with ssl
errors in the pine program.

--Stef Caunter <address@hidden>
Mohawk College Department of Computer Science
Hamilton Ontario Canada

README.sslcerts
Description: ssl/certs README revised