Re: Emacs Lisp's future

[Stephen J. Turnbull]

Richard Stallman writes:

 > I think the people who want strict encoding are talking about
 > network communication using open-network-stream.

Speaking only for myself, no, I mean all octet streams purported to
be encoded text, network or local.  Network streams can only be
considered safe in very carefully maintained environments, so the
risks are greatest there.

But there's no such thing as a truly local stream, since any given
stream may be a file downloaded from the network or provided by an
application of uncertain provenance.  There are three cases of
interest, AFAICS:

(1) The file or application is truly local, provided with the OS or
    created by the user.  In that case on a well-maintained system,
    the encoding should be valid, as you pointed out elsewhere.
    Therefore a strict policy should be transparent.  (See (3) for
    what I believe to be the main class of exceptions.)

(2) The file or application was downloaded from the network.  Emacs
    cannot know the provenance, and so the same care should be taken
    as with a network stream.

(3) The application is trustworthy, but produces invalid encoded text
    in some well-understood situations.  In this case the Lisp program
    should be allowed to opt out of default validation and provide its
    own.  Preferably only in the specific situations rather than
    globally.

An example of (3) is David's case, with AUCTeX handling of TeX error
messages containing non-unibyte text.)  AFAIK such applications are
quite rare nowadays.  TeX is a special case because it is one of the
few applications whose behavior is specified extremely precisely but
in an encoding-oblivious way.[1]

As an example of special validation in (3), AIUI in TeX error
messages, only a very few leading and trailing bytes of quoted source
text should be invalid.  Thus the rest should be valid, and the user
probably should be notified of unexpected rawbytes.  (That's up to the
Lisp programmer, of course.  Still I think such flexible validation is
in the user's interest if the programmer is willing to provide it.)

I am unaware of other large classes of exceptional cases in modern
GNU/Linux systems, or the major proprietary OSes.

I understand David and Eli to be of the opinion that in practice there
is insignificant risk to Emacs or its users from any form of invalid
or malicious input, from the network or local.  I disagree.

Footnotes: 
[1]  I'm referring to the TRIP test.  This specification effectively
assumes a unibyte encoding, and so it is likely to be very difficult
to create a TeX implementation that handles Unicode conformant to the
standard *and* passes TRIP.  I'll take a TeX that passes TRIP any day!