Re: Coverity Open Source Defect Scan of Emacs

[Alan Mackenzie]

Good morning, Ben!

Just to make things quite clear from the start, I'm a contributor to
Emacs, but have no other responsibilities on the project.  So what I'm
writing here is a purely personal point of view; I don't speak for the
project.

On Wed, 5 Apr 2006, Ben Chelf wrote:

>Hello Emacs Developers,

>   As some of you may have heard, last month Coverity set up 
>http://scan.coverity.com as a site dedicated to scanning open source 
>projects for defects.

May I suggest you write here "_free_ and open source projects"?  These
two terms describe two distinct philosophical stances, although they have
much in common.  Amongst the people to whom the differences matter is
Richard Stallman, the maintainer of Emacs.  ;-)

On a similar note, using "charset=windows-1252" rather than UTF (or even
plain old ASCII) might drop you the odd percentage point in approval
ratings.

>In just 1 month, over 4500 defects have been examined by various open
>source developers, and from what we can tell, it seems that there have
>been over 2500 patches to the scanned code bases! Due to popular
>request, I~Rm happy to announce that we~Rve added Emacs to the list of
>projects scanned on the site. For those of you not familiar with "scan"
>yet and by way of introduction ...

As a matter of interest, _who_ has been asking?  Are they, in general,
individual programmers, or project managers who perhaps have reservations
about Emacs, or possibly other people.

>   I'm the CTO of Coverity, Inc., a company that has technology that 
>performs static source code analysis to look for defects in code. You 
>may have heard of us or of our technology from its days at Stanford (the 
>"Stanford Checker"). The reason I'm writing is because we have set up a 
>framework internally to continually scan open source projects and 
>provide the results of our analysis back to the developers of those 
>projects. To see the results of the project, check out:

>http://scan.coverity.com

I've just had a look at it.  The page seems aimed more at managers than
hackers.  Have you got a page which goes into more technical detail?  For
example, what sort of bugs does your product find, what sort does it
_not_ find, what resources does it need (OS, amount of disk space,
recommended minimum processing power...), what source languages does it
scan?  The last is particularly important to Emacs since, as you will
surely know, the bulk of Emacs is written in Lisp, with only a
(relatively) small core in C.  A product which doesn't analyse Lisp would
be of limited benefit to Emacs.  And, critically important, is your
product free software?

You are also asking for people's telephone numbers to register, something
unusual indeed, but don't give any assurance that they won't be passed on
to evil people or used inappropriately.  Nobody likes being rung up
during American working hours if it happens to be 3 o'clock in the
morning and they're fast asleep.

>   My belief is that we (Coverity) must reach out to the developers of 
>these packages (you) in order to make progress in actually fixing the
>defects that we happen to find, so this is my first step in that
>mission. Of course, I think Coverity technology is great, but I want to
>hear what you think and that's why I worked with folks at Coverity to
>put this infrastructure in place. The process is simple -- it checks out
>your code each night from your repository and scans it so you can always
>see the latest results.

>   Right now, we're guarding access to the actual defects that we report 
>for a couple of reasons: (1) We think that you, as developers of Emacs, 
>should have the chance to look at the defects we find to patch them 
>before random other folks get to see what we found ....

I don't think making these bugs public would be a problem at all.  Emacs
is a program with very few security aspects (though there are some).  As
mature hackers, none of us would take it as a personal slight to have our
bugs pointed out.  This mailing list, address@hidden, would be an
appropriate place to report these bugs.  address@hidden would be
the other appropriate place.  Bugs are to be fixed, not hidden!

>... and (2) From a support perspective, we want to make sure that we
>have the appropriate time to engage with those who want to use the
>results to fix the code.   Because of this second point, I'd ask that if
>you are interested in really digging into the results a bit further for
>your project, please have a couple of core maintainers and/or developers
>reach out to us to request access.

Emacs is about to release a new version, and most of us are busy tidying
up the "last few bugs", so now isn't a good time to be learning new tools
and processes, though there may be people who'll have a look at them.

>As this is a new process for us and still involves a small number of
>packages, I want to make sure that I personally can be involved with the
>activity that is generated from this effort.

Probably the best way would be to discuss the bugs on this mailing list,
which would be much more efficient than exchanging emails with
individuals, since the details will have to be discussed here anyway.  No
request for any sort of confidentiality would be acceptable, since this
would utterly violate the spirit of free software.  Richard Stallman
doesn't let _any_ bugs slip by, and is well known (and appreciated) for
nagging people to get them fixed.  ;-)

>   So I'm basically asking for people who want to play around with some 
>cool new technology to help make source code better. If this interests 
>you, please feel free to register on our site or email me directly. And 
>of course, if there are other packages you care about that aren't 
>currently on the list, I want to know about those too.

OK.  Again, could you either give a link to a page describing your
product in technical terms, or create such a page if it doesn't already
exist.  Even if the product doesn't cost any money, it will certainly
cost a fair bit of time to download, install and evaluate, so it's only
reasonable to expect this description.

Is the product free software?  If not, it will meet with some antipathy
from free software projects such as Emacs.

>   If this is the wrong list, my sincerest apologies and please let me 
>know where would be a more appropriate forum for this type of message.

No, I think this the right place.

>Many thanks for reading this far...

No problem!

>-ben

>  Ben Chelf
>  Chief Technology Officer
>  Coverity, Inc.

-- 
Alan Mackenzie (Munich, Germany)