[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: testing framework and package.el

From: Christian Ohler
Subject: Re: testing framework and package.el
Date: Mon, 11 Oct 2010 20:26:38 +1100

 On 5/10/10 14:09, Stephen J. Turnbull wrote:
Christian Ohler writes:

  >    On 5/10/10 2:18, Chong Yidong wrote:
  >  >  Christian Ohler<address@hidden>   writes:
  >  >>  Is there a deep reason
  >  >>  why Emacs currently has test/ as a separate top-level directory?
  >  >  Because the test files are not to be distributed with the tarball.

  >  Why not?

Conceptually, tests should be developed separately from the core, just
as the Lisp Reference is.  Of course, in practice it's not that easy,
but I think it's a good idea to structure the tests and the reference
at the same level as the core sources.

XEmacs experience with its tests has shown that having the tests
subsidiary to the source tree leads to regressions.  Specifically,
changing the source code leads to changes in behavior, and these
changes are "sanctified" by writing tests after the fact.  But these
tests are *not* run against older versions, leading to unplanned
divergence in behavior.  Occasionally these divergences are annoying
to users or third-party libraries.

AIUI, XEmacs has its tests in a tests/ directory next to lisp/, in the same repository. Are you saying this has proven to be a bad idea?

The most frequent use cases of the testing system are probably browsing existing tests, writing new tests, running tests against locally a modified Emacs, and maintaining tests. We should lay things out in such a way that these actions are as easy as possible. I expect that running the current tests against older Emacs versions is something that we will want to do much less frequently, so we should not optimize for this use case at the expense of the others.

Anyway, I don't think the directory layout matters much as long as there are very few tests. I went with a lightweight solution and put the tests all in one directory test/automated/ like XEmacs does. (I'm referring to the current state of the ERT repository here, this code isn't installed yet.) I think the primary use cases would benefit from having the tests right next to the corresponding sources rather than in a separate tree, but I'm not going to fight over it. (Emacs functions for jumping from a file in lisp/ to a corresponding file in test/ solve only part of the problem since they only help when viewing or editing the files, but not while operating on them with shell commands, VC, or other tools.)

We can always move the files around later if we decide on a different convention.


reply via email to

[Prev in Thread] Current Thread [Next in Thread]