Re: Locations of Tests

[Phillip Lord]

John Wiegley <address@hidden> writes:

>>>>>> Michael Albinus <address@hidden> writes:
>
>> Before we rename everything, wouldn't a simple directive in the source file
>> do the job?
>
>> ;; Package-Tests: test/automated/file-notify-tests.el
>
> That would also work, but having a naming structure also makes it easier for
> contributors to know where new files they create should go. So I'd prefer to
> have a convention, rather than make it freeform, with a set of textual links
> to relate the two sets.

I'd tend to agree.

I think I'd like to get a specification sorted first before making the
changes. I've included my starter-for-ten below. I'm willing to take any
feedback here -- or I could add it to the emacs-repo as an orphan
branch, then people could change directly.

I've given locations for test files, and resources (data files). This is
both for Emacs-lisp source files and for lisp tests for features
provided in C (inotify-test.el for example). There are also suggestions
for ELPA packages.

I've said nothing about C tests for the C core, because I know nothing
about C testing (and almost nothing about C coding!). Adding a
buffer-local as suggested above as a legacy might make sense.

Phil

#+TITLE: The Location of Emacs-Lisp Tests

* Introduction

In this document, we describe the relationship between Emacs-Lisp files and
their associated automated test files.


* In Emacs

The Emacs repository contains a very large number of Emacs-Lisp files, many of
which pre-date both formal package support for Emacs and automated unit
testing. The test layout is, therefore, somewhat different than for ELPA
packages.

All paths are relative to the Emacs root directory.

** Source

Lisp files are stored in the ~lisp~ directory or its sub-directories.
Sub-directories are in many cases themed after packages (~gnus~, ~org~,
~calc~), related functionality (~net~, ~emacs-lisp~, ~progmodes~) or status
(~obsolete~).

C source is stored in the ~src~ directory, which is flat.

** Test Files

Automated tests should be stored in the ~test/automated~ directory. Tests
should reflect the directory structure of the source tree; so tests for files
in the ~emacs-lisp~ source directory should reside in the
~test/automated/emacs-lisp~ directory.

Tests should normally reside in a file with an ~-test~ added to the name of
the tested source file; hence ~ert.el~ is tested in ~ert-test.el~, or
~pcase.el~ is tested in ~pcase-test.el~.

Where features of the C source are tested using Emacs-Lisp test files, these
should reside in ~/test/automated/c~ and be named after the C file.

** Resource Files

Resource files for tests (containing test data) should reside in a directory
named after the feature with a ~-resources~ suffix. Hence, tests files for
~flymake.el~ should reside in a directory called ~flymake-resources~.

No guidance is given for the organisation of resource files inside the
~-resource~ directory; files can be organised at the author's discretion.


* In ELPA

All paths are given relative to the package root.

** Source Files

ELPA lisp files should be stored at top-level within the package.

** Test Files

Test files should normally reside in the ~test~ directory, and be named after
the file being tested.

** Resources Files

Resource files for tests should reside in the ~dev-resources~ directory. No
guidance is given for the organisation of resource files inside the
~dev-resource~ directory; files can be organised at the author's discretion.