Re: [Emacs-diffs] master 75336a2: Merge branch 'feature/standard-test-location'

[Phillip Lord]

Dmitry Gutov <address@hidden> writes:

> On 11/28/2015 11:20 PM, Phillip Lord wrote:
>
>> In my experience, pulling from master leaves the "automatic" directory
>> there, although it's gone from git. I'd expected it to delete when the
>> equivalent was removed from git, but apparently not.
>
> Apparently it's being left there as long as there are some untracked (or
> ignored) files inside.

Ah okay. So if you make distclean before pulling it would be okay.

>> Well, there is a place for manual test material. ERT is good for doing
>> unit tests, but some integration tests are likely to remain. But if a
>> test is automatable, then I see no reason it should not run.
>
> Right. But all "manual" tests we have now should be automatable, I think.

I'll have a look at this maybe, although I'd prefer the original authors
of the tests ported them.


>> If we need an indent test location, then I think this would be good to
>> support, but it would be nice to have a standard naming scheme. I'd like
>> to add support to the make file so that tests (or the .log file) depend
>> on the file that they are testing. At the moment, if you edit (for
>> example) lisp/calc/calc.el, and then run make check, the calc-tests.el
>> file does not get run. This makes no sense to me. The same should be
>> true with indentation tests.
>
> 'make check' runs all tests, doesn't it? Including calc-tests.el.

No. make check runs all tests that are not up to date. So, it runs
calc-tests.el only if calc-tests.el has been changed.

> It would be nice to be able to run the tests corresponding to the
> current file only, but Makefile might not be the best place to
> implement that.

The make file before my changes allowed

make calc-tests
make calc-tests.log

The former dumps output to screen, the latter to file.

Now you can do any of:

make calc-tests
make lisp/calc/calc-tests
make lisp/calc/calc-tests.log

The make file on feature/tests-depend-on-source also makes tests depend
on the source file (based on file name). It uses Emacs to generate a
include make file like so:

lisp/calc/calc-tests.log: ../lisp/calc/calc.elc
lisp/calendar/icalendar-tests.log: ../lisp/calendar/icalendar.elc
lisp/emacs-lisp/cl-generic-tests.log: ../lisp/emacs-lisp/cl-generic.elc

Seems to work. If you are building Emacs a lot, then you just switch
from "make" to "make check" and it runs the relevant tests.

At the moment, I think, tests do not get routinely run. The coverage
hydra build has been failing on tests for 2 or 3 weeks, for instance.


> The .el extension points to emacs-lisp-mode already, doesn't it?

Then you are testing two things: auto-mode-alist *and* indentation.

>> I hadn't thought of the idea of unindenting first -- I shall add that.
>
> Actually, having -unindented files seems largely unnecessary to me. I ported
> that idea straight from the test suite inside the Ruby core, but the benefits
> are relatively small, and that would double the number of files. But hey, we
> can try it both ways.

No, I agree with you. unindenting first seems generally better to me.


>> It's got a "with-temp-buffers" macro also (for when you need several at
>> once), and "with-preserved-buffer-list".
>>
>> Anyway, that package is early days yet. We shall see if it become useful
>> as it develops.
>
> Have you considered just adding a few functions to ert-x? As a first step, I
> mean.

Not yet, no. I wanted to get something working and useful first. The
problem with adding it to ert-x is that it's core, and I think I'd
rather have it in ELPA in the short term.

Phil