[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: mark expensive tests

From: Michael Albinus
Subject: Re: mark expensive tests
Date: Mon, 04 Jan 2016 09:16:28 +0100
User-agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1.50 (gnu/linux)

John Wiegley <address@hidden> writes:

>> make check
>> make check-expensive
>> make check-very-expensive
> Prior art from an existing GNU project is even better. Now, who wants to make
> it so?

I could start with it, in the emacs-25 branch. Those targets do cover #2,
#3 and #4 tests. Which make target shall we use for #1? "make check-smoke"?

Also, we need to decide how to distinguish such test categories. I don't
believe we shall do it on test file basis, except maybe for the smoke
tests. For the other tests, we could request something like this in the
test functions:

#3 tests: (skip-unless (or (getenv "RUN_EXPENSIVE_TESTS")
                           (getenv "RUN_VERY_EXPENSIVE_TESTS")))

#4 tests: (skip-unless (getenv "RUN_VERY_EXPENSIVE_TESTS"))

Alternatively, we could extend ert.el by a mechanism, marking tests
optionally as "smoke", "normal", "expensive", "very-expensive". "normal"
would be the default category. This shall be in accordance to the GNU
Coding Standards, as Richard proposes.

Best regards, Michael.

reply via email to

[Prev in Thread] Current Thread [Next in Thread]