emacs-bug-tracker
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

bug#51941: closed (Detect duplication of ERT tests)


From: GNU bug Tracking System
Subject: bug#51941: closed (Detect duplication of ERT tests)
Date: Thu, 18 Nov 2021 11:55:01 +0000

Your message dated Thu, 18 Nov 2021 12:54:18 +0100
with message-id <F8A641F2-CA3B-4EC1-A149-611EB571D7DD@acm.org>
and subject line Re: bug#51941: Detect duplication of ERT tests
has caused the debbugs.gnu.org bug report #51941,
regarding Detect duplication of ERT tests
to be marked as done.

(If you believe you have received this mail in error, please contact
help-debbugs@gnu.org.)


-- 
51941: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=51941
GNU Bug Tracking System
Contact help-debbugs@gnu.org with problems
--- Begin Message --- Subject: Detect duplication of ERT tests Date: Thu, 18 Nov 2021 11:16:09 +0100
Duplicated `ert-deftest` forms (ie, two or more definitions with the same test 
name) is a serious problem: the older test is silently ignored, which has the 
consequence of the test covering less than intended.

A quick experiment in the Emacs tree immediately reveals 51 instances. There's 
no telling how many lie waiting in external code.

Having explored various options I've come to the conclusion that it needs to be 
a hard error, at run-time, when running non-interactively (see attached diff). 
Rationale:

* We can't just warn at run time because tests tend to spam a lot when run and 
users only care about whether the tests passed and won't even look at the logs 
otherwise.
* We can't just warn at compile time because many tests aren't compiled at all, 
and where they are (as in the Emacs tree) nobody cares much about the warnings, 
because they are "just tests".
* We can't issue a complaint when running interactively because it's natural to 
keep redefining tests during development.

Since `ert-deftest` forms are often generated as a result of macro-expansion, 
passive static textual linting won't do.

The effect of this change will be a visible and easily remedied failure in 
broken test code. I volunteer to fix the first-order errors in Emacs (by 
renaming the clashes); domain specialists may need to help fixing secondary 
failures (failures in previously ignored tests).

Attachment: ert-deftest-redefine-error.diff
Description: Binary data


--- End Message ---
--- Begin Message --- Subject: Re: bug#51941: Detect duplication of ERT tests Date: Thu, 18 Nov 2021 12:54:18 +0100
18 nov. 2021 kl. 11.35 skrev Lars Ingebrigtsen <larsi@gnus.org>:

> Sounds good.

Thank you, all now done. The tests seem to pass after clashes were resolved but 
do tell if I missed something.



--- End Message ---

reply via email to

[Prev in Thread] Current Thread [Next in Thread]