[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: master 262d0c6: Mark some tests as expensive

From: Daniel Martín
Subject: Re: master 262d0c6: Mark some tests as expensive
Date: Sat, 12 Sep 2020 12:25:17 +0200
User-agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (darwin)

Stefan Kangas <stefan@marxist.se> writes:
> The idea is to avoid that it will take a very long time to run the unit
> tests as the test suite grows.  Arguably a unit test should never take
> longer than a second, and even that is in the slow end.  If we have
> 10.000 unit tests taking a second each, just do the math of how long it
> will take to run (even with parallelization).  We should not postpone
> working on this until we are in that situation, IMHO, because by then it
> will be a major pain.

I agree. We should differentiate between unit tests (they run in under a
second) and end-to-end tests, which are slower but still valuable
because they cover whole scenarios that unit tests can't cover. To keep
things scalable, we should strive to have plenty of fast unit tests and
fewer end-to-end tests. See

> I think the solution for important tests is to refactor the code to make
> them run faster, for example by avoiding I/O or to make timers trigger
> immediately.

Yes, I think this is a good approach to follow in general. Many of the
test flakiness in the Emacs suite happen because many tests depend
implicitly in things outside the module they are testing (like
configuration, OS, etc.). Having more stable tests would help us detect
real regressions more accurately.

> Best regards,
> Stefan Kangas

reply via email to

[Prev in Thread] Current Thread [Next in Thread]