Re: master 262d0c6: Mark some tests as expensive

[Eli Zaretskii]

> From: Daniel Martín <mardani29@yahoo.es>
> Cc: stefan@marxist.se,  michael.albinus@gmx.de,  emacs-devel@gnu.org
> Date: Sat, 12 Sep 2020 16:00:00 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> >
> > Tramp tests need more time because they involve a remote system.
> > Moreover, the time taken by each Tramp test depends on the speed of
> > the connection, which you cannot know in advance.
> 
> If they depend on the connection speed, or if a host is online or
> offline, they also introduce non-determinism that perhaps is not
> apparent now, but can cause problems when the Tramp codebase (and its
> number of tests) scale. For example, if a Tramp test has a 0.1% chance
> of failure because of an unrelated network problem, then if the Tramp
> test suite reaches a point where 10000 tests are run per day, people
> would be investigating 10 test flakes per day. That's a good reason for
> people to lose confidence in the Tramp test suite and ignore failures.

I don't understand what you are arguing here.  AFAIK, Michael already
either marked "unstable" or rewrote any tests that are known to suffer
from non-determinism.  Slow speed doesn't necessarily add
non-determinism, not if Emacs waits for the operation to complete.

So I don't see how this is relevant to the issue at hand, which is the
criteria for marking tests "expensive" and not running them by
default.  What am I missing?