[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Orgmode] Re: How you can help

From: Ben Alexander
Subject: Re: [Orgmode] Re: How you can help
Date: Thu, 23 Oct 2008 17:22:22 +0100

On 2008-Oct-23, at 16:49, Richard Riley wrote:

Sebastian Rose <address@hidden> writes:

Bernt Hansen <address@hidden> writes:
Running a minimal emacs should suppress custom config files:
        emacs -q -l yourtest.el

Added this one to the Clippboard section on new org-tests/index.org in
Worg.git. (this section will be temporary...)

Something like the above should only be a link (at most) to the emacs
manual. Reproducing standard info is bad in the long run in case things
in the base product (emacs) change for example.

Some kind of regression testing framework would be awesome. Org- mode is large enough that this is almost a necessity to keep things stable and

It's big and feature-RICH.

The nature of OSS means that the community using the product keep it
stable and bug free. I dont think the efforts to produce meaningful
regression tests would be beneficial in an ever morphing product like
org-mode. Clearly my humble opinion on that one :-;

I'm a bit clue-free on what the words mean. I think of regression tests as ways of defining the way "Things Should Work(TM)" so if someone commits as fix to org-mode that works fine on Debian, but breaks on Windows, there's a way for someone on Windows to easily (brainlessly) run the test and report back to the developer.

I don't think of org-mode as 'ever morphing'. There are some features that are stable, and should remain so. A regression test would be valuable, presumably, for developers working on a new feature (don't screw up important existing functionality!)

I must say I am dubious about this. It means, for the tests to be
meaningful, that the output must be a fixed format in base org. I doubt
this will ever be the case. The presentation will fluctuate while the
core information (dates, schedules periods etc) will remain pretty much

I agree that the presentation is fluid and it was hard for me to imagine how to test it in an automated way. But I disagree (I think) in that having a test on the presentation is extremely powerful.

I'm thinking that one example of how to write such a test could be copied when someone is trying to explain what they want to happen, and how it is not happening right now.

Perhaps such a test would have a very limited useful life. I'd like to think that it could live and be useful for a very long time (that's why thinking about configuration/user customization issues in a test is important). But even if the test is only useful until the bug is fixed (or the feature implemented) , so be it.

The majority of bugs that I see are often down to people misusing or
using things in the base which are not fully explored. No amount of
regression testing can cover things like that unless the regression
tests include everyones customisations.

Not everyone's customizations, just every customization org-mode has. I mean, if the feature is there, it should work. If the feature is important, it should work in the same way on every platform.

The trick -- and I hope it's either solved or easy -- is exactly how to show that "the feature works the same way as it should, right now, on this platform"

If I understand you properly, you're just saying that the user customization creates so much variability in the output, there isn't anyway you could capture all the possible presentations in a single test. Well, yeah, so user customizations need to be left out of the test case.

Except that in the scenario I'm thinking of where someone (a mere mortal using org-mode, with limited lisp knowledge) is trying to report a bug, user customization will be critical. My first thought was asking org-mode to dump all the customization variables it has into a buffer (as buffer-local variables maybe?) would help someone else recreate the bug. (By starting org-mode with NO customizations, except what's found in the test case)

Or are you suggesting that the source of presentation variation is due mostly to OS, Emacs version, or something else beyond the scope of our tests, so they couldn't be replicated on another machine?

Do I think regression testing is important? Yes - in certain
environments. But every time Carsten, you, myself or anyone else fires
up org-mode we are already doing just that.

Inventor: A person who makes an ingenious arrangement of wheels, levers and springs, and believes it civilization. ~Ambrose Bierce, The Devil's Dictionary

reply via email to

[Prev in Thread] Current Thread [Next in Thread]