emacs-orgmode
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Orgmode] Re: Testing --- again...


From: Carsten Dominik
Subject: [Orgmode] Re: Testing --- again...
Date: Sat, 2 Oct 2010 10:09:03 +0200

Hi Sebastian,

the lack of a testing suite for Org-mode is really frustrating,
and even more frustrating is that we have had like seven attempts
to start one, and each of these lead to nothing.  So I would
be perfectly happy to give a free hand, write access to the repo
and a full directory in the distribution to implement one.
Once there is a framework, I am sure many people would be
willing to contribute tests.

More comments below.

On Oct 2, 2010, at 5:51 AM, Sebastian Rose wrote:

Hi,


I thought about testing again recently.  This is something, that never
really got started.  For a reason:  there's no framework for testing.

I therefore wrote a very rough proposal,  found  on
http://github.com/SebastianRose/org-test

The idea is, to provide two simple commands:


 *  org-test-test-current-defun
    will search for tests for the defun point is in or behind
    (`beginning-of-defun') and execute them surrounded by

 (let ((select (or selector "^org"))
        (deactivate-mark nil))
   (save-excursion
     (save-match-data


 *  org-test-test-buffer-file
will search for tests for the entire file and execute them the same
    way.

FIrst:  I have *no* clue about testing.

Second, I am surprised that you want to structure it by function. I would have thought that it could be structure by file at the most. And then there will
be tests that involve code from many files.

But I guess


If you use one of these commands, all currently registered ERT tests are
deleted, and files are reloaded (since you're likely to work on the
tests, too).  To repeat the tests without reloading, you will use the
ERT commands like `ert-results-rerun-all-tests', bound to `r' in the ERT
results buffer.



I choose ERT (git clone http://github.com/ohler/ert.git) because that's
likely to go into Emacs core (or elpa.gnu.org).




The idea is to search the directory structure from the current source
file upwards for a directory named "tests/" if it exists. Else ask the
user.  Similar to what `add-change-log-entry' does.

Below that directory, a tree like the source tree exists:

project
  +-- lisp/
  |     +-- a.el
  |     `-- b/
  |         +-- b.el
  |
  `-- tests/
        +-- a.el/
        |     +-- tests.el
        |     `-- a-defun.el
        `-- b/
            +-- b.el/
                  +-- tests.el
                  `-- b-defun.el

If this setup exists, when editing defun-x in lisp/a.el,
`M-x org-test-test-current-defun' will load tests/a.el/defun-x.el
(fallback: tests.el there) and execute all tests with selector
"^a-defun".

Well, OK, this is fine.  But under a.el and b.el there should also be
general tests that are not function dependent, and there should be a place
to put tests that you do not want to assign to a specific file.

We do have a "testing" directory already, you can use that.
I would prefer the tests to be in testing, not in lisp/testing
if possible. I would like to have the lisp directory contain
only code.  If possible.

It would be OK to have a lisp subdirectory in testing,
just as it would be OK to have contrib/lisp in testing
for the contributed packages.

PLEASE, go ahead.  I do not think you have write access
yet on repo - give me your user name and I'll activate you.

- Carsten

`M-x org-test-test-buffer-file' in that same source file will load all
*.el files in tests/a.el/ and execute all ERT tests for selector "^a".


Thus tests for
   org-mode/lisp/org-protocol.el
will be searched in the directory
   org-mode/tests/lisp/org-protocol.el/*.el


Once the basic route of testing is clear, I'd like to "translate" the
existing tests for org-html.el to work with ERT, which will involve
writing more tools (create output buffers, compare output with control
files using ediff etc.).  I know Lennart Borgman has wrote that stuff
for nXhtml already.  I hope we can use his stuff and help here.

The directory org-mode/lisp/tests/ would not need to be part of the
"official" Org mode package.  It could as well be checked out
separately, if "tests" is part of org-mode/lisp/.gitignore (e.g.).


Any thoughts?




 Sebastian




reply via email to

[Prev in Thread] Current Thread [Next in Thread]