Re: [Texmacs-dev] Roadmap for GUI's and porting TeXmacs

[Joris van der Hoeven]

> For example, I encourage you to read the "overviews"
> entry in http://doc.trolltech.com/3.0/index.html (as I should have
> first done).

Yes, I looked at parts of that and it is indeed well written.

> High Level Event Framework
> 
>   More generally, we need an powerful event model that also deals with
>   timers and all kinds of inputs/outputs. Better, we need a higher
>   level framework on top of that. In fact, I have been amazed how far
>   TeXmacs has been able to go without these basic facilities.  It is
>   nevertheless a major obstacle to further growth.

I do not really understand what you are aiming at here,
since we do have timers and so in TeXmacs...

> Embedding
> 
>   We want TeXmacs to be abble to be embed and to be embedded. 

I agree with this, but this can only be realised in steps.
Embedding TeXmacs is not at all the same thing as embed in TeXmacs.
Also, gluing a new extension language is not the same problem
as providing an interface for plugging in extern layout programs.
This again is not the same as plugging in a Cas and that is yet
different from including an extern widget. So I plan to do these
things one by one.

>   The toolkit(s) we will choose has/have to support that embedding.  I
>   really believed that TeXmacs will achieve fame thru applications
>   that will use it as a widget. Then everyone will be ready to TeXmacs
>   as a tool in its own right.

OK, this may indeed be a design goal when porting TeXmacs to a new GUI.

>   We eventually have to think of a
>   convention for embedding non TeXmacs documnent in TeXmacs documents. 

This already exists; it relies on converters and later, whenever possible,
on rewriters.

>   I am speaking here of the external format of a document:
>   as it is saved to disk.  Also, we must be modal: many people will
>   only read stuff in a TeXmacs widget: there are many readers for a
>   writer.  We read a lot of documents a write only a few even if the
>   later is more complex.  So, we need a read/only browsing mode.

Well, that is not a big deal, even though we have to think a bit
about the appropriate markup and a safe way to enforce read/only text.

> Limited Resources
> 
>   We want to avoid multiple toolkit ports if not needed so we really
>   wants a toolkit that buys us a side effetc a portability to other
>   OSs.

No, we do not want to stick to one particular toolkit and
certainly not if this toolkit would be Qt.

> Data Representation and other Extensions Languages
> 
>   As I said, I am biased and have a sincere hate of non infixed
>   languages even if/because I have practicized many of then
>   (PostScript, FORTH...) and want Perl in.  The recent events, the
>   release of Perl 5.8 and the design of the upcoming perl6 and its
>   parrot runtime confort me in that thought.
> 
>   We should be eventually able to support interpretors embedding
>   (single and multiple).
> 
>   So, The Guile start-up code must be easily interpretable in another
>   potential extension language. Reciprocally an extension language
>   is interesting only if one can write easily such an intepretor.
> 
>   We must eventually have clear conventions for the representations of
>   menus and TeXmacs GUI.  The tendancy in current toolkits (Gtk, Qt)
>   is to read a XML representation of the GUI and build it using a
>   widget factory.

We run out of scope here. At the moment we will use Guile,
although we will provide support for other extension languages later.
We must avoid to reconsider everything at the same time.
Also, I actually came to like guile. This is *not* because
I have a mathematical background; in fact I also had a strong
bias for infixed languages. But scheme is so fucking flexible,
uniform and simple...

> Exposing an API
> 
>   Modern toolkit have widgets that support concepts like slots and
>   object properties (like you youselve noted). We must think of the
>   main TeXmacs widget that way.

Yes, the slots are the event handlers as well as an additional
slot for executing a scheme expression.

>   I think the best thing Joris can be do for TeXmacs in the short
>   term, is to work on that.  I know that internals TeXmacs structure
>   may change but an API is better than none. You don't want someone
>   who feel TeXmacs cool and mess directly with the internals to make
>   it his own thing (say for embedding perl :P for example). You want a
>   maximum of people to hook to TeXmacs, you don't want them to mess
>   with it or to make a mess of it or to come with a fork to support
>   their own things.  Think TeXmacs as an object oriented library.

OK, so make up a proposal :^)

>   A trivial example that is very involved when it should not be: I
>   want to support a menu that opens buffer in a new window as well as in
>   the current window. I have to mess with code that pee guile. No way!

I'm sorry, but this is ridiculous; you have too much bias agains guile.
What you want can be done in a few lines:

Add a line

    ("load and clone" (choose-file "Load and clone file" "" 
'load-and-clone-buffer))

to the file menu and declare the function

    (define (load-and-clone-buffer s)
      (load-buffer s)
      (clone-window))

Where is your problem?

>   You must expose an API to your internal structures. They will change
>   as your understanding of the issues will progress (no way around
>   that) and this API will become an adaptater to next one.

On the other hand, we indeed *do* have to specify a cleaner guile API.
This is a quite urgent project for which I did not find time yet...

>      Power users just use (La)TeX and don't quite figure out the
>      benefit of a wisiwig typesetter.  I am amazed at the popular myth
>      initiated by the LateX book: wisiwig interface leads to less
>      strutured documents. In fact, my understanding is that Joris is
>      internally moving out of TeX because it is not structured enough
>      and that intelligent interactivity like structured navigation
>      force the TeXmacs designer to think more structured document
>      formats. Am right or wrong on that one, Joris?

You are completely right. Not only TeX (certainly) and LaTeX (also)
lack structure, but also, they lack semantics. Moreover, the formats
are not even well defined...

>  2/ It is way to difficult to learn 
> 
>   I am not even speaking as a programmer here.
> 
>   In demos, many people are amazed at the facility Joris can built a
>   complex strucuted document with mind and eye boggling maths but they
>   can't reproduce that at home. It is incredibly frustrating.  The
>   layout of the menus is only part of the problem. Wizars/druids will
>   help there. Without a real toolkit, we can't even think to build
>   them.

I think that this is not a matter of lacking tools, but more a lack of
people who are willing to write documentation. My first goal is to
make it easy to write and translate documentation for TeXmacs
in an online way. But this has been significantly delayed,
because I did not get the help that I was hoping for.
Nevertheless, a lot of progress has been made and
the new type of documentation might become default during autumn.

>   TeXmacs will eventually be a widget and a library (the sooner, the
>   better) and it may indirectly find sucess that way.  We rightly are
>   concerned by what external tools can bring to us and at what
>   cost. Licensing problems come to mind.  But Open Source/Free
>   Software is really an ecosystem and must be thought so.

You are repeating yourself...

>   For example, I am amazed how weak are the interactive text widget
>   classes in Qt comparatively to the Tk Text widget for example. In Tk
>   you can tie tags to text parts and bind properties (like font and
>   colors) and event handlers to tags.  Perl programmers are addicted
>   to such features and despite the poor look and perfmonce of TK, they
>   seem not uninterested to move to Qt and Gtk or whatever.

Yes, I can only agree. The really deep and ultimate philosophy behind
TeXmacs and GUI design is that the TeXmacs box and widget classes should
somehow merge together. Then a widget is a text and a text a widget...

>   David is rightly bothered by the Trolltech license.  I am more
>   thinking of what TeXmacs will bring to the Qt echosystem if it is
>   available as a Qt widget. Or as Qt widgets, if we also want to
>   mimick existing APIs like the Tk Text widget. It is OK to have
>   "degenerated" version of the TeXmacs widget if that can lead more
>   people toward TeXmacs.  Also curiously, when most widgets class
>   dealing with text layout are read only/mode (for outputting
>   document, not write them), TeXmacs is lacking a read-only mode to
>   browse documents. Who do the most should be the best as doing the
>   least.

I also have the impression that Qt is best from a technical point of view,
but I could not study wxWindows in depth yet. However, we may have
a serious license problem. What if the GPL just forbids us to use Qt
on non-GNU/Linux platforms?

> The problem is the main widget. The rest, you alreasy almost
> abstracted as an API. I made an experiment to untie the main widget by
> making TeXmacs a main only widget API to gain some understanding. It
> was surprisingly easy.  You have a skewed view of the problem of the
> other widgets because you had to implement and maintain them. Really
> it should be a no brainer.

Yes, but we do need to keep a library of widget construction functions
as you find in widget.hh. This is important, because this provides us
with the flexibility we need to build a GUI directly from the extension
languages. This is only half the case now. Later on, a file like
tm_menus should be move to the Guile part. Also, it should become
easy to implement something like multiple views from Guile.

> Once I had finished cooking two patches for the menus (more on that
> later!), I will investigage the QXtWidget. It seems that we should be
> able to first derive the TeXmacs main widget from it but that still
> ties us to X so it is a short term solution.

I have the impression that you want to completely circumvent
the abstract display layer. Maybe you are right, but maybe not...
In any case, I insist on keeping the possibility to use the TeXmacs
fonts in the menus. This is a feature that many users like (some kind
of trademark). So if you find a way to completely throw away
the current GUI and replace it by one single widget,
please find out how to keep this part...
Another important feature is the dynamic menus.
Actually both things are tied together: you should find a way
to customize (Qt) menu entries in an extreme manner
(i.e. a user paint method and on the fly generation of popup menus).

> The best bet is on widget factories

I do not want widget factories; this is a poormans and old-fashioned way
of designing GUI's. What current widget factories can do, TeXmacs will
be able to do it better, but *not* vice versa. Don't forget: a GUI should
be a text and a text a GUI. Also, we want to create GUI's dynamically,
in a similar way as you may create web pages dynamically.

> that are fed XML (both Qt and Gtk
> do that) We will have to do that dynamically for menus. I don't like
> much this format but this is the norm. It should be easier to adapt
> from one XML format to another than from an API to another.

We have scheme now, so we will stick to that for a while.
Anyway, it is a trivial task to translate scheme expressions
to XML and vice versa.

> Probably the main work is to translate the underling toolkit event
> model to the TeXmacs one. This goes for d), g).

Yes.

> This means that the coordiante space of the main gadget in TeXmacs is
> translated compared to the toolkit that use real windows.

This is not a big deal.

> >    e) Fonts. We currently use almost only our own fonts,
> >       which require clipped pixmap operations and invisible pixmaps.
> >       Later on, we might want to use other fonts.
> 
> On my experiment, I don't even try to interface with the Qt widget API.

You are right not to try :^)
Anyway, this is an issue for later.
For the moment, we should just deal with clipped pixmaps.

> >    f) Character encodings. We have a potentially universal encoding
> >       scheme. 
> 
> I don't understant what you are talking about.

We have an alphabet with an infinite number of characters
(all one byte characters except < and > and any string without
< or > enclosed between < and >).

>         It should be possible to make our scheme compatible
> >       with Unicode, like in Qt. We have to be careful that something
> >       similar can be done for other GUI's.
> 
> Unicode is most important. It is complex. The best tutorial document I
> know about Unicode is
> http://www.netthink.co.uk/downloads/unicode.ps.gz.  It is targetted to
> a Perl crowd but there is no need to know Perl to read it.
>
> BTW if I got it right and if my memory serves me well, the UTF8 encoding
> can encode strings of integer whatever the size of the interger.
> 
> In fact, when I will start my Perl embedding, the string class will be
> derived from the Perl one that supports very well TeXmacs since the
> very recent 5.008. Before that the perl Unicode support was very incomplete.

Unicode will be dealt with; don't bother about this now.
 
> >    h) Graphics. All is done using the SI-based units.
> >       Some changes are to be foreseen in order to make wysiwyg font handling
> >       dpi-independent.
> 
> We want a toolkit that has a PostScript-like graphics model.
> QPainter is nice.

Yes, I agree; maybe we'll get rid of the SI units.

> > 4. I am willing to make changes in the existing GUI if this is really 
> > necessary
> >    or if it leads to a fundamentally better design. However, as a general
> >    strategy, we should first look whether a toolkit can be made compatible
> >    with the existing conventions. Please throroughly argument proposed 
> > changes!
> 
> I think your GUI is an incredible achievement and it made me learn
> patterns, C++ and now guile (no less). but it just gets in the way today.
> There are a lot of documented GUIs around while TeXmacs is really
> a unique beast (albeit difficult to ride).
> 
> I think that the best things you can do to TeXmacs is to expose as
> much of it as an API and make it a well documented library. 

I agree. If we can port everything to a better GUI and
maybe even get rid of the existing one (although the port has
to be really good for me to accept that (in particular TeXmacs fonts
in the menus and dynamic menus)), that would be great.

On the long run, the GUI will remake a new appearance inside
the typesetter anyway and that will be so much better.
In the meantime, we have to learn how to master these issues.

> So am I. But I think this discussion is worth it.

I agree.