Re: [Texmacs-dev] Experiences using TeXmacs

[Joris van der Hoeven]

On Fri, Sep 16, 2011 at 12:02:44PM +0200, Jan Lellmann wrote:
> Joris, I noted that you are quite active recently so I'd like to
> bump the issues below. With hindsight, some of the comments may seem
> a bit harsh, for which I apologize. This was partly caused by the
> fact that I see tremendous potential in TeXmacs, and I'm convinced
> that it just needs some basic stability and usability improvements
> to really take off.

No problem, I will quickly try to address your comments.

However, I will soon have to work on other things again.
Neverthless, I will get back on TeXmacs later this academic year,
and some of your comments should be fixed.

As a general rule, I agree with you that we have to do better what
we already do before adding to many new features. We are definitely
working on this, but there are a tremendous number of details to
be dealt with, so this is advancing quite slowly.

> 1a) Deleted references: TeXmacs indefinitely remembers references it
> has once seen. If I create a <label>, reference it using <reference>
> and then delete the <label>, the reference is unchanged (even after
> Document > Update > All, multiple times), while it should display
> "?" instead. This can create very hard-to-spot errors in your
> document. As a workaround, I manually opened the .tm file and
> deleted everything in the <\references> and <\auxiliary> sections,
> which forces TeXmacs to regenerate everything, and searched for "?"
> in the PDF to check if there are any undefined reference. However,
> this should never happen in the first place. Moreover, there should
> be an indicator/warning if there are any undefined references or
> "cite"s in the document.

On the list to be fixed.

> 1b) .tm~ Files: By default, the "File > Load" (BTW, why not "Open"?)
> dialog shows all file types (in fact there is no option to restrict
> it to TeXmacs files, e.g. .tm + .ts). In particular, it shows the
> .tm files side by side with their .tm~ files. It is very easy to
> mis-click the .tm~ file, which in an unfortunate setting may mean
> that you lose all of your work. TeXmacs should either not allow to
> open .tm~ files at all, hide them by default, or at least issue a
> warning when the user tries to open one.

Also on the list. There are other issues: when opening a .tex file,
it should not be possible to save without renaming; only export should work.

> 1c) Page Setup: There are two places where the page size can be set:
> Under File > Page Setup and under Document > Page > Size. At the
> least, the menu should be called "Print setup". Such things sound
> like minor issues, but anyone will FIRST find the prominent "Page
> setup" menu, change the size there, and assume that it did the
> trick, when all that is changed is the print setup. This issue alone
> cost me 15 hours of re-breaking the whole document close to the
> deadline.

On this list; the whole interface to printing should change.

> 1d) Document > Magnification: This is a similarly dangerous item,
> which should at least issue a warning that it changes the FONT SIZE,
> i.e. the amount of text that fits on a page. This is very easy to
> confuse with View > Shrinking factor. If you are required to submit
> an 11pt document and accidentally mess with the "Magnification" you
> are in trouble, since TeXmacs will display 11pt font size, but it
> won't be 11pt in print.

On the list. We want to get rid of the somewhat prehistoric
shrinking factor and replace it by something like zoom.
Nevertheless, we will keep magnification, since it does
a different thing; i.e. useful for slides and so.

> 1e) Consistency: I do not know how exactly, but I managed to get two
> installations of TeXmacs 1.0.7.10 that work exactly the same, except
> that in one copy the *bold* fonts (and only those) are a tiny bit
> wider. Therefore headings (and figures for captions) wrap
> differently. I have no idea if the font was changed recently or if
> the issue is caused by the setup, but in order to be exchangeable
> TeXmacs should make absolutely sure that documents look 100% the
> same when compiled on different systems (maybe use a checksum on the
> font files?). This one took me one day to figure out.

We now ship fonts with TeXmacs; this should reduce risks,
although minor typesetting changes might occur between version
(although I tend to warn users when this happens).

> 1f) Sums & Integrals: There seems to have been a semantic change
> with respect to sums and integrals, which are now considered as
> "functions" (i.e. they have an argument), rather than just a symbol.
> This introduced a nasty behavior: If one creates an integral (say
> <big-around|<int>|<rsub|0><rsub|1>xyz>), puts the cursor at the end
> and presses "backspace" in order to delete the x, the <int> sign at
> the front is deleted instead! This is actually very easy to miss if
> you press "backspace" multiple times, e.g. in order to delete
> multiple characters, and corrupts the formulas. Instead, the last
> character should be deleted.

I reverted back to the old behaviour. The semantics of
big operators is now detected directly by a suitable grammar.

> 1g) PDF Export: Embedded raster images (especially black & white)
> look ugly in the exported PDF. They seem to be compressed as very
> low-quality JPG, with clearly visible JPEG artifacts especially in
> purely black/purely white regions. The only workaround is to export
> as PS and convert to PDF manually using Ghostscript:
> 
> gswin32c -sPAPERSIZE=a4 -sDEVICE=pdfwrite -q -dBATCH -dNOPAUSE
> -dSAFER -dPDFX -dPDFSETTINGS=/prepress -dAutoFilterColorImages=false
> -dColorImageFilter=/FlateEncode -dAutoFilterGrayImages=false
> -dGrayImageFilter=/FlateEncode -sOutputFile=thesis.pdf -c save pop
> -f thesis.ps
> 
> There should be a menu item to change the "PS to PDF" command, and a
> better preset.

We may indeed change the command, but more feedback from other users
is needed in order to test the options you mention.

> 1h) Document > Update: One of the things that annoy me most when
> using LaTeX is that it doesn't automatically
>  perform another run if it is required in order to get all the
> references right. However, LaTeX at least issues a warning. Using
> TeXmacs there is no warning, but the same annoying process of
> Document * click * Update * click * all * wait * Document * click *
> Update * click * etc. about 5 times in a row to be sure that all
> references are up-to-date. In my opinion, TeXmacs should
> automatically perform as many runs as required so that the labels do
> not change anymore.

Yes, also on the list.

> 1i) Breaking of inline math: Inline formulas that are wrapped around
> a line ending are just plain ugly. For this reason LaTeX does not
> wrap them by default, and issues an "overfull hbox" warning instead.
> TeXmacs wraps them quietly, which requires to manually check every
> single line if it contains a silently wrapped inline formula.

Wrapping has slightly improved, but it is still not always perfect.
Nevertheless

1) With M-R, it is easy to group parts of formulas so that they
   become unbreakable; whenever a line-break is bad, then you may want
   to do this kind of thing. Maybe this deserves to be documented better.

2) Long inline formulas are a bad writing style. If you tend to have
   many incorrect line breaks, then you probably should use more
   displayed formulas.

3) At a certain point, we might implement a tool which marks all
   lines with ugly line breaks in the margin. This might be implemented
   together with marginal notes.

> 1k) Style files: I use a custom macro package for shortcuts, pretty
> formatting of algorithms, etc. All documents are in a Subversion
> repository, so it regularly happens (by an "svn update") that a
> style file changes without TeXmacs noticing (TeXmacs is not running
> during the "svn update"). I regularly found TeXmacs marking
> "undefined macros" in the main document, when they were cleary
> defined in the package file. It took me about 2 months to figure out
> that the styles need to be refreshed using "Tools > Update >
> Styles", and that restarting TeXmacs does NOT help. Why this doesn't
> happen automatically when TeXmacs is restarted is completely
> counter-intuitive. However, it may mess up your whole document, or
> make it use old macros, which both is very dangerous.

Yes, we need a better mechanism for cache invalidation.
This is also on the list.

> 2) Annoying things
> ------------------
> 
> 2a) Stylesheet Language: I had to implement some extensions using
> the stylesheet language. If one goes one step beyond the standard
> functionality, TeXmacs crashes at the blink of an eye. I found
> myself spending about 20% of the time figuring out a solution for a
> problem, and then spending 80% of the time trying 4 other solutions
> until I find one that doesn't crash TeXmacs. While LaTeX can also be
> incredibly tiresome to work with, it is solid as a rock. I think if
> TeXmacs aims at production quality and use in professional
> environment, these issues -- which mostly concern power-users --
> desperately need to get fixed. Also, there needs to be a clear
> documentation of the evaluation order of the document tree. I spent
> several hours trying to figure out the exact semantics of the
> various "quote" tags, and eventually gave up.

We probably should make the stylesheet language *less* powerful,
so that users are not invited to use "quote" like tags (which are
indeed somewhat more advanced to manipulate).

We also should make it easier to define really advanced macros
externally using Scheme, rather than mistreating the style language.

> 2b) Images: I found that the only image type that can be reliably
> used in Texmcas is EPS written by Ghostscript. PNG tends to crash
> TeXmacs with a memory overflow when exporting to PS, I found myself
> finally converting all raster images to EPS in order to get it to
> work. EPS files written by other applications (e.g. Illustrator)
> often result in zero-byte .PS files, or .PS files that are truncated
> at the page that contains the EPS. The workaround is to run
> Ghostscript's "eps2eps" on all .EPS files, which creates .EPS files
> that TeXmacs doesn't crash on.

This should be better with the Qt version.

> 2c) Menu Layout: Generally, I found that TeXmacs menus are laid out
> in the way TeXmacs thinks about the document, not in the way that is
> most intuitive for the user. For example, inserting a float is done
> via "Format > Page > Insertion > Floating Object". I think the
> "Format" menu is the last place that anyone would look, in
> particular when there is a top-level "Insert" menu. Other examples:
> Tools > Update menu and Document > Update menu with similar
> functions but at different places, the same holds for Edit >
> Preferences > Keyboard and Edit > Preferences > Mathematics >
> Keyboard.

We are working on this. Many things already changed at the end of
last year, but several items remain on our to do list.

> 2d) Bugs when using parts: There is an ugly bug when working with
> parts that causes TeXmacs to reproducibly crash at harmless actions
> such as selecting text across a chapter boundary (see my previous
> posts on this list).

Yes, there are too many problems with editing large documents.
This is top priority, but I need one or two month of continuous time
in order to get this right.

> 2e) Yes/No Dialog: Is there any reason for the cumbersome "yes/no"
> prompt? Having to backspace away the "no" and type "yes" is very
> VERY annoying. Even a simple "y"/"n" choice with one keypress would
> have been more usable. Similarly for the strange Qt "yes/no" dialog
> with the combobox. This should just use the default yes/no dialog
> that is provided by all OS and by Qt.

You may use arrow up and arrow down.
But true, we might use the standard Qt dialogue for this.

> 2f) Extensible brackets: With the automatic extensible brackets
> there was apparantly introduced a bug: When extensible brackets are
> turned off (Edit > Preferences > Keyboard > Mathematics > Use
> extensible brackets), I am unable to insert extensible brackets
> anymore, even using the toolbox icon ("Insert large delimiter"). On
> the other hand, if I turn extensible brackets on, I cannot type left
> angles anymore ("<" + 5x TAB), it just displays as
> "<left-<langle>-0>". I ended up copy & pasting either the large
> brackets or the angles each time I needed them.

Please check whether this is still the case; I corrected several bugs
which were introduced when moving to the new "matching bracket markup".

> 2g) Editing large documents: TeXmacs should be able to view separate
> parts of a document in different windows. Currently this is not
> possible, which makes creating references in large documents very
> slow: Switch to other part * wait * look up label * Switch back to
> current part * wait * insert label. The same holds for copy &
> pasting. Editing the whole document in one window is too slow.

You may use the old system of project files and includes.
I actually want to switch back to this system.

> 2h) Search: The search function needs a window, which should be
> non-modal. A very common task is to search for all occurrences of a
> text and change something close to that text which cannot be
> automated (for example search for all "TODO" and remove them, fix
> them, or ignore them). The problem is that after typing something,
> the (apparently undocumented) "Search next" shortcut (F3) does not
> work anymore, and one has to type Ctrl+F + the whole search phrase
> again, every single time. The layout for the "Search" dialog box is
> be quite standard by now (at least on Windows systems): "Match
> Case", "Up/Down", "Find Next", "Cancel". The same holds for the
> "Search & Replace" function.

Yes, better search and replace is also on the list.

> 2i) Windows Version: I couldn't get the Windows version to compile,
> I ended up installing TeXmacs under Ubuntu in a VirtualBox VM, and
> using a shared folder to access the .tm files on the Windows drive.

Please retry now.

> 2k) Redraw errors: When I show the whole document (Part > Show all
> parts), I frequently get redraw errors, where text from the current
> page and text from a completely different section at the end of the
> document are draw on top of each other, resulting in garbage text.
> The workaround is to force a redraw e.g. using "page down" + "page
> up".

It would be interesting for me to know which of these artefacts are
due to the Qt version and which ones are also present in the X11 version
(or maybe only in the X11 version).

> 2l) Zooming: Zooming in and out should be seamless e.g. using
> Ctrl+scroll wheel, as is standard in most applications, instead of
> the "View > Shrinking factor" workaround.

Yes, already mentioned above.

> 2m) TeXmacs frequently loses its configuration and displays the
> "Welcome to your new TeXmacs system" screen.

Strange; can you tell me how to reproduce this bug?

> 3a) Editing anything above about 15 pages in "Paper" mode is just
> too slow to be usable. A part of the problem is that TeXmacs updates
> the whole document at every keypress. Since this cannot always be
> avoided, e.g. when re-wording some sentences in order to get a
> prettier line wrap, I ended up typing text fragments in an external
> editor, and copy & pasting them into TeXmacs as a block in order to
> avoid the update at every keypress. If one views the whole document
> ("Parts > All") and accidentally types a complete word (with, say 10
> characters), it is faster to kill TeXmacs, restart it and reload the
> document than having to wait for the updates.

Yes, editing in paper mode is not recommended.

The process is even worse if you use floating objects or
multiple column content.

> 3b) Editing paragraphs that contain floats or captions of figures
> that contain images is incredibly slow, even in very small
> documents.

Yes, floating material deserves a better and faster implementation.

> 3c) The same holds for the "Papyrus" mode when some floats are
> present. The unsatisfying workaround is to insert only figures while
> working with the text, and converting them to floats just before
> creating the final PDF.

It is not about inserting figures, but rather about inserting
floating figures.