[Bibulus-dev] All Torsten's questions answered

[Thomas Widmann]

Hi,

today Torsten Bronger and I had a nice chat on #Bibulus.  Since some
of the questions he asked might interest the rest of you, here's a
digest:


TB: Anyway. I have some very basic questions. How far has the project
    come already?  Is it already usable?

TW: Well, yes and no.
    It is producing usable bibliographies for many (most) entries, but
    there are many small bugs in the code.

TB: Eventually it should be a drop-in replacement for BibTeX, right?

TW: Oh yes.
    One minute, I'll put a DVI file onto my website.

TB: Okay, let's assume I have an old bib file.  This is then
    transformed *once* to XML and I can throw it away?

TW: Correct.
    The conversion will never be perfect, so you should check it and
    throw away the old one.

TB: Now a more difficult question.  How does style come into play?

TW: Please download http://www.twid.bibulus.org/test2.dvi and
    .../test2.tex

    It's a very silly example, but it demonstrates style and several
    other things.

    What I was testing in this file was having multiple
    bibliographies.

    And languages: Try to compare section 7 with section 9.

    But, alas, as you can see in this file, it does not work well.

TB: Okay, but how was it created?  There is the XML database of course
    ... LaTeX generates a wishlist an the aux file, Bibulus makes a
    bbl file.

TW: There is a compatibility mode which does work much better, but in
    that case you don't get any real benefits compared to BibTeX.

TB: So, two questions remain: From where comes the style and with which
    LaTeX package do you communicate?

TW: Did you download the .tex file as well?

    But you were right.  Into the .aux file go things like the following:

    address@hidden@setup {bibliography=foobar,mode=numerical,language=danish,}
    address@hidden@setup {bibliography=foobar,mode=footnotes,language=british,}
    address@hidden@cite 
{key={article-full},bibliography=foobar,type=title,language=danish,section=6,page=3}
    address@hidden@source {bibliography=foobar,filename={xampl}}
    address@hidden@bib {bibliography=foobar,filename=test2.bi4}

    Bibulus then creates at least two files: A .cit file and one or
    more .bi$n$ files (.bi1, .bi2...)

[Dinner break.]

TB: I remember it.  It looks like you have a special Bibulus LaTeX
    style file active in the LaTeX document.

TW: Yes.  What you write in the LaTeX file is for instance the
    following:

    \usepackage{bibulus}
    \bibulussetup{mode=numerical}
    How to program Bib\TeX\ is described in \citet{article-full}.
    \bibulussource{xampl}
    \printbibliography

    If you have several bibliographies active at the same time, write
    the following:

    \bibulussetup{mode=alpha,bibliography=first}
    \bibulussetup{mode=numerical,bibliography=second}
    \bibulussource[first]{xampl}
    \bibulussource[second]{xampl}
    \printbibliography[first]
    \printbibliography[second]

    All of this produces stuff in the aux file like the following:

    address@hidden@setup {bibliography=foobar,mode=footnotes,language=british,}
    address@hidden@source {bibliography=foobar,filename={xampl}}
    address@hidden@bib {bibliography=foobar,filename=test2.bi4}

TB: Is bibulus.sty supposed to be universal, so that all flexibility
    is realised via style files?  At the moment we have many LaTeX
    packages for that which is a little bit awkward.

TW: bibulus.sty is rather simple, it's really just an interface to the
    underlying Perl functions

TB: .... which may be enough.

TW: What happens is what instead of producing a single .bbl file, it
    produces one .bi$n$ file per \printbibliography, and one .cit file for
    the who document.

TW: The .cit file has one line per \cite command, e.g.:

    {{manual-full}}{[3]}
    {{article-full}}{Aamport, 1986}
    {{article-full}}{(ibid.)}

    This means that LaTeX and bibulus.sty really don't have to do very
    much.
    .bi1, .bi2, .bi3 etc. just contain the formatted lists of references.

    Does that make sense?

TB: My problem is that people want to use different styles.  With the
    current tools this means differend .bst files *and* different
    \cite syntaxes which is horrible for an automatic system such as
    an XML-->LaTeX converter.  If there was a system where the LaTeX
    file remains untouched and -- apart from necessary re-formulations
    of the human text -- I could change the citation and bibliography
    style with external files (counterparts of bst files) this would
    help a lot.

TW: You can do that here.

TB: And how do the bst files look like?

TW: The \bibulussetup command will just pass its arguments to Bibulus,
    which will eventually know about more or less everything.

    There are no bst files.

    You don't need them.

TB: Perl modules?

TW: $self->style(von => 'partoffamily', nameorder => 'givenfirst',
    titlefont => 'emph', reportfont => 'emph', thesisfont => 'plain',
    articlefont => 'plain', abbr => 'full', );

    The above line is part of what happens at start-up.

TB: ... and then the thing is full of style hooks?

TW: Yes, the source code is horrible. ;-)

    If, for instance, you want the font for article titles to be in
    bold, you just say something like

    \bibulussetup{....., articlefont=bold, ....}

TB: And if you want to drop a field or change order?

TW: At the moment, you cannot really do that from within the LaTeX
    file, but we just have to find a way to do it.

    For instance, to output the note before every other field, you
    need to redefine the function l_note from returning 1800 to
    returning a small number, e.g., 10.

    So one could make a LaTeX command, say, \bibulusorder{note, 10},
    which does exactly that.

    (To drop a field, make it return 0.)

TB: Well, actually the publishers should provide us with their styles,
    and I'd like to see their styles to be external.  For example,
    your publisher provides you with a Perl module that takes most of
    the work from you.

TW: Yes, that is one possibility.  They could also supply a .sty file
    that just contains one (long) \bibulussetup command.

TB: Okay.  This sound much more homogeneous than the current mess.
    Apart from all the bugs and shortcomings in BibTeX.

TW: Yes, I think that part should work.

TB: What is your current biggest issue with Bibulus?

TW: There are several.

    1) Defining all the formatting options (I'm using Custom-bib as a
    guide).

    2) Defining all the reference (citation) commands.  At the moment,
    numerical ones ([1]) and a few others are nearly working, but many
    haven't been implemented at all.

    3) Finalising the DTD (and implementing all of it).

    4) Making a really nice \cite command.

    5) I'm sure I've forgotten a lot. :-)

    I pretty much know what needs doing for (1) and (2).

TB: Why doesn't 2) include 4)?  Is natbib a guide for 2)?

TW: The existing \cite commands seem not to be powerful enough.

    Feedback on (3) and (4) would be appreciated.

    Did you see one of the last postings I made to the mailing list?

    \citep{For a summary see [lewis1968, 194-207]; for the Nationality
    Law see [davison1963, 262]}

    This was the suggestion I made.  Does it make sense to you?

TB: Well, to be honest I'm a little bit surprised.  You probably know
    what I would expect: The human language fragments outside the command.

TW: Yes, I can understand that.

    However, things can get quite complex.  Can you find a better
    command that will allow you to cite Abel (2000, p. 100), Zorro
    (1900, p. 200)?

    This needs to be one command, since it could be output as [42,
    87].

    For (2), I think I know all of the types (having looked at natbib,
    jurabib and other packages).  The problem is implementation.

TB: Back to 1).  If the underlying system is Perl, isn't the
    formatting arbitrarily flexible?  If the format is not defined via
    LaTeX hooks but external, at least.

    [I'll come back to the \cite later]

TW: Oh yes, of course.  There already is an HTML output module.

    The main problem with other output formats is to find a way to
    cite and include the references.

    But you could for instance make a Word or Staroffice module, I
    presume.

TB: So, you have infinity possibilities with Perl, and in LaTeX an
    interface to a *particular* Perl realisation to make things easier
    to access for pure LaTeX people?

TW: To answer your last question, it's all object-oriented, so the
    LaTeX interface will just override certain functions.

    And then there's a special program, bibulustex, which does the
    parsing of the aux file and so on.

TB: Ah!  Only because I'm curious: How do you parse BibTeX files?  I
    heard that it is all but easy.  There is a package by N. Beebe as
    far as I remember ...

TW: I use BibTeX.
    Yes, I simply wrote the .bib to XML converter in BibTeX.
    ... with some postprocessing in Perl.

TB: Ah, funny.

TW: Yes, but it actually was rather easy.
    :-)

TB: How easy is the Windows port?

TW: I've no idea.

    I've tested it on Mac OS X.

TB: It *could* be compiled to one more or less big EXE file?

TW: I presume it will work fine on Windows, although the installation
    program might fail.

    No, I doubt you could compile it.

    But you would simply install Perl and run it.

    The problem with the installation is that it uses Unix makefiles.

TB: I don't think -- without knowing it -- that Perl is part of
    Windows TeX distributions, but Bibulus should eventually.

TW: Correct.  But Perl is freely available.

    This is the price you pay for relying on other programs.

    Perl is almost always pre-installed on Unix, so Unix users won't
    have any problems.

TB: Yes, but 95% uses Windows, and limited to LaTeX users still say 80%.

TW: I know. :-(

    The problem is that if I had had to write everything from scratch,
    I might never have completed anything. :-(

TB: I just don't want to see something like xindy's fate again: Better
    than makeindex, but no chance because written in CLISP -- freely
    available too -- and nobody [really!] was prepared to install
    that.

TW: Oh, is that the problem with xindy?  I had never realised that!

    But that reminds me: Better sorting also still needs to be
    implemented.

    Perhaps I should also write a small makeindex/xindy replacement in
    Perl, so that people only need to install one program to solve
    most problems.

TB: Yes, very sad.  Then a couple of people wanted to re-implement it,
    one of them in Ruby.  I felt like an exot with my suggestion to
    use C++. ;)

TW: ;-)

TB: As far as I know, you can make EXE files from Perl programs.

TW: Yes, I have heard that, but I never tried it.  I believe, however,
    that they become huge because they need to include much of the
    Perl interpreter.

    I just checked the net: There seems to be a commercial program
    called perl2exe which could be used (if somebody is willing to buy
    the program).

TB: I the *result* is free -- and I assume that -- this is sufficient.

    Hmmm, there seems also to be a version doesn't cost money (it's
    still not free, though): http://www.indigostar.com/perl2exe.htm

TB: About the file sizes: People tend to prefer simplicity, so I think
    a big file is not a big problem.  Be that as it may, the Windows
    port will work somehow I think.

TW: Yes.  I think I won't concentrate on this, but hopefully somebody
    will have time to make a nice packaged version for Windows users
    (preferably as part of MikTeX or the like).

TB: Okay, I have to drive home now.

/Thomas
-- 
Thomas Widmann          Bye-bye to BibTeX: join the Bibulus project now!
address@hidden                                <http://www.bibulus.org>
Glasgow, Scotland, EU     <http://savannah.nongnu.org/projects/bibulus/>