Re: [RFC] Combining of Exuberant Ctags and GNU GLOBAL

[Shigio YAMAGUCHI]

Hello all,
I have updated the proposal. Your opinion were reflected on it.
If there is no problem, I would like to post it during this month.
Thank you for your cooperation.

Here is the updating version.

---------------------------------------------------------------------------

Combining of Exuberant Ctags and GNU GLOBAL
                                             by Shigio YAMAGUCHI

0. Introduction

GNU GLOBAL is a source code tagging system. The feature of it is an
ability to treat not only definitions but also references. Although
it can be used independently, it can also be used in combination
with Exuberant Ctags. This proposal is a thing about the method of
combining two tools.


1. Current Status

Now, you can replace the built-in parser of GNU GLOBAL by Exuberant Ctags.
With some setup, gtags invokes ctags with the --format=1 and --filter
option, and read tag records from it.

The concrete setup method is as follows.

        (Installation of GLOBAL-5.8.1 or later)
        $ ./configure --with-exuberant-ctags=/usr/local/bin/ctags
        $ sudo make install

        (Execution of GLOBAL)
        $ export GTAGSCONF=/usr/local/share/gtags/gtags.conf
        $ export GTAGSLABEL=exuberant-ctags
        $ gtags                         # invokes Exuberant Ctags internally
        $ global -x func                # locate tags
        func                5 test.c           func() {
        $ _

That is, GLOBAL already supports 41 programming languages supported
by Exuberant Ctags.  But it cannot generate reference tags, because
Exuberant Ctags does not pick up them. My proposal is to bury this lack.


2. Adding a type string to the cross reference format to treat references.

How about adding --gtags option to Exuberant Ctags?

The --gtags option adds a type string to the cross reference format
of ctags. This enables GLOBAL to treat references.

<a type string>    <cross reference format (level 1 or 2)>
                 ^
                 |
        one or more blanks

Type strings: (explained later)

        "D": definition
        "R": other than definition

Examples
--------

$ ctags -x --format=1 --gtags test.c
D func                5 test.c           func() {

$ ctags -x --format=2 --gtags test.c
D func             function      5 test.c           func() {

We should be able to consider the type string does not belong
to the format.


3. How to use a type string.

Please assume the following source code.

        [test.c]
        +---------------------
1       |sub() {
2       |       ...
3       |       return 1;
4       |}
5       |func() {
6       |       strcpy(...);
7       |       if (sub()) {
8       |               ...
9       |       }
10      |}
        +---------------------

Now, Exuberant Ctags with the --format=1 option generates
the following output.

$ ctags -x --format=1 test.c
func                5 test.c           func() {
sub                 1 test.c           sub() {

Please care about that the following symbols are ignored,
since they are not a definition.

'strcpy' in the 6th line
'sub' in the 7th line

When the --gtags option is specified, Exuberant Ctags may
pick up reference tags with a type string like follows.

$ ctags -x --format=1 --gtags test.c
D func                5 test.c           func() {
R strcpy              6 test.c                  strcpy(...);   <== ADD
D sub                 1 test.c           sub() {
R sub                 7 test.c                  if (sub()) {   <== ADD

Algorithm of the parser may be like this.

        foreach of all tokens {
                if (symbol?)
                        if (definition?)
                                Put a record with "D"
                        else
                                Put a record with "R"          <== ADD
        }

GLOBAL can use the type string for classification of tags.
Rewriting GLOBAL to accept above format, GLOBAL commands
will work like follows.

$ gtags                                                         # make tag files
$ global -xd sub
sub                 1 test.c           sub() {                  # show 
definitions
$ global -xr sub
sub                 7 test.c                  if (sub()) {      # show 
references
$ global -xs strcpy
strcpy              6 test.c                  strcpy(...);      # show other 
symbols
$ _

        [-d] shows definitions
        -r   shows references (defined in other places)
        -s   shows other symbols (not defined anywhere)

Please note that the definitions, references and other symbols are
treated separately.  This means that Exuberant Ctags become a perfect
parser of GLOBAL.


4. For the parsers which don't support reference tags.

The treatment of the parsers which don't generate reference tags is easy.
What is necessary is just to add a "D" string unconditionally at the head
of the records when the --gtags option is specified.

$ ctags -x --format=1 --gtags test.c
D func                5 test.c           func() {
D sub                 1 test.c           sub() {

Though such output don't have "R" records, it is no problem.


5. Another solution

The reference tag support is also implementable by using special kind name
such as "reference" in the level 2 format.

In the following example, a kind name 'reference' is used
instead of a type string "R".

$ ctags -x --format=2 --gtags test.c
func             function      5 test.c           func() {
strcpy           reference     6 test.c                  strcpy(...);         
<== ADD
sub              function      1 test.c           sub() {
sub              reference     7 test.c                  if (sub()) {         
<== ADD

If the kind name includes blanks (like "function reference"), the means
of quote marks is required. From the GLOBAL's viewpoint, the followings
are needed.

* be able to extract a kind name of tag from a tag record.
* be able to decide whether a tag record is a definition or not.


6. Is this useful?

Even if this mechanism is realized, parsers do not necessarily
increase automatically. But it will become an infrastructure for
those who try to write a new parser, I believe.

I think that it is difficult for Exuberant Ctags to realize reference
tag support independently. But it is easy if combined with GLOBAL.


What do you think?

-- End of proposal
---------------------------------------------------------------------------
--
Shigio YAMAGUCHI <address@hidden>
PGP fingerprint: D1CB 0B89 B346 4AB6 5663  C4B6 3CA5 BBB3 57BE DDA3