[bug-idutils] [patch #5358] scanners for Python and R

[Ramon Diaz-Uriarte]

Follow-up Comment #7, patch #5358 (project idutils):

I think I know what is happeing. In both cases, the problem seems to be the
ordering of the output. It is not the individual tokens that differ, but
where they appear in test-file.{R,py}.bad-{xtokid,fid}.

In the file "consistency" (within the /testsuite dir of idutils) I see that
the xtokid output is piped through sort (whereas the fid output seems to be
sorted "internally"). I think this might be the cause of the apparent
problem: the sort is breaking mutliline tokens (and sorting _within_ a
token), whereas fid is doing it "right". 

I think this is happening in test-file.py.bad-{xtokid,fid}. The original
multiline comment

"""A file that does an atomic-rename to move into place.
    This also causes hardlinks to break when it's written out.
    Open this as for a regular file, then use commit() to move into
    place or abort() to cancel.
    An encoding can be specified; otherwise the default is ascii.
    """

is preserved in fid's output. But the sorted xtokid has all these lines
reordered.

So, is it possible that the consitency check, because of the sort of xtokid's
output, is leading us to believe there is a problem (when there is none)?

R.

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/patch/?5358>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/