[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[no subject]
|
From: |
Patrice Dumas |
|
Date: |
Sun, 22 Jan 2023 12:07:07 -0500 (EST) |
branch: master
commit 9d0742670d96124eb8f6a0b66b897e70aaaee8ff
Author: Patrice Dumas <pertusus@free.fr>
AuthorDate: Sat Jan 21 15:43:28 2023 +0100
Keep removed elements with source marks and transfer source marks
* tp/Texinfo/ParserNonXS.pm (_add_source_marks)
(_pop_element_from_contents), tp/Texinfo/XS/parsetexi/source_marks.c
(add_source_mark, add_source_marks, register_source_mark),
tp/Texinfo/XS/parsetexi/tree.c (pop_element_from_contents): transfer
source marks from a popped element to the parent element in
pop_element_from_contents(), using a second argument to signal if
source marks should be transferred. Update pop_element_from_contents
callers. Use more _pop_element_from_contents in pure perl parser, and
check better contents key hash not being defined.
* tp/Texinfo/ParserNonXS.pm (_register_source_mark, _next_text),
tp/Texinfo/XS/parsetexi/end_line.c (end_line_misc_line),
tp/Texinfo/XS/parsetexi/input.c (INPUT, set_input_source_mark)
(next_text), tp/Texinfo/XS/parsetexi/source_marks.c
(register_source_mark): do not increment counter for the end of
@include. add a second argument to next_text to pass an element that
can be used to register the end of an @include. Associate @include
start source mark to the input stack corresponding to the @include
file. Update callers.
* tp/Texinfo/ParserNonXS.pm (_end_line_misc_line),
tp/Texinfo/XS/parsetexi/end_line.c (end_line_misc_line),
tp/Texinfo/XS/parsetexi/api.c (store_source_mark_list),
tp/Texinfo/XS/parsetexi/source_marks.c (source_marks_reset_counters),
tp/Texinfo/XS/parsetexi/tree.c (destroy_source_mark): add a source
mark for ignored @setfilename. Associate removed element, @include or
@setfilename, to source mark, to keep source marks that are in those
elements and also for the XS parser to keep source marks associated to
their elements to simplify management of memory.
* tp/Texinfo/XS/parsetexi/api.c (store_source_mark_list): fix
setting up the array of source marks that was mixed up with
type setting.
* tp/Texinfo/Common.pm (modify_tree): recurse in source marks.
---
ChangeLog | 40 ++++
tp/TODO | 23 ++-
tp/Texinfo/Common.pm | 11 ++
tp/Texinfo/ParserNonXS.pm | 218 ++++++++++++++-------
tp/Texinfo/XS/parsetexi/api.c | 18 +-
tp/Texinfo/XS/parsetexi/close.c | 8 +-
tp/Texinfo/XS/parsetexi/def.c | 2 +-
tp/Texinfo/XS/parsetexi/end_line.c | 35 +++-
tp/Texinfo/XS/parsetexi/handle_commands.c | 4 +-
tp/Texinfo/XS/parsetexi/input.c | 25 ++-
tp/Texinfo/XS/parsetexi/input.h | 3 +-
tp/Texinfo/XS/parsetexi/menus.c | 6 +-
tp/Texinfo/XS/parsetexi/multitable.c | 2 +-
tp/Texinfo/XS/parsetexi/parser.c | 29 +--
tp/Texinfo/XS/parsetexi/separator.c | 4 +-
tp/Texinfo/XS/parsetexi/source_marks.c | 56 ++++--
tp/Texinfo/XS/parsetexi/source_marks.h | 1 +
tp/Texinfo/XS/parsetexi/tree.c | 24 ++-
tp/Texinfo/XS/parsetexi/tree.h | 2 +-
tp/Texinfo/XS/parsetexi/tree_types.h | 13 +-
tp/t/results/include/cpp_lines.pl | 37 ++++
tp/t/results/include/double_include.pl | 82 ++++++++
tp/t/results/include/include_comment.pl | 49 +++++
tp/t/results/include/include_space.pl | 37 ++++
tp/t/results/include/include_space_comment.pl | 50 +++++
tp/t/results/include/include_with_setfilename.pl | 71 +++++++
.../include_with_setfilename_in_preformatted.pl | 82 +++++++-
.../macro_and_commands_in_early_commands.pl | 102 ++++++++++
.../results/include/macro_definition_in_include.pl | 35 ++++
tp/t/results/include/macro_in_early_commands.pl | 96 +++++++++
tp/t/results/include/no_eol_in_included_file.pl | 37 ++++
tp/t/results/include/simple.pl | 37 ++++
tp/t/results/include/simple_no_eol.pl | 33 ++++
tp/t/results/include/test_two_include.pl | 74 +++++++
tp/t/results/include/value_expansion_in_include.pl | 111 +++++++++++
tp/t/results/macro/bib_example.pl | 37 ++++
tp/t/results/macro/include_after_empty_line_arg.pl | 37 ++++
tp/t/results/moresectioning/raiselowersections.pl | 37 ++++
38 files changed, 1424 insertions(+), 144 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index e39479f2bb..f04bc6b1f2 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,43 @@
+2023-01-21 Patrice Dumas <pertusus@free.fr>
+
+ Keep removed elements with source marks and transfer source marks
+
+ * tp/Texinfo/ParserNonXS.pm (_add_source_marks)
+ (_pop_element_from_contents), tp/Texinfo/XS/parsetexi/source_marks.c
+ (add_source_mark, add_source_marks, register_source_mark),
+ tp/Texinfo/XS/parsetexi/tree.c (pop_element_from_contents): transfer
+ source marks from a popped element to the parent element in
+ pop_element_from_contents(), using a second argument to signal if
+ source marks should be transferred. Update pop_element_from_contents
+ callers. Use more _pop_element_from_contents in pure perl parser, and
+ check better contents key hash not being defined.
+
+ * tp/Texinfo/ParserNonXS.pm (_register_source_mark, _next_text),
+ tp/Texinfo/XS/parsetexi/end_line.c (end_line_misc_line),
+ tp/Texinfo/XS/parsetexi/input.c (INPUT, set_input_source_mark)
+ (next_text), tp/Texinfo/XS/parsetexi/source_marks.c
+ (register_source_mark): do not increment counter for the end of
+ @include. add a second argument to next_text to pass an element that
+ can be used to register the end of an @include. Associate @include
+ start source mark to the input stack corresponding to the @include
+ file. Update callers.
+
+ * tp/Texinfo/ParserNonXS.pm (_end_line_misc_line),
+ tp/Texinfo/XS/parsetexi/end_line.c (end_line_misc_line),
+ tp/Texinfo/XS/parsetexi/api.c (store_source_mark_list),
+ tp/Texinfo/XS/parsetexi/source_marks.c (source_marks_reset_counters),
+ tp/Texinfo/XS/parsetexi/tree.c (destroy_source_mark): add a source
+ mark for ignored @setfilename. Associate removed element, @include or
+ @setfilename, to source mark, to keep source marks that are in those
+ elements and also for the XS parser to keep source marks associated to
+ their elements to simplify management of memory.
+
+ * tp/Texinfo/XS/parsetexi/api.c (store_source_mark_list): fix
+ setting up the array of source marks that was mixed up with
+ type setting.
+
+ * tp/Texinfo/Common.pm (modify_tree): recurse in source marks.
+
2023-01-22 Gavin Smith <gavinsmith0123@gmail.com>
* NEWS: Add 7.0.2 news.
diff --git a/tp/TODO b/tp/TODO
index 21478b05cc..5034715edc 100644
--- a/tp/TODO
+++ b/tp/TODO
@@ -10,6 +10,15 @@ This is the todo list for texi2any
Before next release
===================
+Add some elements of texinfo that are not of interest in the
+tree, like beginning and end of included file, @ protecting
+end of line in @def*, begin and end of user defined macro
+or value expansion. Add source_marks in the tree.
+
+In a similar way, keep the information about aliases in the tree?
+And about @setfilename in included files which is for now completly
+removed (consistently with documentation).
+
Bugs
====
@@ -77,6 +86,8 @@ Some erroneous constructs not already warned against:
nested footnotes
-- done in XS parser
+ @ref{ccc@b{@ref{ddd}}}
+
@example
@heading A heading
@end example
@@ -180,18 +191,6 @@ Update Unicode-EastAsianWidth. The new version has
improved tests
and a new license, but no functional change. So maybe wait a
bit to update.
-Add some elements of texinfo that are not of interest in the
-tree, like beginning and end of included file, @ protecting
-end of line in @def*, begin and end of user defined macro
-or value expansion. For @include @-commands, which are kept,
-the type is set to 'replaced'. Could add 'sourcemark'
-instead of 'type' for the marks that are not associated with
-@-commands.
-
-In a similar way, keep the information about aliases in the tree?
-And about @setfilename in included files which is for now completly
-removed (consistently with documentation).
-
Right now VERBOSE is almost not used.
Should we warn if output is on STDOUT and OUTPUT_ENCODING_NAME !=
MESSAGE_OUTPUT_ENCODING_NAME?
diff --git a/tp/Texinfo/Common.pm b/tp/Texinfo/Common.pm
index 9a8f1d2f9e..d57aea8401 100644
--- a/tp/Texinfo/Common.pm
+++ b/tp/Texinfo/Common.pm
@@ -1965,6 +1965,17 @@ sub modify_tree($$;$)
splice (@{$tree->{'contents'}}, $i - $#contents -1, 1, @new_contents);
}
}
+ if ($tree->{'source_marks'}) {
+ my @source_marks = @{$tree->{'source_marks'}};
+ for (my $i = 0; $i <= $#source_marks; $i++) {
+ if ($source_marks[$i]->{'element'}) {
+ my @new_element
+ = &$operation('source_mark', $source_marks[$i]->{'element'},
+ $argument);
+ $source_marks[$i]->{'element'} = $new_element[0];
+ }
+ }
+ }
return $tree;
}
diff --git a/tp/Texinfo/ParserNonXS.pm b/tp/Texinfo/ParserNonXS.pm
index 92cceb6468..d06f7e11b2 100644
--- a/tp/Texinfo/ParserNonXS.pm
+++ b/tp/Texinfo/ParserNonXS.pm
@@ -884,7 +884,7 @@ sub parse_texi_text($$;$)
# Returns binary strings too.
sub _input_push_file
{
- my ($self, $input_file_path, $file_name_encoding, $source_mark) = @_;
+ my ($self, $input_file_path, $file_name_encoding) = @_;
my $filehandle = do { local *FH };
if (!open($filehandle, $input_file_path)) {
@@ -918,8 +918,6 @@ sub _input_push_file
$file_input->{'file_name_encoding'} = $file_name_encoding
if (defined($file_name_encoding));
- $file_input->{'file_source_mark'} = $source_mark
- if (defined($source_mark));
$self->{'input'} = [] if (!defined($self->{'input'}));
unshift @{$self->{'input'}}, $file_input;
@@ -1270,18 +1268,15 @@ sub _register_source_mark
{
my ($self, $element, $source_mark) = @_;
- #return;
-
- my $counter_name;
- if ($source_mark->{'sourcemark_type'}) {
- $counter_name = $source_mark->{'sourcemark_type'};
- }
-
- if (!$self->{'source_mark_counters'}->{$counter_name}) {
- $self->{'source_mark_counters'}->{$counter_name} = 0;
+ if (!defined($source_mark->{'counter'})) {
+ my $counter_name = $source_mark->{'sourcemark_type'};
+ if (!$self->{'source_mark_counters'}->{$counter_name}) {
+ $self->{'source_mark_counters'}->{$counter_name} = 0;
+ }
+ $self->{'source_mark_counters'}->{$counter_name} += 1;
+ $source_mark->{'counter'}
+ = $self->{'source_mark_counters'}->{$counter_name};
}
- $self->{'source_mark_counters'}->{$counter_name} += 1;
- $source_mark->{'counter'} = $self->{'source_mark_counters'}->{$counter_name};
# the element that holds the source mark
my $mark_element;
@@ -1310,14 +1305,27 @@ sub _register_source_mark
$source_mark->{'position'} = 0;
}
+ # cannot currently happen
if (!defined($mark_element)) {
return;
}
+ print STDERR "MARKS: $source_mark->{'sourcemark_type'} "
+ ."$source_mark->{'counter'} $source_mark->{'location'} "
+ .(defined($source_mark->{'status'}) ? $source_mark->{'status'}: 'UNDEF')
+ .' '.Texinfo::Common::debug_print_element_short($mark_element)."\n"
+ if ($self->{'DEBUG'});
+ ;
+ _add_source_marks([$source_mark], $mark_element);
+}
- if (!$mark_element->{'source_marks'}) {
- $mark_element->{'source_marks'} = [];
+sub _add_source_marks($$)
+{
+ my $source_marks = shift;
+ my $element = shift;
+ if (!$element->{'source_marks'}) {
+ $element->{'source_marks'} = [];
}
- push @{$mark_element->{'source_marks'}}, $source_mark;
+ push @{$element->{'source_marks'}}, @{$source_marks};
}
# parse a @macro line
@@ -1603,7 +1611,8 @@ sub _end_preformatted($$$;$$)
print STDERR "CLOSE PREFORMATTED\n" if ($self->{'DEBUG'});
# completly remove void preformatted contexts
if (!$current->{'contents'}) {
- my $removed = pop @{$current->{'parent'}->{'contents'}};
+ my $removed
+ = _pop_element_from_contents($current->{'parent'}, 1);
print STDERR "popping $removed->{'type'}\n" if ($self->{'DEBUG'});
}
$current = $current->{'parent'};
@@ -1750,12 +1759,9 @@ sub _gather_previous_item($$;$$)
eq 'c'
or $before_item->{'contents'}->[-1]->{'cmdname'}
eq 'comment')))) {
- my $element = pop @{$before_item->{'contents'}};
+ my $element = _pop_element_from_contents($before_item);
unshift @{$table_term->{'contents'}}, $element;
}
- if (scalar(@{$before_item->{'contents'}}) == 0) {
- delete $before_item->{'contents'};
- }
}
if (scalar(@{$table_after_terms->{'contents'}})) {
push @{$table_entry->{'contents'}}, $table_after_terms;
@@ -1823,7 +1829,8 @@ sub _close_command_cleanup($$) {
# @item or @headitem rows.
if ($current->{'cmdname'} eq 'multitable') {
my $in_head_or_rows;
- my @contents = @{$current->{'contents'}};
+ my @contents;
+ @contents = @{$current->{'contents'}} if ($current->{'contents'});
$current->{'contents'} = [];
foreach my $row (@contents) {
if ($row->{'type'} and $row->{'type'} eq 'row') {
@@ -1865,16 +1872,18 @@ sub _close_command_cleanup($$) {
and $block_commands{$current->{'cmdname'}} eq 'item_line') {
# At this point the end command hasn't been added to the command contents.
# so checks cannot be done at this point.
- if (@{$current->{'contents'}}) {
+ if ($current->{'contents'} and scalar(@{$current->{'contents'}})) {
_gather_previous_item($self, $current);
}
}
+ # Block commands that contain @item's - e.g. @multitable, @table,
+ # @itemize.
# put end out of before_item, and replace it at the end of the parent.
# remove empty before_item.
# warn if not empty before_item, but format is empty
if ($blockitem_commands{$current->{'cmdname'}}) {
- if (@{$current->{'contents'}}) {
+ if ($current->{'contents'} and scalar(@{$current->{'contents'}})) {
my $leading_spaces = 0;
my $before_item;
if ($current->{'contents'}->[0]->{'type'}
@@ -2006,7 +2015,7 @@ sub _close_current($$$;$$)
ord('@'), $current->{'cmdname'}),
$source_info);
if ($block_commands{$current->{'cmdname'}} eq 'conditional') {
- # in this case we are within an ignored conditional
+ # In ignored conditional.
_pop_element_from_contents($current->{'parent'});
}
}
@@ -2035,10 +2044,11 @@ sub _close_current($$$;$$)
}
} elsif ($current->{'type'} eq 'menu_comment'
or $current->{'type'} eq 'menu_entry_description') {
- # close empty menu_comment
+ # Remove empty menu_comment
if ($current->{'type'} eq 'menu_comment'
- and !@{$current->{'contents'}}) {
- pop @{$current->{'parent'}->{'contents'}};
+ and (!$current->{'contents'}
+ or !scalar(@{$current->{'contents'}}))) {
+ _pop_element_from_contents($current->{'parent'}, 1);
}
} elsif ($current->{'parent'}
and $current->{'parent'}->{'type'}
@@ -2246,9 +2256,10 @@ sub _save_line_directive
# returns next text fragment with source information, be it
# pending from a macro expansion or pending text, or read from file.
-sub _next_text($)
+# $CURRENT is the current container that can be used for source marks.
+sub _next_text($;$)
{
- my ($self) = @_;
+ my ($self, $current) = @_;
while (@{$self->{'input'}}) {
my $input = $self->{'input'}->[0];
@@ -2324,6 +2335,14 @@ sub _next_text($)
$file_name, $!));
}
delete $previous_input->{'fh'};
+ if (defined($previous_input->{'file_source_mark'})) {
+ my $end_include_source_mark
+ = { %{$previous_input->{'file_source_mark'}} };
+ $end_include_source_mark->{'status'} = 'end';
+ delete $end_include_source_mark->{'element'};
+ _register_source_mark($self, $current,
+ $end_include_source_mark);
+ }
}
# keep the first input level to have a permanent source for
# source_info, even when nothing is returned and the first input
@@ -2515,12 +2534,48 @@ sub _set_non_ignored_space_in_index_before_command($)
}
}
-sub _pop_element_from_contents($)
+# for debug
+sub _check_popped_source_marks
+{
+ my $popped_element = shift;
+ my $level = shift;
+
+ if ($popped_element->{'source_marks'}) {
+ print STDERR "BUG: level $level element with source_marks\n";
+ }
+ if ($popped_element->{'args'}) {
+ my $arg_nr = 0;
+ foreach my $arg (@{$popped_element->{'args'}}) {
+ _check_popped_source_marks($arg, $level . 'A'.$arg_nr);
+ $arg_nr += 1;
+ }
+ }
+ if ($popped_element->{'contents'}) {
+ my $content_nr = 0;
+ foreach my $content (@{$popped_element->{'contents'}}) {
+ _check_popped_source_marks($content, $level . 'C'.$content_nr);
+ $content_nr += 1;
+ }
+ }
+}
+
+sub _pop_element_from_contents($;$$)
{
my $parent_element = shift;
+ my $reparent_source_marks = shift;
+ my $check_source_marks = shift;
my $popped_element = pop @{$parent_element->{'contents'}};
-
+ #if ($popped_element->{'source_marks'}
+ # and scalar(@{$popped_element->{'source_marks'}})) {
+ # print STDERR "BUG: popping element with source_marks\n";
+ # cluck;
+ #}
+ if ($reparent_source_marks and $popped_element->{'source_marks'}) {
+ _add_source_marks($popped_element->{'source_marks'},
+ $parent_element);
+ }
+ _check_popped_source_marks($popped_element, 'I') if ($check_source_marks);
delete $parent_element->{'contents'}
if (scalar(@{$parent_element->{'contents'}}) == 0);
@@ -2555,7 +2610,7 @@ sub _abort_empty_line {
# remove empty 'empty*before'. Happens in many situations.
if ($spaces_element->{'text'} eq '') {
- _pop_element_from_contents($current);
+ _pop_element_from_contents($current, 1);
} elsif ($spaces_element->{'type'} eq 'empty_line') {
# exactly the same condition as to begin a paragraph
if ((!$current->{'type'} or $type_with_paragraph{$current->{'type'}})
@@ -2568,7 +2623,7 @@ sub _abort_empty_line {
or $spaces_element->{'type'} eq
'internal_spaces_before_argument') {
# Remove element from main tree. It will still be referenced in
# the 'info' hash as 'spaces_before_argument'.
- _pop_element_from_contents($current);
+ _pop_element_from_contents($current, 1);
my $owning_element
= $spaces_element->{'extra'}->{'spaces_associated_command'};
#$owning_element->{'info'} = {} if (! $owning_element->{'info'});
@@ -2660,7 +2715,8 @@ sub _isolate_last_space
if ($current->{'contents'}->[-1]->{'text'} !~ /\S/) {
$current->{'info'}->{'spaces_after_argument'}
= $current->{'contents'}->[-1]->{'text'};
- _pop_element_from_contents($current);
+ # FIXME transfer mark sources text to content?
+ _pop_element_from_contents($current, 1);
} else {
$current->{'contents'}->[-1]->{'text'} =~ s/(\s+)$//;
$current->{'info'}->{'spaces_after_argument'} = $1;
@@ -3130,13 +3186,13 @@ sub _end_line_misc_line($$$)
= Texinfo::Common::locate_include_file($self, $file_path);
if (defined($included_file_path)) {
my ($status, $file_name, $directories)
- = _input_push_file($self, $included_file_path,
$file_name_encoding,
- $include_source_mark);
+ = _input_push_file($self, $included_file_path,
$file_name_encoding);
if ($status) {
$included_file = 1;
print STDERR "Included $included_file_path\n" if
($self->{'DEBUG'});
$include_source_mark = {'sourcemark_type' => $command,
'status' => 'start'};
+ $self->{'input'}->[0]->{'file_source_mark'} = $include_source_mark;
} else {
# FIXME $text does not show the include directory. Using
# $included_file_path would require decoding to character string
@@ -3265,9 +3321,10 @@ sub _end_line_misc_line($$$)
}
}
$current = $current->{'parent'};
- if ($end_command) {
+ if ($end_command) { # Set above
+ # More processing of @end
print STDERR "END COMMAND $end_command\n" if ($self->{'DEBUG'});
- # reparent to block command
+ # Reparent the "@end" element to be a child of the block element.
my $end = _pop_element_from_contents($current);
if ($block_commands{$end_command} ne 'conditional') {
# here close some empty types. Typically empty preformatted
@@ -3277,7 +3334,7 @@ sub _end_line_misc_line($$$)
if (not defined($current->{'cmdname'}) and $current->{'type'}
and !$current->{'contents'}
and $current->{'parent'}) {
- my $removed = pop @{$current->{'parent'}->{'contents'}};
+ my $removed = _pop_element_from_contents($current->{'parent'}, 1);
print STDERR "popping at end command $end_command:
$removed->{'type'}\n"
if ($self->{'DEBUG'});
$current = $current->{'parent'};
@@ -3312,10 +3369,19 @@ sub _end_line_misc_line($$$)
# Ignore @setfilename in included file, as said in the manual.
if ($included_file
or ($command eq 'setfilename' and _in_include($self))) {
- pop @{$current->{'contents'}};
+ my $source_mark;
if ($included_file) {
- _register_source_mark($self, $current, $include_source_mark);
+ $source_mark = $include_source_mark;
+ } else {
+ $source_mark = { 'sourcemark_type' => $command };
}
+ # this is in order to keep source marks that are within a
+ # removed element. For the XS parser it is also easier to
+ # manage the source mark memory which can stay associated
+ # to the element.
+ my $removed_element = _pop_element_from_contents($current);
+ $source_mark->{'element'} = $removed_element;
+ _register_source_mark($self, $current, $source_mark);
}
$current = _begin_preformatted($self, $current)
if ($close_preformatted_commands{$command});
@@ -3730,7 +3796,8 @@ sub _end_line($$$)
and $current->{'contents'}->[-1]->{'type'} eq 'empty_line') {
print STDERR "END EMPTY LINE\n" if ($self->{'DEBUG'});
if ($current->{'type'} and $current->{'type'} eq 'paragraph') {
- my $empty_line = pop @{$current->{'contents'}};
+ # Remove empty_line element.
+ my $empty_line = _pop_element_from_contents($current);
$current = _end_paragraph($self, $current, $source_info);
push @{$current->{'contents'}}, $empty_line;
$empty_line->{'parent'} = $current;
@@ -3738,10 +3805,12 @@ sub _end_line($$$)
and $current->{'type'} eq 'preformatted'
and $current->{'parent'}->{'type'}
and $current->{'parent'}->{'type'} eq 'menu_entry_description') {
- my $empty_line = pop @{$current->{'contents'}};
+ # FIXME transfer source marks
+ my $empty_line = _pop_element_from_contents($current, 0);
my $preformatted = $current;
$current = $current->{'parent'};
- if (! scalar(@{$preformatted->{'contents'}})) {
+ if (not $preformatted->{'contents'} or
+ not scalar(@{$preformatted->{'contents'}})) {
_pop_element_from_contents($current);
}
@@ -3775,9 +3844,9 @@ sub _end_line($$$)
and $current->{'contents'}->[-1]->{'cmdname'}
and ($current->{'contents'}->[-1]->{'cmdname'} eq 'c'
or $current->{'contents'}->[-1]->{'cmdname'} eq 'comment')) {
- $end_comment = pop @{$current->{'contents'}};
+ $end_comment = _pop_element_from_contents($current);
}
- if (!@{$current->{'contents'}}
+ if (not $current->{'contents'} or not scalar(@{$current->{'contents'}})
# empty if only the end of line or spaces, including non ascii
spaces
or (@{$current->{'contents'}} == 1
and defined($current->{'contents'}->[-1]->{'text'})
@@ -3792,9 +3861,10 @@ sub _end_line($$$)
my $menu_type_reopened = 'menu_description';
print STDERR "FINALLY NOT MENU ENTRY\n" if ($self->{'DEBUG'});
my $menu = $current->{'parent'}->{'parent'};
- my $menu_entry = pop @{$menu->{'contents'}};
- if (@{$menu->{'contents'}} and $menu->{'contents'}->[-1]->{'type'}
- and $menu->{'contents'}->[-1]->{'type'} eq 'menu_entry') {
+ my $menu_entry = _pop_element_from_contents($menu);
+ if ($menu->{'contents'} and scalar(@{$menu->{'contents'}})
+ and $menu->{'contents'}->[-1]->{'type'}
+ and $menu->{'contents'}->[-1]->{'type'} eq 'menu_entry') {
my $entry = $menu->{'contents'}->[-1];
my $description;
foreach my $entry_element (reverse(@{$entry->{'args'}})) {
@@ -3813,8 +3883,9 @@ sub _end_line($$$)
'parent' => $entry, };
$description_or_menu_comment = $entry->{'args'}->[-1];
}
- } elsif (@{$menu->{'contents'}} and $menu->{'contents'}->[-1]->{'type'}
- and $menu->{'contents'}->[-1]->{'type'} eq 'menu_comment') {
+ } elsif ($menu->{'contents'} and scalar(@{$menu->{'contents'}})
+ and $menu->{'contents'}->[-1]->{'type'}
+ and $menu->{'contents'}->[-1]->{'type'} eq 'menu_comment') {
$description_or_menu_comment = $menu->{'contents'}->[-1];
$menu_type_reopened = 'menu_comment';
}
@@ -3842,6 +3913,7 @@ sub _end_line($$$)
$current = $current->{'contents'}->[-1];
print STDERR "THEN MENU_COMMENT OPEN\n" if ($self->{'DEBUG'});
}
+ # FIXME check that source marks are transfered
while (@{$menu_entry->{'args'}}) {
my $arg = shift @{$menu_entry->{'args'}};
if (defined($arg->{'text'})) {
@@ -4320,7 +4392,7 @@ sub _process_remaining_on_line($$$$)
$line), $source_info)
if ($line =~ /\S/ and $line !~ /^\s*\@c(omment)?\b/);
$current = $current->{'parent'};
- # don't store ignored @if*
+ # Remove an ignored block @if*
my $conditional = _pop_element_from_contents($current);
if (!defined($conditional->{'cmdname'}
or $conditional->{'cmdname'} ne $end_command)) {
@@ -4399,7 +4471,7 @@ sub _process_remaining_on_line($$$$)
while ($line eq '') {
print STDERR "EMPTY TEXT\n"
if ($self->{'DEBUG'});
- ($line, $source_info) = _next_text($self);
+ ($line, $source_info) = _next_text($self, $current);
if (!defined($line)) {
# end of the file or of a text fragment.
$current = _end_line($self, $current, $source_info);
@@ -4705,13 +4777,11 @@ sub _process_remaining_on_line($$$$)
} else {
print STDERR "MENU ENTRY (certainly)\n" if ($self->{'DEBUG'});
# this is the menu star collected previously
- pop @{$current->{'contents'}};
+ _pop_element_from_contents($current, 1);
$line =~ s/^(\s+)//;
my $leading_text = '*' . $1;
# FIXME remove empty description too? In that case there won't be
# a need to delete preformatted 'contents'
- delete $current->{'contents'}
- if (scalar(@{$current->{'contents'}}) == 0);
if ($current->{'type'} eq 'preformatted'
and $current->{'parent'}->{'type'}
and ($current->{'parent'}->{'type'} eq 'menu_comment'
@@ -4719,9 +4789,13 @@ sub _process_remaining_on_line($$$$)
)) {
my $menu = $current->{'parent'}->{'parent'};
if (!$current->{'contents'}) {
- pop @{$current->{'parent'}->{'contents'}};
- if (!scalar(@{$current->{'parent'}->{'contents'}})) {
- pop @{$menu->{'contents'}};
+ # not sure that it is possible to have source marks, nor
+ # that they will be correctly placed
+ _pop_element_from_contents($current->{'parent'}, 1);
+ if (not $current->{'parent'}
+ or not $current->{'parent'}->{'contents'}
+ or not scalar(@{$current->{'parent'}->{'contents'}})) {
+ _pop_element_from_contents($menu, 1);
}
}
$current = $menu;
@@ -5215,7 +5289,8 @@ sub _process_remaining_on_line($$$$)
}
if ($current->{'cmdname'}
and $current->{'cmdname'} eq $base_command) {
- pop @{$current->{'contents'}};
+ # popped element should be the same as $misc
+ _pop_element_from_contents($current);
_gather_def_item($current, $command);
push @{$current->{'contents'}}, $misc;
}
@@ -5383,10 +5458,11 @@ sub _process_remaining_on_line($$$$)
or $current->{'type'} eq 'menu_entry_description')) {
my $menu;
-
+ # This is for @detailmenu within @menu
$menu = $current->{'parent'};
- pop @{$menu->{'contents'}}
- if (!@{$current->{'contents'}});
+ _pop_element_from_contents($menu, 1)
+ if (not $current->{'contents'}
+ or not scalar(@{$current->{'contents'}}));
if ($menu->{'type'} and $menu->{'type'} eq 'menu_entry') {
$menu = $menu->{'parent'};
@@ -5964,7 +6040,10 @@ sub _process_remaining_on_line($$$$)
}
} else {
my $new_text;
- ($new_text, $source_info) = _next_text($self);
+ ($new_text, $source_info)
+ # TODO a test a situation with @include closing in ignored
+ # @inlinefmtifelse first arg (which maybe is not possible).
+ = _next_text($self, $current->{'args'}->[-1]);
if (not defined($new_text)) {
$retval = $GET_A_NEW_LINE; # error - unbalanced brace
goto funexit;
@@ -6000,7 +6079,10 @@ sub _process_remaining_on_line($$$$)
}
} else {
my $new_text;
- ($new_text, $source_info) = _next_text($self);
+ # TODO a test a situation with @include closing in ignored
+ # @inline* second arg (which maybe is not possible).
+ ($new_text, $source_info)
+ = _next_text($self, $current->{'args'}->[-1]);
if (not defined($new_text)) {
$retval = $GET_A_NEW_LINE; # error - unbalanced brace
goto funexit;
@@ -6092,7 +6174,7 @@ sub _parse_texi($$$)
NEXT_LINE:
while (1) {
my $line;
- ($line, $source_info) = _next_text($self);
+ ($line, $source_info) = _next_text($self, $current);
last if (!defined($line));
if ($self->{'DEBUG'}) {
diff --git a/tp/Texinfo/XS/parsetexi/api.c b/tp/Texinfo/XS/parsetexi/api.c
index 9e69040c78..8342e19406 100644
--- a/tp/Texinfo/XS/parsetexi/api.c
+++ b/tp/Texinfo/XS/parsetexi/api.c
@@ -525,6 +525,7 @@ store_source_mark_list (ELEMENT *e)
IV source_mark_position;
IV source_mark_counter;
source_mark = newHV ();
+#define STORE(key, value) hv_store (source_mark, key, strlen (key), value, 0)
/* A simple integer. The intptr_t cast here prevents
a warning on MinGW ("cast from pointer to integer of
different size"). */
@@ -534,6 +535,13 @@ store_source_mark_list (ELEMENT *e)
source_mark_counter = (IV) (intptr_t) s_mark->counter;
hv_store (source_mark, "counter", strlen ("counter"),
newSViv (source_mark_counter), 0);
+ if (s_mark->element)
+ {
+ ELEMENT *e = s_mark->element;
+ if (!e->hv)
+ element_to_perl_hash (e);
+ STORE("element", newRV_inc ((SV *)e->hv));
+ }
#define SAVE_S_M_STATUS(X) \
case SM_status_ ## X: \
@@ -562,15 +570,17 @@ store_source_mark_list (ELEMENT *e)
#define SAVE_S_M_TYPE(X) \
case SM_type_ ## X: \
sv = newSVpv_utf8 (#X, 0);\
- hv_store (source_mark, "sourcemark_type", \
- strlen ("sourcemark_type"), sv, 0); \
- av_push (av, newRV_inc ((SV *)source_mark));
+ STORE("sourcemark_type", sv); \
+ break;
+
switch (s_mark->type)
{
SAVE_S_M_TYPE (include)
+ SAVE_S_M_TYPE (setfilename)
}
-
+ av_push (av, newRV_inc ((SV *)source_mark));
+#undef STORE
}
}
}
diff --git a/tp/Texinfo/XS/parsetexi/close.c b/tp/Texinfo/XS/parsetexi/close.c
index e4df60b60e..1d4323b328 100644
--- a/tp/Texinfo/XS/parsetexi/close.c
+++ b/tp/Texinfo/XS/parsetexi/close.c
@@ -184,7 +184,7 @@ close_command_cleanup (ELEMENT *current)
if (e && e->cmd == CM_end)
{
add_to_element_contents (current,
- pop_element_from_contents (before_item));
+ pop_element_from_contents (before_item,
0));
}
/* Now if the ET_before_item is empty, remove it. */
@@ -312,12 +312,12 @@ close_current (ELEMENT *current,
line_error ("no matching `@end %s'",
command_name(current->cmd));
- /* Ignored conditional. */
+ /* In ignored conditional. */
if (command_data(current->cmd).data == BLOCK_conditional)
{
parent = current->parent;
destroy_element_and_children (pop_element_from_contents
- (parent));
+ (parent, 0));
}
}
pop_block_command_contexts (cmd);
@@ -357,7 +357,7 @@ close_current (ELEMENT *current,
&& current->contents.number == 0)
{
current = current->parent;
- destroy_element (pop_element_from_contents (current));
+ destroy_element (pop_element_from_contents (current, 1));
}
else
current = current->parent;
diff --git a/tp/Texinfo/XS/parsetexi/def.c b/tp/Texinfo/XS/parsetexi/def.c
index 5fda01cff3..28b2e30aa4 100644
--- a/tp/Texinfo/XS/parsetexi/def.c
+++ b/tp/Texinfo/XS/parsetexi/def.c
@@ -52,7 +52,7 @@ gather_def_item (ELEMENT *current, enum command_id
next_command)
last_child = last_contents_child (current);
if (last_child->type == ET_def_line)
break;
- item_content = pop_element_from_contents (current);
+ item_content = pop_element_from_contents (current, 0);
insert_into_contents (def_item, item_content, 0);
}
diff --git a/tp/Texinfo/XS/parsetexi/end_line.c
b/tp/Texinfo/XS/parsetexi/end_line.c
index d3c4bcd504..aa6bbee2f9 100644
--- a/tp/Texinfo/XS/parsetexi/end_line.c
+++ b/tp/Texinfo/XS/parsetexi/end_line.c
@@ -1483,6 +1483,7 @@ end_line_misc_line (ELEMENT *current)
included_file = 1;
include_source_mark = new_source_mark(SM_type_include);
include_source_mark->status = SM_status_start;
+ set_input_source_mark (include_source_mark);
}
}
}
@@ -1834,7 +1835,7 @@ end_line_misc_line (ELEMENT *current)
debug ("END COMMAND %s", end_command);
/* Reparent the "@end" element to be a child of the block element. */
- end_elt = pop_element_from_contents (current);
+ end_elt = pop_element_from_contents (current, 0);
/* If not a conditional */
if (command_data(end_id).data != BLOCK_conditional)
@@ -1848,7 +1849,7 @@ end_line_misc_line (ELEMENT *current)
&& (current->contents.number == 0) && current->parent)
{
current = current->parent;
- destroy_element (pop_element_from_contents (current));
+ destroy_element (pop_element_from_contents (current, 1));
debug ("popping at end command");
}
/* This closes tree elements (e.g. paragraphs) until we reach
@@ -1889,9 +1890,18 @@ end_line_misc_line (ELEMENT *current)
Also ignore @setfilename in included file, as said in the manual. */
if (included_file || (cmd == CM_setfilename && top_file_index () > 0))
{
- destroy_element_and_children (pop_element_from_contents (current));
+ SOURCE_MARK *source_mark;
if (included_file)
- register_source_mark(current, include_source_mark);
+ source_mark = include_source_mark;
+ else
+ source_mark = new_source_mark(SM_type_setfilename);
+
+ /* this is in order to keep source marks that are within a
+ removed element. For the XS parser it is also easier to
+ manage the source mark memory which can stay associated
+ to the element. */
+ source_mark->element = pop_element_from_contents (current, 0);
+ register_source_mark(current, source_mark);
}
if (close_preformatted_command (cmd))
current = begin_preformatted (current);
@@ -1938,8 +1948,11 @@ end_line_misc_line (ELEMENT *current)
counter_pop (&count_remaining_args);
/* Destroy all contents (TODO: check why do we do this?) */
+ /* FIXME there is no such code in the pure perl parser,
+ and it would be better to remove if not useful, as
+ if useful source mark information is lost */
while (last_contents_child (current))
- destroy_element (pop_element_from_contents (current));
+ destroy_element (pop_element_from_contents (current, 0));
/* Set 'associated_section' extra key for a node. */
if (cmd != CM_node && cmd != CM_part)
@@ -2101,7 +2114,7 @@ end_line (ELEMENT *current)
{
ELEMENT *e;
/* Remove empty_line element. */
- e = pop_element_from_contents (current);
+ e = pop_element_from_contents (current, 0);
current = end_paragraph (current, 0, 0);
@@ -2111,12 +2124,13 @@ end_line (ELEMENT *current)
else if (current->type == ET_preformatted
&& current->parent->type == ET_menu_entry_description)
{
+ /* FIXME transfer source marks */
ELEMENT *empty_line, *e;
- empty_line = pop_element_from_contents (current);
+ empty_line = pop_element_from_contents (current, 0);
if (current->contents.number == 0)
{
current = current->parent;
- destroy_element (pop_element_from_contents (current));
+ destroy_element (pop_element_from_contents (current, 0));
}
else
current = current->parent;
@@ -2156,7 +2170,7 @@ end_line (ELEMENT *current)
if (current->contents.number > 0
&& (last->cmd == CM_c || last->cmd == CM_comment))
{
- end_comment = pop_element_from_contents (current);
+ end_comment = pop_element_from_contents (current, 0);
}
/* If contents empty or is all whitespace. */
@@ -2178,7 +2192,7 @@ end_line (ELEMENT *current)
ELEMENT *menu, *menu_entry, *description_or_menu_comment = 0;
debug ("FINALLY NOT MENU ENTRY");
menu = current->parent->parent;
- menu_entry = pop_element_from_contents (menu);
+ menu_entry = pop_element_from_contents (menu, 0);
if (menu->contents.number > 0
&& last_contents_child(menu)->type == ET_menu_entry)
{
@@ -2240,6 +2254,7 @@ end_line (ELEMENT *current)
debug ("THEN MENU_COMMENT OPEN");
}
{
+ /* FIXME check that source marks are transfered */
int i, j;
for (i = 0; i < menu_entry->args.number; i++)
{
diff --git a/tp/Texinfo/XS/parsetexi/handle_commands.c
b/tp/Texinfo/XS/parsetexi/handle_commands.c
index 52b68fd6b4..b759fe365d 100644
--- a/tp/Texinfo/XS/parsetexi/handle_commands.c
+++ b/tp/Texinfo/XS/parsetexi/handle_commands.c
@@ -601,7 +601,7 @@ handle_line_command (ELEMENT *current, char **line_inout,
if (current->cmd == base_command)
{
- ELEMENT *e = pop_element_from_contents (current);
+ ELEMENT *e = pop_element_from_contents (current, 0);
/* e should be the same as misc */
/* Gather an "inter_def_item" element. */
gather_def_item (current, cmd);
@@ -883,7 +883,7 @@ handle_block_command (ELEMENT *current, char **line_inout,
/* This is for @detailmenu within @menu */
ELEMENT *menu = current->parent;
if (current->contents.number == 0)
- destroy_element (pop_element_from_contents (menu));
+ destroy_element (pop_element_from_contents (menu, 1));
if (menu->type == ET_menu_entry)
menu = menu->parent;
diff --git a/tp/Texinfo/XS/parsetexi/input.c b/tp/Texinfo/XS/parsetexi/input.c
index 454a5adadf..17b4cc22b4 100644
--- a/tp/Texinfo/XS/parsetexi/input.c
+++ b/tp/Texinfo/XS/parsetexi/input.c
@@ -26,6 +26,7 @@
#include "input.h"
#include "text.h"
#include "commands.h"
+#include "source_marks.h"
enum input_type { IN_file, IN_text };
@@ -50,6 +51,7 @@ typedef struct {
into lines. */
char *value_flag; /* value flag if the input text is a @value
explansion */
+ SOURCE_MARK *file_source_mark;
} INPUT;
static char *input_pushback_string;
@@ -115,7 +117,7 @@ new_line (void)
while (1)
{
- new = next_text ();
+ new = next_text (0);
if (!new)
break;
text_append (&t, new);
@@ -387,8 +389,9 @@ input_pushback (char *string)
}
/* Return value to be freed by caller. Return null if we are out of input. */
+/* CURRENT is the current container that can be used for source marks. */
char *
-next_text (void)
+next_text (ELEMENT *current)
{
ssize_t status;
char *line = 0;
@@ -484,6 +487,17 @@ next_text (void)
fprintf (stderr, "error on closing %s: %s",
input_stack[input_number - 1].source_info.file_name,
strerror (errno));
+ if (input_stack[input_number - 1].file_source_mark)
+ {
+ SOURCE_MARK *input_source_mark
+ = input_stack[input_number - 1].file_source_mark;
+ SOURCE_MARK *end_include_source_mark
+ = new_source_mark(input_source_mark->type);
+ end_include_source_mark->counter
+ = input_source_mark->counter;
+ end_include_source_mark->status = SM_status_end;
+ register_source_mark(current, end_include_source_mark);
+ }
}
}
@@ -529,6 +543,12 @@ input_push_text (char *text, int line_number, char *macro,
char *value_flag)
input_number++;
}
+void
+set_input_source_mark (SOURCE_MARK *source_mark)
+{
+ input_stack[input_number - 1].file_source_mark = source_mark;
+}
+
/* For filenames and macro names, it is possible that they won't be referenced
in the line number of any element. It would be too much work to keep
track,
so just keep them all here, and free them all together at the end. */
@@ -706,6 +726,7 @@ input_push_file (char *filename)
input_stack[input_number].source_info.file_name = filename;
input_stack[input_number].source_info.line_nr = 0;
input_stack[input_number].source_info.macro = 0;
+ input_stack[input_number].file_source_mark = 0;
input_stack[input_number].text = 0;
input_stack[input_number].ptext = 0;
input_number++;
diff --git a/tp/Texinfo/XS/parsetexi/input.h b/tp/Texinfo/XS/parsetexi/input.h
index 5fad1ad577..c93cf5dada 100644
--- a/tp/Texinfo/XS/parsetexi/input.h
+++ b/tp/Texinfo/XS/parsetexi/input.h
@@ -5,7 +5,7 @@
#include "tree_types.h"
char *new_line (void);
-char *next_text (void);
+char *next_text (ELEMENT *current);
void save_line_directive (int line_nr, char *filename);
@@ -13,6 +13,7 @@ void input_push_text (char *text, int line_number, char
*macro,
char *value_flag);
int input_push_file (char *filename);
void input_pushback (char *line);
+void set_input_source_mark (SOURCE_MARK *source_mark);
void input_reset_input_stack (void);
int expanding_macro (char *macro);
int top_file_index (void);
diff --git a/tp/Texinfo/XS/parsetexi/menus.c b/tp/Texinfo/XS/parsetexi/menus.c
index 394f2edf5b..28803988e5 100644
--- a/tp/Texinfo/XS/parsetexi/menus.c
+++ b/tp/Texinfo/XS/parsetexi/menus.c
@@ -128,7 +128,7 @@ handle_menu (ELEMENT **current_inout, char **line_inout)
debug ("MENU ENTRY (certainly)");
leading_spaces = strspn (line, whitespace_chars);
- destroy_element (pop_element_from_contents (current));
+ destroy_element (pop_element_from_contents (current, 1));
if (current->type == ET_preformatted
&& current->parent->type == ET_menu_comment)
@@ -138,10 +138,10 @@ handle_menu (ELEMENT **current_inout, char **line_inout)
/* Remove an empty ET_preformatted, and an empty ET_menu_comment. */
if (current->contents.number == 0)
{
- pop_element_from_contents (current->parent);
+ pop_element_from_contents (current->parent, 1);
if (current->parent->contents.number == 0)
{
- pop_element_from_contents (menu);
+ pop_element_from_contents (menu, 1);
destroy_element (current->parent);
}
destroy_element (current);
diff --git a/tp/Texinfo/XS/parsetexi/multitable.c
b/tp/Texinfo/XS/parsetexi/multitable.c
index 70f5f4af66..f907f8baac 100644
--- a/tp/Texinfo/XS/parsetexi/multitable.c
+++ b/tp/Texinfo/XS/parsetexi/multitable.c
@@ -159,7 +159,7 @@ gather_previous_item (ELEMENT *current, enum command_id
next_command)
|| last_contents_child(before_item)->cmd
== CM_comment))
{
- ELEMENT *e = pop_element_from_contents (before_item);
+ ELEMENT *e = pop_element_from_contents (before_item, 0);
insert_into_contents (table_term, e, 0);
}
}
diff --git a/tp/Texinfo/XS/parsetexi/parser.c b/tp/Texinfo/XS/parsetexi/parser.c
index 1aaf3f860c..f659c3b484 100644
--- a/tp/Texinfo/XS/parsetexi/parser.c
+++ b/tp/Texinfo/XS/parsetexi/parser.c
@@ -460,7 +460,7 @@ parse_texi_document (void)
ELEMENT *l;
free (line);
- line = next_text ();
+ line = next_text (0);
if (!line)
break;
@@ -589,7 +589,7 @@ end_preformatted (ELEMENT *current,
if (current->contents.number == 0)
{
current = current->parent;
- destroy_element (pop_element_from_contents (current));
+ destroy_element (pop_element_from_contents (current, 1));
debug ("popping");
}
else
@@ -695,7 +695,7 @@ abort_empty_line (ELEMENT **current_inout, char
*additional_spaces)
/* Remove element altogether if it's empty. */
if (last_child->text.end == 0)
{
- ELEMENT *e = pop_element_from_contents (current);
+ ELEMENT *e = pop_element_from_contents (current, 1);
destroy_element (e);
}
else if (last_child->type == ET_empty_line)
@@ -710,7 +710,7 @@ abort_empty_line (ELEMENT **current_inout, char
*additional_spaces)
the 'info' hash as 'spaces_before_argument'. */
ELEMENT *owning_element;
KEY_PAIR *k;
- ELEMENT *e = pop_element_from_contents (current);
+ ELEMENT *e = pop_element_from_contents (current, 1);
k = lookup_extra (last_child, "spaces_associated_command");
owning_element = (ELEMENT *) k->value;
@@ -743,7 +743,7 @@ isolate_last_space_internal (ELEMENT *current)
{
add_info_string_dup (current, "spaces_after_argument",
last_elt->text.text);
- destroy_element (pop_element_from_contents (current));
+ destroy_element (pop_element_from_contents (current, 1));
}
else
{
@@ -829,7 +829,7 @@ isolate_last_space (ELEMENT *current)
|| last_contents_child(current)->cmd == CM_comment))
{
add_info_element_oot (current, "comment_at_end",
- pop_element_from_contents (current));
+ pop_element_from_contents (current, 0));
}
if (current->contents.number == 0)
@@ -1048,7 +1048,7 @@ check_valid_nesting (ELEMENT *current, enum command_id
cmd)
/* Start by checking if the command is allowed inside a "full text
command" - this is the most permissive. */
/* in the perl parser the checks are not dynamic as in this function,
- a hash is used and modified when defining the definfoencose command */
+ a hash is used and modified when defining the definfoenclose command
*/
/* all the brace commands, not the definfoenclose commands, which
should be consistent with the perl parser */
if (cmd_flags & CF_brace && !(cmd_flags & CF_INFOENCLOSE))
@@ -1362,7 +1362,7 @@ process_remaining_on_line (ELEMENT **current_inout, char
**line_inout)
current = current->parent;
/* Remove an ignored block. */
- popped = pop_element_from_contents (current);
+ popped = pop_element_from_contents (current, 0);
if (popped->cmd != end_cmd)
fatal ("command mismatch for ignored block");
@@ -1512,7 +1512,7 @@ process_remaining_on_line (ELEMENT **current_inout, char
**line_inout)
input in a static variable like allocated_text, to prevent
memory leaks. */
free (allocated_text);
- line = allocated_text = next_text ();
+ line = allocated_text = next_text (current);
if (!line)
{
@@ -1571,7 +1571,7 @@ process_remaining_on_line (ELEMENT **current_inout, char
**line_inout)
line = line_after_command;
current = handle_macro (current, &line, cmd);
free (allocated_line);
- allocated_line = next_text ();
+ allocated_line = next_text (0);
line = allocated_line;
retval = STILL_MORE_TO_PROCESS;
goto funexit;
@@ -1786,8 +1786,11 @@ process_remaining_on_line (ELEMENT **current_inout, char
**line_inout)
line_error ("@dotless expects `i' or `j' as argument, "
"not `%c'", *line);
}
+ /* FIXME not clear that it can happen, there is no need for that
+ in the pure perl parser. If it can happen could be needed
+ to reparent source marks */
while (current->contents.number > 0)
- destroy_element (pop_element_from_contents (current));
+ destroy_element (pop_element_from_contents (current, 0));
line++;
current = current->parent;
}
@@ -2196,7 +2199,7 @@ parse_texi (ELEMENT *root_elt, ELEMENT *current_elt)
while (1)
{
free (allocated_line);
- line = allocated_line = next_text ();
+ line = allocated_line = next_text (current);
if (!allocated_line)
break; /* Out of input. */
@@ -2295,7 +2298,7 @@ finished_totally:
{
ELEMENT *e;
free (allocated_line);
- line = allocated_line = next_text ();
+ line = allocated_line = next_text (0);
if (!allocated_line)
break; /* Out of input. */
diff --git a/tp/Texinfo/XS/parsetexi/separator.c
b/tp/Texinfo/XS/parsetexi/separator.c
index f5f5518157..7402516bbc 100644
--- a/tp/Texinfo/XS/parsetexi/separator.c
+++ b/tp/Texinfo/XS/parsetexi/separator.c
@@ -666,7 +666,7 @@ handle_comma (ELEMENT *current, char **line_inout)
brace_count--;
break;
default:
- line = next_text ();
+ line = next_text (e);
if (!line)
goto funexit;
continue;
@@ -713,7 +713,7 @@ handle_comma (ELEMENT *current, char **line_inout)
break;
default:
free (alloc_line);
- alloc_line = next_text ();
+ alloc_line = next_text (e);
if (!alloc_line)
goto funexit;
line = alloc_line;
diff --git a/tp/Texinfo/XS/parsetexi/source_marks.c
b/tp/Texinfo/XS/parsetexi/source_marks.c
index 0baff07ae1..4d6fcf9cef 100644
--- a/tp/Texinfo/XS/parsetexi/source_marks.c
+++ b/tp/Texinfo/XS/parsetexi/source_marks.c
@@ -20,6 +20,7 @@
#include "errors.h"
int include_counter = 0;
+int setfilename_counter = 0;
SOURCE_MARK *
new_source_mark (enum source_mark_type type)
@@ -31,9 +32,37 @@ new_source_mark (enum source_mark_type type)
source_mark->type = type;
source_mark->status = SM_status_none;
+ source_mark->counter = -1;
return source_mark;
}
+void
+add_source_mark (SOURCE_MARK *source_mark, ELEMENT *e)
+{
+ SOURCE_MARK_LIST *s_mark_list = &(e->source_mark_list);
+ if (s_mark_list->number == s_mark_list->space)
+ {
+ s_mark_list->space++; s_mark_list->space *= 1.5;
+ s_mark_list->list = realloc (s_mark_list->list,
+ s_mark_list->space * sizeof (SOURCE_MARK));
+ if (!s_mark_list->list)
+ fatal ("realloc failed");
+ }
+ s_mark_list->list[s_mark_list->number] = source_mark;
+ s_mark_list->number++;
+}
+
+void
+add_source_marks (SOURCE_MARK_LIST *source_mark_list, ELEMENT *e)
+{
+ if (source_mark_list->number)
+ {
+ int i;
+ for (i = 0; i < source_mark_list->number; i++)
+ add_source_mark (source_mark_list->list[i], e);
+ }
+}
+
/* ELEMENT should be the parent container.
The source mark is put in the last content if it is text
or registered in the parent container. */
@@ -43,10 +72,18 @@ register_source_mark (ELEMENT *e, SOURCE_MARK *source_mark)
ELEMENT *mark_element;
SOURCE_MARK_LIST *s_mark_list;
- if (source_mark->type == SM_type_include)
+ if (source_mark->counter == -1)
{
- include_counter++;
- source_mark->counter = include_counter;
+ if (source_mark->type == SM_type_include)
+ {
+ include_counter++;
+ source_mark->counter = include_counter;
+ }
+ else if (source_mark->type == SM_type_setfilename)
+ {
+ setfilename_counter++;
+ source_mark->counter = setfilename_counter;
+ }
}
if (e->contents.number > 0)
@@ -72,21 +109,12 @@ register_source_mark (ELEMENT *e, SOURCE_MARK *source_mark)
source_mark->position = 0;
}
- s_mark_list = &(mark_element->source_mark_list);
- if (s_mark_list->number == s_mark_list->space)
- {
- s_mark_list->space++; s_mark_list->space *= 1.5;
- s_mark_list->list = realloc (s_mark_list->list,
- s_mark_list->space * sizeof (SOURCE_MARK));
- if (!s_mark_list->list)
- fatal ("realloc failed");
- }
- s_mark_list->list[s_mark_list->number] = source_mark;
- s_mark_list->number++;
+ add_source_mark (source_mark, mark_element);
}
void
source_marks_reset_counters (void)
{
include_counter = 0;
+ setfilename_counter = 0;
}
diff --git a/tp/Texinfo/XS/parsetexi/source_marks.h
b/tp/Texinfo/XS/parsetexi/source_marks.h
index e5d616d241..1ce9ce7d78 100644
--- a/tp/Texinfo/XS/parsetexi/source_marks.h
+++ b/tp/Texinfo/XS/parsetexi/source_marks.h
@@ -21,5 +21,6 @@
SOURCE_MARK *new_source_mark (enum source_mark_type type);
void register_source_mark (ELEMENT *e, SOURCE_MARK *source_mark);
void source_marks_reset_counters (void);
+void add_source_marks (SOURCE_MARK_LIST *source_mark_list, ELEMENT *e);
#endif
diff --git a/tp/Texinfo/XS/parsetexi/tree.c b/tp/Texinfo/XS/parsetexi/tree.c
index 968ec2184b..3dc029fa15 100644
--- a/tp/Texinfo/XS/parsetexi/tree.c
+++ b/tp/Texinfo/XS/parsetexi/tree.c
@@ -19,6 +19,7 @@
#include "errors.h"
#include "tree.h"
+#include "source_marks.h"
//int element_counter;
@@ -56,6 +57,9 @@ new_element (enum element_type type)
e->extra_info = new_associated_info();
e->info_info = new_associated_info();
+ e->source_mark_list.space = 0;
+ e->source_mark_list.number = 0;
+
return e;
}
@@ -160,14 +164,24 @@ destroy_associated_info (ASSOCIATED_INFO *a)
free (a);
}
+void
+destroy_source_mark (SOURCE_MARK *source_mark)
+{
+ if (source_mark->element)
+ destroy_element_and_children (source_mark->element);
+ free (source_mark);
+}
+
void
destroy_source_mark_list (SOURCE_MARK_LIST *source_mark_list)
{
int i;
for (i = 0; i < source_mark_list->number; i++)
- free (source_mark_list->list[i]);
+ destroy_source_mark (source_mark_list->list[i]);
+ source_mark_list->number = 0;
free (source_mark_list->list);
+ source_mark_list->space = 0;
}
void
@@ -359,10 +373,16 @@ pop_element_from_args (ELEMENT *parent)
}
ELEMENT *
-pop_element_from_contents (ELEMENT *parent)
+pop_element_from_contents (ELEMENT *parent, int reparent_source_marks)
{
ELEMENT_LIST *list = &parent->contents;
+ if (reparent_source_marks)
+ {
+ add_source_marks (&(list->list[list->number -1]->source_mark_list),
+ parent);
+ list->list[list->number -1]->source_mark_list.number = 0;
+ }
return list->list[--list->number];
}
diff --git a/tp/Texinfo/XS/parsetexi/tree.h b/tp/Texinfo/XS/parsetexi/tree.h
index 1e58ec933b..fb1e2245fb 100644
--- a/tp/Texinfo/XS/parsetexi/tree.h
+++ b/tp/Texinfo/XS/parsetexi/tree.h
@@ -18,7 +18,7 @@ void remove_slice_from_contents (ELEMENT *parent, int start,
int end);
ELEMENT *last_args_child (ELEMENT *current);
ELEMENT *last_contents_child (ELEMENT *current);
ELEMENT *pop_element_from_args (ELEMENT *parent);
-ELEMENT *pop_element_from_contents (ELEMENT *parent);
+ELEMENT *pop_element_from_contents (ELEMENT *parent, int
reparent_source_marks);
ELEMENT *contents_child_by_index (ELEMENT *e, int index);
ELEMENT *args_child_by_index (ELEMENT *e, int index);
void destroy_element (ELEMENT *e);
diff --git a/tp/Texinfo/XS/parsetexi/tree_types.h
b/tp/Texinfo/XS/parsetexi/tree_types.h
index edac967ca8..4762cae9c5 100644
--- a/tp/Texinfo/XS/parsetexi/tree_types.h
+++ b/tp/Texinfo/XS/parsetexi/tree_types.h
@@ -40,9 +40,14 @@ enum extra_type {
extra_deleted
};
-enum source_mark_type { SM_type_include };
-
-enum source_mark_location { source_mark_location_text,
+/* the *_none enums are not necessarily used, they are there
+ to avoid using 0 for easier to debug code */
+enum source_mark_type { SM_type_none,
+ SM_type_include,
+ SM_type_setfilename };
+
+enum source_mark_location { source_mark_location_none,
+ source_mark_location_text,
source_mark_location_content };
enum source_mark_status {
@@ -82,6 +87,8 @@ typedef struct SOURCE_MARK {
enum source_mark_location location;
size_t position;
int counter;
+ struct ELEMENT *element; /* needed for elements removed
+ from the tree */
} SOURCE_MARK;
typedef struct SOURCE_MARK_LIST {
diff --git a/tp/t/results/include/cpp_lines.pl
b/tp/t/results/include/cpp_lines.pl
index dd4e6086ff..157c94d788 100644
--- a/tp/t/results/include/cpp_lines.pl
+++ b/tp/t/results/include/cpp_lines.pl
@@ -484,6 +484,34 @@ $result_trees{'cpp_lines'} = {
'source_marks' => [
{
'counter' => 1,
+ 'element' => {
+ 'args' => [
+ {
+ 'contents' => [
+ {
+ 'text' => 'file_with_cpp_lines.texi'
+ }
+ ],
+ 'info' => {
+ 'spaces_after_argument' => '
+'
+ },
+ 'type' => 'line_arg'
+ }
+ ],
+ 'cmdname' => 'include',
+ 'extra' => {
+ 'text_arg' => 'file_with_cpp_lines.texi'
+ },
+ 'info' => {
+ 'spaces_before_argument' => ' '
+ },
+ 'source_info' => {
+ 'file_name' => 'g_f',
+ 'line_nr' => 74,
+ 'macro' => ''
+ }
+ },
'location' => 'text',
'position' => 1,
'sourcemark_type' => 'include',
@@ -532,6 +560,15 @@ $result_trees{'cpp_lines'} = {
'type' => 'paragraph'
},
{
+ 'source_marks' => [
+ {
+ 'counter' => 1,
+ 'location' => 'text',
+ 'position' => 1,
+ 'sourcemark_type' => 'include',
+ 'status' => 'end'
+ }
+ ],
'text' => '
',
'type' => 'empty_line'
diff --git a/tp/t/results/include/double_include.pl
b/tp/t/results/include/double_include.pl
index bdf3e52db9..b08531df91 100644
--- a/tp/t/results/include/double_include.pl
+++ b/tp/t/results/include/double_include.pl
@@ -13,6 +13,70 @@ $result_trees{'double_include'} = {
'source_marks' => [
{
'counter' => 2,
+ 'element' => {
+ 'args' => [
+ {
+ 'contents' => [
+ {
+ 'source_marks' => [
+ {
+ 'counter' => 1,
+ 'element' => {
+ 'args' => [
+ {
+ 'contents' => [
+ {
+ 'text' => 'inc_file.texi'
+ }
+ ],
+ 'info' => {
+ 'spaces_after_argument' => '
+'
+ },
+ 'type' => 'line_arg'
+ }
+ ],
+ 'cmdname' => 'include',
+ 'extra' => {
+ 'text_arg' => 'inc_file.texi'
+ },
+ 'info' => {
+ 'spaces_before_argument' => ' '
+ },
+ 'source_info' => {
+ 'file_name' => '',
+ 'line_nr' => 2,
+ 'macro' => ''
+ }
+ },
+ 'location' => 'text',
+ 'position' => 14,
+ 'sourcemark_type' => 'include',
+ 'status' => 'start'
+ }
+ ],
+ 'text' => 'inc_file.texi'
+ }
+ ],
+ 'info' => {
+ 'spaces_after_argument' => ' '
+ },
+ 'type' => 'line_arg'
+ }
+ ],
+ 'cmdname' => 'include',
+ 'extra' => {
+ 'text_arg' => 'inc_file.texi'
+ },
+ 'info' => {
+ 'spaces_before_argument' => ' '
+ },
+ 'source_info' => {
+ 'file_name' => '',
+ 'line_nr' => 2,
+ 'macro' => ''
+ }
+ },
'location' => 'text',
'position' => 1,
'sourcemark_type' => 'include',
@@ -26,10 +90,28 @@ $result_trees{'double_include'} = {
{
'contents' => [
{
+ 'source_marks' => [
+ {
+ 'counter' => 2,
+ 'location' => 'text',
+ 'position' => 18,
+ 'sourcemark_type' => 'include',
+ 'status' => 'end'
+ }
+ ],
'text' => 'In included file.
'
},
{
+ 'source_marks' => [
+ {
+ 'counter' => 1,
+ 'location' => 'text',
+ 'position' => 18,
+ 'sourcemark_type' => 'include',
+ 'status' => 'end'
+ }
+ ],
'text' => 'In included file.
'
}
diff --git a/tp/t/results/include/include_comment.pl
b/tp/t/results/include/include_comment.pl
index fd39260221..de942d810c 100644
--- a/tp/t/results/include/include_comment.pl
+++ b/tp/t/results/include/include_comment.pl
@@ -12,6 +12,15 @@ $result_trees{'include_comment'} = {
{
'contents' => [
{
+ 'source_marks' => [
+ {
+ 'counter' => 1,
+ 'location' => 'text',
+ 'position' => 18,
+ 'sourcemark_type' => 'include',
+ 'status' => 'end'
+ }
+ ],
'text' => 'In included file.
'
}
@@ -22,6 +31,46 @@ $result_trees{'include_comment'} = {
'source_marks' => [
{
'counter' => 1,
+ 'element' => {
+ 'args' => [
+ {
+ 'contents' => [
+ {
+ 'text' => 'inc_file.texi'
+ }
+ ],
+ 'info' => {
+ 'comment_at_end' => {
+ 'args' => [
+ {
+ 'text' => ' comment',
+ 'type' => 'misc_arg'
+ }
+ ],
+ 'cmdname' => 'c',
+ 'extra' => {
+ 'misc_args' => [
+ ' comment'
+ ]
+ }
+ }
+ },
+ 'type' => 'line_arg'
+ }
+ ],
+ 'cmdname' => 'include',
+ 'extra' => {
+ 'text_arg' => 'inc_file.texi'
+ },
+ 'info' => {
+ 'spaces_before_argument' => ' '
+ },
+ 'source_info' => {
+ 'file_name' => '',
+ 'line_nr' => 1,
+ 'macro' => ''
+ }
+ },
'location' => 'content',
'position' => 0,
'sourcemark_type' => 'include',
diff --git a/tp/t/results/include/include_space.pl
b/tp/t/results/include/include_space.pl
index e53bbd68c4..8583055dc4 100644
--- a/tp/t/results/include/include_space.pl
+++ b/tp/t/results/include/include_space.pl
@@ -12,6 +12,15 @@ $result_trees{'include_space'} = {
{
'contents' => [
{
+ 'source_marks' => [
+ {
+ 'counter' => 1,
+ 'location' => 'text',
+ 'position' => 18,
+ 'sourcemark_type' => 'include',
+ 'status' => 'end'
+ }
+ ],
'text' => 'In included file.
'
}
@@ -22,6 +31,34 @@ $result_trees{'include_space'} = {
'source_marks' => [
{
'counter' => 1,
+ 'element' => {
+ 'args' => [
+ {
+ 'contents' => [
+ {
+ 'text' => 'inc_file.texi'
+ }
+ ],
+ 'info' => {
+ 'spaces_after_argument' => '
+'
+ },
+ 'type' => 'line_arg'
+ }
+ ],
+ 'cmdname' => 'include',
+ 'extra' => {
+ 'text_arg' => 'inc_file.texi'
+ },
+ 'info' => {
+ 'spaces_before_argument' => ' '
+ },
+ 'source_info' => {
+ 'file_name' => '',
+ 'line_nr' => 1,
+ 'macro' => ''
+ }
+ },
'location' => 'content',
'position' => 0,
'sourcemark_type' => 'include',
diff --git a/tp/t/results/include/include_space_comment.pl
b/tp/t/results/include/include_space_comment.pl
index 9e56bb3e2b..3c38b2d780 100644
--- a/tp/t/results/include/include_space_comment.pl
+++ b/tp/t/results/include/include_space_comment.pl
@@ -12,6 +12,15 @@ $result_trees{'include_space_comment'} = {
{
'contents' => [
{
+ 'source_marks' => [
+ {
+ 'counter' => 1,
+ 'location' => 'text',
+ 'position' => 18,
+ 'sourcemark_type' => 'include',
+ 'status' => 'end'
+ }
+ ],
'text' => 'In included file.
'
}
@@ -22,6 +31,47 @@ $result_trees{'include_space_comment'} = {
'source_marks' => [
{
'counter' => 1,
+ 'element' => {
+ 'args' => [
+ {
+ 'contents' => [
+ {
+ 'text' => 'inc_file.texi'
+ }
+ ],
+ 'info' => {
+ 'comment_at_end' => {
+ 'args' => [
+ {
+ 'text' => ' comment',
+ 'type' => 'misc_arg'
+ }
+ ],
+ 'cmdname' => 'c',
+ 'extra' => {
+ 'misc_args' => [
+ ' comment'
+ ]
+ }
+ },
+ 'spaces_after_argument' => ' '
+ },
+ 'type' => 'line_arg'
+ }
+ ],
+ 'cmdname' => 'include',
+ 'extra' => {
+ 'text_arg' => 'inc_file.texi'
+ },
+ 'info' => {
+ 'spaces_before_argument' => ' '
+ },
+ 'source_info' => {
+ 'file_name' => '',
+ 'line_nr' => 1,
+ 'macro' => ''
+ }
+ },
'location' => 'content',
'position' => 0,
'sourcemark_type' => 'include',
diff --git a/tp/t/results/include/include_with_setfilename.pl
b/tp/t/results/include/include_with_setfilename.pl
index 49751e0cd6..74f649b74c 100644
--- a/tp/t/results/include/include_with_setfilename.pl
+++ b/tp/t/results/include/include_with_setfilename.pl
@@ -17,6 +17,15 @@ $result_trees{'include_with_setfilename'} = {
{
'contents' => [
{
+ 'source_marks' => [
+ {
+ 'counter' => 1,
+ 'location' => 'text',
+ 'position' => 18,
+ 'sourcemark_type' => 'include',
+ 'status' => 'end'
+ }
+ ],
'text' => 'In included file.
'
}
@@ -55,10 +64,72 @@ $result_trees{'include_with_setfilename'} = {
'source_marks' => [
{
'counter' => 1,
+ 'element' => {
+ 'args' => [
+ {
+ 'contents' => [
+ {
+ 'text' => 'included_file_with_setfilename.texi'
+ }
+ ],
+ 'info' => {
+ 'spaces_after_argument' => '
+'
+ },
+ 'type' => 'line_arg'
+ }
+ ],
+ 'cmdname' => 'include',
+ 'extra' => {
+ 'text_arg' => 'included_file_with_setfilename.texi'
+ },
+ 'info' => {
+ 'spaces_before_argument' => ' '
+ },
+ 'source_info' => {
+ 'file_name' => '',
+ 'line_nr' => 1,
+ 'macro' => ''
+ }
+ },
'location' => 'content',
'position' => 0,
'sourcemark_type' => 'include',
'status' => 'start'
+ },
+ {
+ 'counter' => 1,
+ 'element' => {
+ 'args' => [
+ {
+ 'contents' => [
+ {
+ 'text' => 'included_file.info'
+ }
+ ],
+ 'info' => {
+ 'spaces_after_argument' => '
+'
+ },
+ 'type' => 'line_arg'
+ }
+ ],
+ 'cmdname' => 'setfilename',
+ 'extra' => {
+ 'text_arg' => 'included_file.info'
+ },
+ 'info' => {
+ 'spaces_before_argument' => ' '
+ },
+ 'source_info' => {
+ 'file_name' => 'included_file_with_setfilename.texi',
+ 'line_nr' => 1,
+ 'macro' => ''
+ }
+ },
+ 'location' => 'content',
+ 'position' => 0,
+ 'sourcemark_type' => 'setfilename'
}
],
'type' => 'before_node_section'
diff --git a/tp/t/results/include/include_with_setfilename_in_preformatted.pl
b/tp/t/results/include/include_with_setfilename_in_preformatted.pl
index 9f48243adb..a709f8e7ec 100644
--- a/tp/t/results/include/include_with_setfilename_in_preformatted.pl
+++ b/tp/t/results/include/include_with_setfilename_in_preformatted.pl
@@ -29,6 +29,15 @@ $result_trees{'include_with_setfilename_in_preformatted'} = {
'type' => 'empty_line'
},
{
+ 'source_marks' => [
+ {
+ 'counter' => 1,
+ 'location' => 'text',
+ 'position' => 18,
+ 'sourcemark_type' => 'include',
+ 'status' => 'end'
+ }
+ ],
'text' => 'In included file.
'
},
@@ -68,7 +77,78 @@ $result_trees{'include_with_setfilename_in_preformatted'} = {
'file_name' => '',
'line_nr' => 1,
'macro' => ''
- }
+ },
+ 'source_marks' => [
+ {
+ 'counter' => 1,
+ 'element' => {
+ 'args' => [
+ {
+ 'contents' => [
+ {
+ 'text' => 'included_file_with_setfilename.texi'
+ }
+ ],
+ 'info' => {
+ 'spaces_after_argument' => '
+'
+ },
+ 'type' => 'line_arg'
+ }
+ ],
+ 'cmdname' => 'include',
+ 'extra' => {
+ 'text_arg' => 'included_file_with_setfilename.texi'
+ },
+ 'info' => {
+ 'spaces_before_argument' => ' '
+ },
+ 'source_info' => {
+ 'file_name' => '',
+ 'line_nr' => 2,
+ 'macro' => ''
+ }
+ },
+ 'location' => 'content',
+ 'position' => 0,
+ 'sourcemark_type' => 'include',
+ 'status' => 'start'
+ },
+ {
+ 'counter' => 1,
+ 'element' => {
+ 'args' => [
+ {
+ 'contents' => [
+ {
+ 'text' => 'included_file.info'
+ }
+ ],
+ 'info' => {
+ 'spaces_after_argument' => '
+'
+ },
+ 'type' => 'line_arg'
+ }
+ ],
+ 'cmdname' => 'setfilename',
+ 'extra' => {
+ 'text_arg' => 'included_file.info'
+ },
+ 'info' => {
+ 'spaces_before_argument' => ' '
+ },
+ 'source_info' => {
+ 'file_name' => 'included_file_with_setfilename.texi',
+ 'line_nr' => 1,
+ 'macro' => ''
+ }
+ },
+ 'location' => 'content',
+ 'position' => 0,
+ 'sourcemark_type' => 'setfilename'
+ }
+ ]
}
],
'type' => 'before_node_section'
diff --git a/tp/t/results/include/macro_and_commands_in_early_commands.pl
b/tp/t/results/include/macro_and_commands_in_early_commands.pl
index f12606fb4b..87a478d5af 100644
--- a/tp/t/results/include/macro_and_commands_in_early_commands.pl
+++ b/tp/t/results/include/macro_and_commands_in_early_commands.pl
@@ -326,6 +326,34 @@ $result_trees{'macro_and_commands_in_early_commands'} = {
'source_marks' => [
{
'counter' => 1,
+ 'element' => {
+ 'args' => [
+ {
+ 'contents' => [
+ {
+ 'text' => 'macro_included.texi'
+ }
+ ],
+ 'info' => {
+ 'spaces_after_argument' => '
+'
+ },
+ 'type' => 'line_arg'
+ }
+ ],
+ 'cmdname' => 'include',
+ 'extra' => {
+ 'text_arg' => 'macro_included.texi'
+ },
+ 'info' => {
+ 'spaces_before_argument' => ' '
+ },
+ 'source_info' => {
+ 'file_name' => '',
+ 'line_nr' => 21,
+ 'macro' => ''
+ }
+ },
'location' => 'text',
'position' => 1,
'sourcemark_type' => 'include',
@@ -392,6 +420,15 @@ $result_trees{'macro_and_commands_in_early_commands'} = {
{
'contents' => [
{
+ 'source_marks' => [
+ {
+ 'counter' => 2,
+ 'location' => 'text',
+ 'position' => 18,
+ 'sourcemark_type' => 'include',
+ 'status' => 'end'
+ }
+ ],
'text' => 'In included file.
'
}
@@ -463,8 +500,49 @@ $result_trees{'macro_and_commands_in_early_commands'} = {
'macro' => ''
},
'source_marks' => [
+ {
+ 'counter' => 1,
+ 'location' => 'content',
+ 'position' => 2,
+ 'sourcemark_type' => 'include',
+ 'status' => 'end'
+ },
{
'counter' => 2,
+ 'element' => {
+ 'args' => [
+ {
+ 'contents' => [
+ {
+ 'text' => 'inc_'
+ },
+ {
+ 'cmdname' => '@'
+ },
+ {
+ 'text' => 'f--ile.texi'
+ }
+ ],
+ 'info' => {
+ 'spaces_after_argument' => '
+'
+ },
+ 'type' => 'line_arg'
+ }
+ ],
+ 'cmdname' => 'include',
+ 'extra' => {
+ 'text_arg' => 'inc_@f--ile.texi'
+ },
+ 'info' => {
+ 'spaces_before_argument' => ' '
+ },
+ 'source_info' => {
+ 'file_name' => '',
+ 'line_nr' => 21,
+ 'macro' => 'multiinclude'
+ }
+ },
'location' => 'content',
'position' => 2,
'sourcemark_type' => 'include',
@@ -539,8 +617,20 @@ $result_sectioning{'macro_and_commands_in_early_commands'}
= {
},
'info' => {},
'source_marks' => [
+ {
+ 'counter' => 1,
+ 'location' => 'content',
+ 'position' => 2,
+ 'sourcemark_type' => 'include',
+ 'status' => 'end'
+ },
{
'counter' => 2,
+ 'element' => {
+ 'cmdname' => 'include',
+ 'extra' => {},
+ 'info' => {}
+ },
'location' => 'content',
'position' => 2,
'sourcemark_type' => 'include',
@@ -566,8 +656,20 @@ $result_nodes{'macro_and_commands_in_early_commands'} = {
'extra' => {},
'info' => {},
'source_marks' => [
+ {
+ 'counter' => 1,
+ 'location' => 'content',
+ 'position' => 2,
+ 'sourcemark_type' => 'include',
+ 'status' => 'end'
+ },
{
'counter' => 2,
+ 'element' => {
+ 'cmdname' => 'include',
+ 'extra' => {},
+ 'info' => {}
+ },
'location' => 'content',
'position' => 2,
'sourcemark_type' => 'include',
diff --git a/tp/t/results/include/macro_definition_in_include.pl
b/tp/t/results/include/macro_definition_in_include.pl
index 454ccef1ae..d62b1ced08 100644
--- a/tp/t/results/include/macro_definition_in_include.pl
+++ b/tp/t/results/include/macro_definition_in_include.pl
@@ -84,10 +84,45 @@ $result_trees{'macro_definition_in_include'} = {
'source_marks' => [
{
'counter' => 1,
+ 'element' => {
+ 'args' => [
+ {
+ 'contents' => [
+ {
+ 'text' => 'macro_included.texi'
+ }
+ ],
+ 'info' => {
+ 'spaces_after_argument' => '
+'
+ },
+ 'type' => 'line_arg'
+ }
+ ],
+ 'cmdname' => 'include',
+ 'extra' => {
+ 'text_arg' => 'macro_included.texi'
+ },
+ 'info' => {
+ 'spaces_before_argument' => ' '
+ },
+ 'source_info' => {
+ 'file_name' => '',
+ 'line_nr' => 1,
+ 'macro' => ''
+ }
+ },
'location' => 'content',
'position' => 0,
'sourcemark_type' => 'include',
'status' => 'start'
+ },
+ {
+ 'counter' => 1,
+ 'location' => 'content',
+ 'position' => 1,
+ 'sourcemark_type' => 'include',
+ 'status' => 'end'
}
],
'type' => 'before_node_section'
diff --git a/tp/t/results/include/macro_in_early_commands.pl
b/tp/t/results/include/macro_in_early_commands.pl
index d61ff42564..41eacfc8c9 100644
--- a/tp/t/results/include/macro_in_early_commands.pl
+++ b/tp/t/results/include/macro_in_early_commands.pl
@@ -434,6 +434,34 @@ $result_trees{'macro_in_early_commands'} = {
'source_marks' => [
{
'counter' => 1,
+ 'element' => {
+ 'args' => [
+ {
+ 'contents' => [
+ {
+ 'text' => 'macro_included.texi'
+ }
+ ],
+ 'info' => {
+ 'spaces_after_argument' => '
+'
+ },
+ 'type' => 'line_arg'
+ }
+ ],
+ 'cmdname' => 'include',
+ 'extra' => {
+ 'text_arg' => 'macro_included.texi'
+ },
+ 'info' => {
+ 'spaces_before_argument' => ' '
+ },
+ 'source_info' => {
+ 'file_name' => '',
+ 'line_nr' => 28,
+ 'macro' => ''
+ }
+ },
'location' => 'text',
'position' => 1,
'sourcemark_type' => 'include',
@@ -500,6 +528,15 @@ $result_trees{'macro_in_early_commands'} = {
{
'contents' => [
{
+ 'source_marks' => [
+ {
+ 'counter' => 2,
+ 'location' => 'text',
+ 'position' => 18,
+ 'sourcemark_type' => 'include',
+ 'status' => 'end'
+ }
+ ],
'text' => 'In included file.
'
}
@@ -561,8 +598,43 @@ $result_trees{'macro_in_early_commands'} = {
'macro' => ''
},
'source_marks' => [
+ {
+ 'counter' => 1,
+ 'location' => 'content',
+ 'position' => 2,
+ 'sourcemark_type' => 'include',
+ 'status' => 'end'
+ },
{
'counter' => 2,
+ 'element' => {
+ 'args' => [
+ {
+ 'contents' => [
+ {
+ 'text' => 'inc_file.texi'
+ }
+ ],
+ 'info' => {
+ 'spaces_after_argument' => '
+'
+ },
+ 'type' => 'line_arg'
+ }
+ ],
+ 'cmdname' => 'include',
+ 'extra' => {
+ 'text_arg' => 'inc_file.texi'
+ },
+ 'info' => {
+ 'spaces_before_argument' => ' '
+ },
+ 'source_info' => {
+ 'file_name' => '',
+ 'line_nr' => 28,
+ 'macro' => 'multiinclude'
+ }
+ },
'location' => 'content',
'position' => 2,
'sourcemark_type' => 'include',
@@ -648,8 +720,20 @@ $result_sectioning{'macro_in_early_commands'} = {
},
'info' => {},
'source_marks' => [
+ {
+ 'counter' => 1,
+ 'location' => 'content',
+ 'position' => 2,
+ 'sourcemark_type' => 'include',
+ 'status' => 'end'
+ },
{
'counter' => 2,
+ 'element' => {
+ 'cmdname' => 'include',
+ 'extra' => {},
+ 'info' => {}
+ },
'location' => 'content',
'position' => 2,
'sourcemark_type' => 'include',
@@ -675,8 +759,20 @@ $result_nodes{'macro_in_early_commands'} = {
'extra' => {},
'info' => {},
'source_marks' => [
+ {
+ 'counter' => 1,
+ 'location' => 'content',
+ 'position' => 2,
+ 'sourcemark_type' => 'include',
+ 'status' => 'end'
+ },
{
'counter' => 2,
+ 'element' => {
+ 'cmdname' => 'include',
+ 'extra' => {},
+ 'info' => {}
+ },
'location' => 'content',
'position' => 2,
'sourcemark_type' => 'include',
diff --git a/tp/t/results/include/no_eol_in_included_file.pl
b/tp/t/results/include/no_eol_in_included_file.pl
index cf4bdbbf83..9a41f043fd 100644
--- a/tp/t/results/include/no_eol_in_included_file.pl
+++ b/tp/t/results/include/no_eol_in_included_file.pl
@@ -15,6 +15,34 @@ $result_trees{'no_eol_in_included_file'} = {
'source_marks' => [
{
'counter' => 1,
+ 'element' => {
+ 'args' => [
+ {
+ 'contents' => [
+ {
+ 'text' => 'include_noeol.texi'
+ }
+ ],
+ 'info' => {
+ 'spaces_after_argument' => '
+'
+ },
+ 'type' => 'line_arg'
+ }
+ ],
+ 'cmdname' => 'include',
+ 'extra' => {
+ 'text_arg' => 'include_noeol.texi'
+ },
+ 'info' => {
+ 'spaces_before_argument' => ' '
+ },
+ 'source_info' => {
+ 'file_name' => '',
+ 'line_nr' => 2,
+ 'macro' => ''
+ }
+ },
'location' => 'text',
'position' => 12,
'sourcemark_type' => 'include',
@@ -25,6 +53,15 @@ $result_trees{'no_eol_in_included_file'} = {
'
},
{
+ 'source_marks' => [
+ {
+ 'counter' => 1,
+ 'location' => 'text',
+ 'position' => 3,
+ 'sourcemark_type' => 'include',
+ 'status' => 'end'
+ }
+ ],
'text' => 'AA
'
},
diff --git a/tp/t/results/include/simple.pl b/tp/t/results/include/simple.pl
index a203022fad..c372dc5b85 100644
--- a/tp/t/results/include/simple.pl
+++ b/tp/t/results/include/simple.pl
@@ -12,6 +12,15 @@ $result_trees{'simple'} = {
{
'contents' => [
{
+ 'source_marks' => [
+ {
+ 'counter' => 1,
+ 'location' => 'text',
+ 'position' => 18,
+ 'sourcemark_type' => 'include',
+ 'status' => 'end'
+ }
+ ],
'text' => 'In included file.
'
}
@@ -22,6 +31,34 @@ $result_trees{'simple'} = {
'source_marks' => [
{
'counter' => 1,
+ 'element' => {
+ 'args' => [
+ {
+ 'contents' => [
+ {
+ 'text' => 'inc_file.texi'
+ }
+ ],
+ 'info' => {
+ 'spaces_after_argument' => '
+'
+ },
+ 'type' => 'line_arg'
+ }
+ ],
+ 'cmdname' => 'include',
+ 'extra' => {
+ 'text_arg' => 'inc_file.texi'
+ },
+ 'info' => {
+ 'spaces_before_argument' => ' '
+ },
+ 'source_info' => {
+ 'file_name' => '',
+ 'line_nr' => 1,
+ 'macro' => ''
+ }
+ },
'location' => 'content',
'position' => 0,
'sourcemark_type' => 'include',
diff --git a/tp/t/results/include/simple_no_eol.pl
b/tp/t/results/include/simple_no_eol.pl
index f4815aa7e3..1e14bd2d49 100644
--- a/tp/t/results/include/simple_no_eol.pl
+++ b/tp/t/results/include/simple_no_eol.pl
@@ -12,6 +12,15 @@ $result_trees{'simple_no_eol'} = {
{
'contents' => [
{
+ 'source_marks' => [
+ {
+ 'counter' => 1,
+ 'location' => 'text',
+ 'position' => 18,
+ 'sourcemark_type' => 'include',
+ 'status' => 'end'
+ }
+ ],
'text' => 'In included file.
'
}
@@ -22,6 +31,30 @@ $result_trees{'simple_no_eol'} = {
'source_marks' => [
{
'counter' => 1,
+ 'element' => {
+ 'args' => [
+ {
+ 'contents' => [
+ {
+ 'text' => 'inc_file.texi'
+ }
+ ],
+ 'type' => 'line_arg'
+ }
+ ],
+ 'cmdname' => 'include',
+ 'extra' => {
+ 'text_arg' => 'inc_file.texi'
+ },
+ 'info' => {
+ 'spaces_before_argument' => ' '
+ },
+ 'source_info' => {
+ 'file_name' => '',
+ 'line_nr' => 1,
+ 'macro' => ''
+ }
+ },
'location' => 'content',
'position' => 0,
'sourcemark_type' => 'include',
diff --git a/tp/t/results/include/test_two_include.pl
b/tp/t/results/include/test_two_include.pl
index 4d693762c9..e078b696c9 100644
--- a/tp/t/results/include/test_two_include.pl
+++ b/tp/t/results/include/test_two_include.pl
@@ -15,6 +15,34 @@ $result_trees{'test_two_include'} = {
'source_marks' => [
{
'counter' => 1,
+ 'element' => {
+ 'args' => [
+ {
+ 'contents' => [
+ {
+ 'text' => 'version.texi'
+ }
+ ],
+ 'info' => {
+ 'spaces_after_argument' => '
+'
+ },
+ 'type' => 'line_arg'
+ }
+ ],
+ 'cmdname' => 'include',
+ 'extra' => {
+ 'text_arg' => 'version.texi'
+ },
+ 'info' => {
+ 'spaces_before_argument' => ' '
+ },
+ 'source_info' => {
+ 'file_name' => '',
+ 'line_nr' => 2,
+ 'macro' => ''
+ }
+ },
'location' => 'text',
'position' => 16,
'sourcemark_type' => 'include',
@@ -204,6 +232,34 @@ $result_trees{'test_two_include'} = {
'source_marks' => [
{
'counter' => 2,
+ 'element' => {
+ 'args' => [
+ {
+ 'contents' => [
+ {
+ 'text' => 'inc_file.texi'
+ }
+ ],
+ 'info' => {
+ 'spaces_after_argument' => '
+'
+ },
+ 'type' => 'line_arg'
+ }
+ ],
+ 'cmdname' => 'include',
+ 'extra' => {
+ 'text_arg' => 'inc_file.texi'
+ },
+ 'info' => {
+ 'spaces_before_argument' => ' '
+ },
+ 'source_info' => {
+ 'file_name' => '',
+ 'line_nr' => 5,
+ 'macro' => ''
+ }
+ },
'location' => 'text',
'position' => 17,
'sourcemark_type' => 'include',
@@ -214,6 +270,15 @@ $result_trees{'test_two_include'} = {
'
},
{
+ 'source_marks' => [
+ {
+ 'counter' => 2,
+ 'location' => 'text',
+ 'position' => 18,
+ 'sourcemark_type' => 'include',
+ 'status' => 'end'
+ }
+ ],
'text' => 'In included file.
'
}
@@ -235,6 +300,15 @@ $result_trees{'test_two_include'} = {
'type' => 'paragraph'
}
],
+ 'source_marks' => [
+ {
+ 'counter' => 1,
+ 'location' => 'content',
+ 'position' => 5,
+ 'sourcemark_type' => 'include',
+ 'status' => 'end'
+ }
+ ],
'type' => 'before_node_section'
}
],
diff --git a/tp/t/results/include/value_expansion_in_include.pl
b/tp/t/results/include/value_expansion_in_include.pl
index 4d02174980..3f3c1f68d5 100644
--- a/tp/t/results/include/value_expansion_in_include.pl
+++ b/tp/t/results/include/value_expansion_in_include.pl
@@ -161,6 +161,34 @@ $result_trees{'value_expansion_in_include'} = {
'source_marks' => [
{
'counter' => 1,
+ 'element' => {
+ 'args' => [
+ {
+ 'contents' => [
+ {
+ 'text' => 'incl-incl.txi'
+ }
+ ],
+ 'info' => {
+ 'spaces_after_argument' => '
+'
+ },
+ 'type' => 'line_arg'
+ }
+ ],
+ 'cmdname' => 'include',
+ 'extra' => {
+ 'text_arg' => 'incl-incl.txi'
+ },
+ 'info' => {
+ 'spaces_before_argument' => ' '
+ },
+ 'source_info' => {
+ 'file_name' => '',
+ 'line_nr' => 12,
+ 'macro' => ''
+ }
+ },
'location' => 'text',
'position' => 17,
'sourcemark_type' => 'include',
@@ -188,6 +216,15 @@ $result_trees{'value_expansion_in_include'} = {
}
},
{
+ 'source_marks' => [
+ {
+ 'counter' => 1,
+ 'location' => 'text',
+ 'position' => 36,
+ 'sourcemark_type' => 'include',
+ 'status' => 'end'
+ }
+ ],
'text' => ' file (include-value2.txi) <> ---.
'
}
@@ -252,6 +289,34 @@ $result_trees{'value_expansion_in_include'} = {
'source_marks' => [
{
'counter' => 2,
+ 'element' => {
+ 'args' => [
+ {
+ 'contents' => [
+ {
+ 'text' => 'incl-incl.txi'
+ }
+ ],
+ 'info' => {
+ 'spaces_after_argument' => '
+'
+ },
+ 'type' => 'line_arg'
+ }
+ ],
+ 'cmdname' => 'include',
+ 'extra' => {
+ 'text_arg' => 'incl-incl.txi'
+ },
+ 'info' => {
+ 'spaces_before_argument' => ' '
+ },
+ 'source_info' => {
+ 'file_name' => '',
+ 'line_nr' => 17,
+ 'macro' => ''
+ }
+ },
'location' => 'text',
'position' => 18,
'sourcemark_type' => 'include',
@@ -279,6 +344,15 @@ $result_trees{'value_expansion_in_include'} = {
}
},
{
+ 'source_marks' => [
+ {
+ 'counter' => 2,
+ 'location' => 'text',
+ 'position' => 36,
+ 'sourcemark_type' => 'include',
+ 'status' => 'end'
+ }
+ ],
'text' => ' file (include-value2.txi) <> ---.
'
}
@@ -343,6 +417,34 @@ $result_trees{'value_expansion_in_include'} = {
'source_marks' => [
{
'counter' => 3,
+ 'element' => {
+ 'args' => [
+ {
+ 'contents' => [
+ {
+ 'text' => 'incl-incl.txi'
+ }
+ ],
+ 'info' => {
+ 'spaces_after_argument' => '
+'
+ },
+ 'type' => 'line_arg'
+ }
+ ],
+ 'cmdname' => 'include',
+ 'extra' => {
+ 'text_arg' => 'incl-incl.txi'
+ },
+ 'info' => {
+ 'spaces_before_argument' => ' '
+ },
+ 'source_info' => {
+ 'file_name' => '',
+ 'line_nr' => 22,
+ 'macro' => ''
+ }
+ },
'location' => 'text',
'position' => 18,
'sourcemark_type' => 'include',
@@ -370,6 +472,15 @@ $result_trees{'value_expansion_in_include'} = {
}
},
{
+ 'source_marks' => [
+ {
+ 'counter' => 3,
+ 'location' => 'text',
+ 'position' => 36,
+ 'sourcemark_type' => 'include',
+ 'status' => 'end'
+ }
+ ],
'text' => ' file (include-value2.txi) <> ---.
'
}
diff --git a/tp/t/results/macro/bib_example.pl
b/tp/t/results/macro/bib_example.pl
index 8791f78e2b..fa6c17b595 100644
--- a/tp/t/results/macro/bib_example.pl
+++ b/tp/t/results/macro/bib_example.pl
@@ -331,6 +331,34 @@ $result_trees{'bib_example'} = {
'source_marks' => [
{
'counter' => 1,
+ 'element' => {
+ 'args' => [
+ {
+ 'contents' => [
+ {
+ 'text' => 'my-bib-macros.texi'
+ }
+ ],
+ 'info' => {
+ 'spaces_after_argument' => '
+'
+ },
+ 'type' => 'line_arg'
+ }
+ ],
+ 'cmdname' => 'include',
+ 'extra' => {
+ 'text_arg' => 'my-bib-macros.texi'
+ },
+ 'info' => {
+ 'spaces_before_argument' => ' '
+ },
+ 'source_info' => {
+ 'file_name' => 'bib-example.texi',
+ 'line_nr' => 25,
+ 'macro' => ''
+ }
+ },
'location' => 'text',
'position' => 1,
'sourcemark_type' => 'include',
@@ -3411,6 +3439,15 @@ $result_trees{'bib_example'} = {
'type' => 'preamble_before_content'
}
],
+ 'source_marks' => [
+ {
+ 'counter' => 1,
+ 'location' => 'content',
+ 'position' => 214,
+ 'sourcemark_type' => 'include',
+ 'status' => 'end'
+ }
+ ],
'type' => 'before_node_section'
},
{
diff --git a/tp/t/results/macro/include_after_empty_line_arg.pl
b/tp/t/results/macro/include_after_empty_line_arg.pl
index 6f9a27edfb..c43c6cff36 100644
--- a/tp/t/results/macro/include_after_empty_line_arg.pl
+++ b/tp/t/results/macro/include_after_empty_line_arg.pl
@@ -90,6 +90,34 @@ $result_trees{'include_after_empty_line_arg'} = {
'source_marks' => [
{
'counter' => 1,
+ 'element' => {
+ 'args' => [
+ {
+ 'contents' => [
+ {
+ 'text' => 'inc_file.texi'
+ }
+ ],
+ 'info' => {
+ 'spaces_after_argument' => '
+'
+ },
+ 'type' => 'line_arg'
+ }
+ ],
+ 'cmdname' => 'include',
+ 'extra' => {
+ 'text_arg' => 'inc_file.texi'
+ },
+ 'info' => {
+ 'spaces_before_argument' => ' '
+ },
+ 'source_info' => {
+ 'file_name' => '',
+ 'line_nr' => 6,
+ 'macro' => ''
+ }
+ },
'location' => 'text',
'position' => 1,
'sourcemark_type' => 'include',
@@ -100,6 +128,15 @@ $result_trees{'include_after_empty_line_arg'} = {
'
},
{
+ 'source_marks' => [
+ {
+ 'counter' => 1,
+ 'location' => 'text',
+ 'position' => 18,
+ 'sourcemark_type' => 'include',
+ 'status' => 'end'
+ }
+ ],
'text' => 'In included file.
'
}
diff --git a/tp/t/results/moresectioning/raiselowersections.pl
b/tp/t/results/moresectioning/raiselowersections.pl
index a3f80aeef0..9fcbf44c65 100644
--- a/tp/t/results/moresectioning/raiselowersections.pl
+++ b/tp/t/results/moresectioning/raiselowersections.pl
@@ -454,6 +454,34 @@ $result_trees{'raiselowersections'} = {
'source_marks' => [
{
'counter' => 1,
+ 'element' => {
+ 'args' => [
+ {
+ 'contents' => [
+ {
+ 'text' => 'section_file.texi'
+ }
+ ],
+ 'info' => {
+ 'spaces_after_argument' => '
+'
+ },
+ 'type' => 'line_arg'
+ }
+ ],
+ 'cmdname' => 'include',
+ 'extra' => {
+ 'text_arg' => 'section_file.texi'
+ },
+ 'info' => {
+ 'spaces_before_argument' => ' '
+ },
+ 'source_info' => {
+ 'file_name' => '',
+ 'line_nr' => 24,
+ 'macro' => ''
+ }
+ },
'location' => 'text',
'position' => 1,
'sourcemark_type' => 'include',
@@ -921,6 +949,15 @@ $result_trees{'raiselowersections'} = {
'type' => 'empty_line'
},
{
+ 'source_marks' => [
+ {
+ 'counter' => 1,
+ 'location' => 'text',
+ 'position' => 1,
+ 'sourcemark_type' => 'include',
+ 'status' => 'end'
+ }
+ ],
'text' => '
',
'type' => 'empty_line'