[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
branch master updated: * tp/Texinfo/ParserNonXS.pm (%parser_state_initia
|
From: |
Patrice Dumas |
|
Subject: |
branch master updated: * tp/Texinfo/ParserNonXS.pm (%parser_state_initialization, _next_text) (_process_remaining_on_line): add a value_stack to put expanded @value. Use a sourcemark structure put in source_info to mark the end of @value expansion. Also use such a structure for @macro, replacing 'end_macro'. Check number of values expansion number and error out if there is more than MAX_MACRO_CALL_NESTING. correct MAX_MACRO_CALL_NESTING name in warning message. |
|
Date: |
Thu, 05 Jan 2023 16:55:25 -0500 |
This is an automated email from the git hooks/post-receive script.
pertusus pushed a commit to branch master
in repository texinfo.
The following commit(s) were added to refs/heads/master by this push:
new e3d0afa566 * tp/Texinfo/ParserNonXS.pm (%parser_state_initialization,
_next_text) (_process_remaining_on_line): add a value_stack to put expanded
@value. Use a sourcemark structure put in source_info to mark the end of
@value expansion. Also use such a structure for @macro, replacing 'end_macro'.
Check number of values expansion number and error out if there is more than
MAX_MACRO_CALL_NESTING. correct MAX_MACRO_CALL_NESTING name in warning message.
e3d0afa566 is described below
commit e3d0afa5660abe191f8d7b76ed27ba14c3583703
Author: Patrice Dumas <pertusus@free.fr>
AuthorDate: Thu Jan 5 22:55:02 2023 +0100
* tp/Texinfo/ParserNonXS.pm (%parser_state_initialization, _next_text)
(_process_remaining_on_line): add a value_stack to put expanded
@value. Use a sourcemark structure put in source_info to mark
the end of @value expansion. Also use such a structure for
@macro, replacing 'end_macro'.
Check number of values expansion number and error out if
there is more than MAX_MACRO_CALL_NESTING.
correct MAX_MACRO_CALL_NESTING name in warning message.
* tp/Texinfo/XS/parsetexi/Parsetexi.pm (parser),
tp/Texinfo/XS/parsetexi/Parsetexi.xs
(conf_set_MAX_MACRO_CALL_NESTING), tp/Texinfo/XS/parsetexi/conf.c
(conf_set_MAX_MACRO_CALL_NESTING, reset_conf),
tp/Texinfo/XS/parsetexi/macro.c (handle_macro): pass
MAX_MACRO_CALL_NESTING to the XS parser. Use that value for the
number of macro call nesting allowed.
correct MAX_MACRO_CALL_NESTING name in warning message.
* tp/t/60macro.t: add recursive_call_in_rmacro to test for an infinite
recursion.
---
ChangeLog | 23 +++++
tp/TODO | 5 +-
tp/Texinfo/ParserNonXS.pm | 51 ++++++++--
tp/Texinfo/XS/parsetexi/Parsetexi.pm | 2 +
tp/Texinfo/XS/parsetexi/Parsetexi.xs | 3 +
tp/Texinfo/XS/parsetexi/conf.c | 7 ++
tp/Texinfo/XS/parsetexi/conf.h | 2 +
tp/Texinfo/XS/parsetexi/macro.c | 7 +-
tp/t/60macro.t | 11 +++
tp/t/results/macro/recursive_call_in_rmacro.pl | 124 +++++++++++++++++++++++++
10 files changed, 220 insertions(+), 15 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index a871a5b752..d416b25a45 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,26 @@
+2023-01-05 Patrice Dumas <pertusus@free.fr>
+
+ * tp/Texinfo/ParserNonXS.pm (%parser_state_initialization, _next_text)
+ (_process_remaining_on_line): add a value_stack to put expanded
+ @value. Use a sourcemark structure put in source_info to mark
+ the end of @value expansion. Also use such a structure for
+ @macro, replacing 'end_macro'.
+ Check number of values expansion number and error out if
+ there is more than MAX_MACRO_CALL_NESTING.
+ correct MAX_MACRO_CALL_NESTING name in warning message.
+
+ * tp/Texinfo/XS/parsetexi/Parsetexi.pm (parser),
+ tp/Texinfo/XS/parsetexi/Parsetexi.xs
+ (conf_set_MAX_MACRO_CALL_NESTING), tp/Texinfo/XS/parsetexi/conf.c
+ (conf_set_MAX_MACRO_CALL_NESTING, reset_conf),
+ tp/Texinfo/XS/parsetexi/macro.c (handle_macro): pass
+ MAX_MACRO_CALL_NESTING to the XS parser. Use that value for the
+ number of macro call nesting allowed.
+ correct MAX_MACRO_CALL_NESTING name in warning message.
+
+ * tp/t/60macro.t: add recursive_call_in_rmacro to test for an infinite
+ recursion.
+
2023-01-04 Patrice Dumas <pertusus@free.fr>
* .gitignore: replace tp/tests/included_lat*n1.texi by
diff --git a/tp/TODO b/tp/TODO
index 921de87825..7e97b81c6b 100644
--- a/tp/TODO
+++ b/tp/TODO
@@ -105,8 +105,9 @@ the context command stack.
@end defun
-Recursive expansion of @value may lead to an infinite loop. Wait for
-marksource to proceed.
+Recursive expansion of @value may lead to an infinite loop. Using
+a stack as for macros seems to be the best, and using sourcemark
+for the end of the expansion. (done in perl parser).
https://bugzilla.redhat.com/show_bug.cgi?id=744878
diff --git a/tp/Texinfo/ParserNonXS.pm b/tp/Texinfo/ParserNonXS.pm
index d3e5484fc5..6f97e0e581 100644
--- a/tp/Texinfo/ParserNonXS.pm
+++ b/tp/Texinfo/ParserNonXS.pm
@@ -149,6 +149,8 @@ my %parser_state_initialization = (
# as obtained by parsing the @macro
'macro_stack' => [], # stack of macros being expanded (more recent
# first)
+ 'value_stack' => [], # stack of values being expanded (more recent
+ # first)
'merged_indices' => {}, # the key is merged in the value
'regions_stack' => [], # a stack of regions commands elements (block
# region commands)
@@ -2190,12 +2192,22 @@ sub _next_text($$)
my $input = $self->{'input'}->[0];
if (@{$input->{'pending'}}) {
my $new_text_and_info = shift @{$input->{'pending'}};
- if ($new_text_and_info->[1] and $new_text_and_info->[1]->{'end_macro'}) {
- delete $new_text_and_info->[1]->{'end_macro'};
- my $top_macro = shift @{$self->{'macro_stack'}};
- print STDERR "SHIFT MACRO_STACK(@{$self->{'macro_stack'}}):"
- ." $top_macro->{'args'}->[0]->{'text'}\n"
- if ($self->{'DEBUG'});
+ if ($new_text_and_info->[1] and $new_text_and_info->[1]->{'sourcemark'})
{
+ my $sourcemark = $new_text_and_info->[1]->{'sourcemark'};
+ delete $new_text_and_info->[1]->{'sourcemark'};
+ if ($sourcemark->{'type'} eq 'macro'
+ and $sourcemark->{'status'} eq 'end') {
+ my $top_macro = shift @{$self->{'macro_stack'}};
+ print STDERR "SHIFT MACRO_STACK(@{$self->{'macro_stack'}}):"
+ ." $top_macro->{'args'}->[0]->{'text'}\n"
+ if ($self->{'DEBUG'});
+ } elsif ($sourcemark->{'type'} eq 'value'
+ and $sourcemark->{'status'} eq 'end') {
+ my $top_value = shift @{$self->{'value_stack'}};
+ print STDERR "SHIFT VALUE_STACK(@{$self->{'value_stack'}}):"
+ . "$top_value\n"
+ if ($self->{'DEBUG'});
+ }
}
# corresponds to (line, new source_info)
return ($new_text_and_info->[0], $new_text_and_info->[1]);
@@ -4392,7 +4404,7 @@ sub _process_remaining_on_line($$$$)
if ($self->{'MAX_MACRO_CALL_NESTING'}
and scalar(@{$self->{'macro_stack'}}) >
$self->{'MAX_MACRO_CALL_NESTING'}) {
$self->_line_warn(sprintf(__(
- "macro call nested too deeply (set MAX_NESTED_MACROS to override; current
value %d)"),
+ "macro call nested too deeply (set MAX_MACRO_CALL_NESTING to override;
current value %d)"),
$self->{'MAX_MACRO_CALL_NESTING'}),
$source_info);
goto funexit;
}
@@ -4423,7 +4435,9 @@ sub _process_remaining_on_line($$$$)
my $new_lines = _complete_line_nr($expanded_lines,
$source_info->{'line_nr'}, $source_info->{'file_name'},
$expanded_macro->{'args'}->[0]->{'text'}, 1);
- $source_info->{'end_macro'} = 1;
+ $source_info->{'sourcemark'} = {'type' => 'macro',
+ 'info' => $expanded_macro->{'args'}->[0]->{'text'},
+ 'status' => 'end'};
# first put the line that was interrupted by the macro call
# on the input pending text with information stack
if (! scalar(@{$self->{'input'}})) {
@@ -4451,12 +4465,29 @@ sub _process_remaining_on_line($$$$)
if ($expanded_line =~ s/^{([\w\-][^\s{\\}~`\^+"<>|@]*)}//) {
my $value = $1;
if (exists($self->{'values'}->{$value})) {
- $line = $self->{'values'}->{$value} . $expanded_line;
+ if ($self->{'MAX_MACRO_CALL_NESTING'}
+ and scalar(@{$self->{'value_stack'}}) >
$self->{'MAX_MACRO_CALL_NESTING'}) {
+ $self->_line_warn(sprintf(__(
+ "value call nested too deeply (set MAX_MACRO_CALL_NESTING to override;
current value %d)"),
+ $self->{'MAX_MACRO_CALL_NESTING'}),
$source_info);
+ $line = $expanded_line;
+ goto funexit;
+ }
+ unshift @{$self->{'value_stack'}}, $value;
+ if (! scalar(@{$self->{'input'}})) {
+ push @{$self->{'input'}}, {'pending' => []};
+ }
+ my $pending_source_info = { %$source_info };
+ $pending_source_info->{'sourcemark'} = {'type' => 'value',
+ 'info' => $value,
+ 'status' => 'end'};
+ unshift @{$self->{'input'}->[0]->{'pending'}},
+ [$expanded_line, $pending_source_info];
+ $line = $self->{'values'}->{$value};
goto funexit;
}
}
}
- #substr($line, 0, $at_command_length) = '';
}
# special case for @-command as argument of @itemize or @*table.
diff --git a/tp/Texinfo/XS/parsetexi/Parsetexi.pm
b/tp/Texinfo/XS/parsetexi/Parsetexi.pm
index 0dbb27ceeb..840fa89c6d 100644
--- a/tp/Texinfo/XS/parsetexi/Parsetexi.pm
+++ b/tp/Texinfo/XS/parsetexi/Parsetexi.pm
@@ -137,6 +137,8 @@ sub parser (;$$)
conf_set_IGNORE_SPACE_AFTER_BRACED_COMMAND_NAME ($conf->{$key});
} elsif ($key eq 'CPP_LINE_DIRECTIVES') {
conf_set_CPP_LINE_DIRECTIVES($conf->{$key});
+ } elsif ($key eq 'MAX_MACRO_CALL_NESTING') {
+ conf_set_MAX_MACRO_CALL_NESTING($conf->{$key});
} elsif ($key eq 'DEBUG') {
set_debug($conf->{$key}) if $conf->{$key};
} elsif ($key eq 'DOC_ENCODING_FOR_INPUT_FILE_NAME') {
diff --git a/tp/Texinfo/XS/parsetexi/Parsetexi.xs
b/tp/Texinfo/XS/parsetexi/Parsetexi.xs
index d5972061fa..74ef870d14 100644
--- a/tp/Texinfo/XS/parsetexi/Parsetexi.xs
+++ b/tp/Texinfo/XS/parsetexi/Parsetexi.xs
@@ -106,6 +106,9 @@ conf_set_CPP_LINE_DIRECTIVES (int i)
void
conf_set_IGNORE_SPACE_AFTER_BRACED_COMMAND_NAME (int i)
+void
+conf_set_MAX_MACRO_CALL_NESTING (int i)
+
void
set_DOC_ENCODING_FOR_INPUT_FILE_NAME (int i)
diff --git a/tp/Texinfo/XS/parsetexi/conf.c b/tp/Texinfo/XS/parsetexi/conf.c
index 5797e12853..4d29326557 100644
--- a/tp/Texinfo/XS/parsetexi/conf.c
+++ b/tp/Texinfo/XS/parsetexi/conf.c
@@ -40,6 +40,12 @@ conf_set_IGNORE_SPACE_AFTER_BRACED_COMMAND_NAME (int i)
conf.ignore_space_after_braced_command_name = i;
}
+void
+conf_set_MAX_MACRO_CALL_NESTING (int i)
+{
+ conf.max_macro_call_nesting = i;
+}
+
void
reset_conf (void)
{
@@ -47,5 +53,6 @@ reset_conf (void)
conf.show_menu = 1;
conf.cpp_line_directives = 1;
conf.ignore_space_after_braced_command_name = 1;
+ conf.max_macro_call_nesting = 100000;
conf.doc_encoding_for_input_file_name = 1;
}
diff --git a/tp/Texinfo/XS/parsetexi/conf.h b/tp/Texinfo/XS/parsetexi/conf.h
index 33ce3b7fbf..9236ea7191 100644
--- a/tp/Texinfo/XS/parsetexi/conf.h
+++ b/tp/Texinfo/XS/parsetexi/conf.h
@@ -20,6 +20,7 @@ typedef struct CONF {
int show_menu;
int cpp_line_directives;
int ignore_space_after_braced_command_name;
+ int max_macro_call_nesting;
int doc_encoding_for_input_file_name;
} CONF;
@@ -28,6 +29,7 @@ extern CONF conf;
void conf_set_show_menu (int i);
void conf_set_CPP_LINE_DIRECTIVES (int i);
void conf_set_IGNORE_SPACE_AFTER_BRACED_COMMAND_NAME (int i);
+void conf_set_MAX_MACRO_CALL_NESTING (int i);
void reset_conf (void);
#endif
diff --git a/tp/Texinfo/XS/parsetexi/macro.c b/tp/Texinfo/XS/parsetexi/macro.c
index a24bb2fc7f..29d4900817 100644
--- a/tp/Texinfo/XS/parsetexi/macro.c
+++ b/tp/Texinfo/XS/parsetexi/macro.c
@@ -558,13 +558,14 @@ handle_macro (ELEMENT *current, char **line_inout, enum
command_id cmd)
if (expanded.end > 0 && expanded.text[expanded.end - 1] == '\n')
expanded.text[--expanded.end] = '\0';
- if (input_number >= 1000)
+ if (conf.max_macro_call_nesting
+ && input_number >= conf.max_macro_call_nesting)
{
line_warn (
"macro call nested too deeply "
- "(set MAX_NESTED_MACROS to override; current value %d)", 1000);
+ "(set MAX_MACRO_CALL_NESTING to override; current value %d)",
+ conf.max_macro_call_nesting);
goto funexit;
- /* TODO: actually check MAX_NESTED_MACROS? */
}
if (macro->cmd == CM_macro)
diff --git a/tp/t/60macro.t b/tp/t/60macro.t
index 0364cdbafe..80e05b1c3d 100644
--- a/tp/t/60macro.t
+++ b/tp/t/60macro.t
@@ -954,6 +954,17 @@ ggg
fff
@end macro
@mac'],
+# takes long with default MAX_MACRO_CALL_NESTING value (tested with
+# pure perl Parser).
+['recursive_call_in_rmacro',
+'
+@rmacro rec
+@rec{}
+@end rmacro
+
+@rec{}
+
+', {'MAX_MACRO_CALL_NESTING' => 100}],
['unknown_macro_on_line_command',
'@setfilename @begin{}file'
],
diff --git a/tp/t/results/macro/recursive_call_in_rmacro.pl
b/tp/t/results/macro/recursive_call_in_rmacro.pl
new file mode 100644
index 0000000000..b07cb8e70c
--- /dev/null
+++ b/tp/t/results/macro/recursive_call_in_rmacro.pl
@@ -0,0 +1,124 @@
+use vars qw(%result_texis %result_texts %result_trees %result_errors
+ %result_indices %result_sectioning %result_nodes %result_menus
+ %result_floats %result_converted %result_converted_errors
+ %result_elements %result_directions_text %result_indices_sort_strings);
+
+use utf8;
+
+$result_trees{'recursive_call_in_rmacro'} = {
+ 'contents' => [
+ {
+ 'contents' => [
+ {
+ 'text' => '
+',
+ 'type' => 'empty_line'
+ },
+ {
+ 'args' => [
+ {
+ 'text' => 'rec',
+ 'type' => 'macro_name'
+ }
+ ],
+ 'cmdname' => 'rmacro',
+ 'contents' => [
+ {
+ 'text' => '@rec{}
+',
+ 'type' => 'raw'
+ },
+ {
+ 'args' => [
+ {
+ 'contents' => [
+ {
+ 'text' => 'rmacro'
+ }
+ ],
+ 'info' => {
+ 'spaces_after_argument' => '
+'
+ },
+ 'type' => 'line_arg'
+ }
+ ],
+ 'cmdname' => 'end',
+ 'extra' => {
+ 'text_arg' => 'rmacro'
+ },
+ 'info' => {
+ 'spaces_before_argument' => ' '
+ },
+ 'source_info' => {
+ 'file_name' => '',
+ 'line_nr' => 4,
+ 'macro' => ''
+ }
+ }
+ ],
+ 'info' => {
+ 'arg_line' => ' rec
+'
+ },
+ 'source_info' => {
+ 'file_name' => '',
+ 'line_nr' => 2,
+ 'macro' => ''
+ }
+ },
+ {
+ 'text' => '
+',
+ 'type' => 'empty_line'
+ },
+ {
+ 'text' => '
+',
+ 'type' => 'empty_line'
+ },
+ {
+ 'text' => '
+',
+ 'type' => 'empty_line'
+ }
+ ],
+ 'type' => 'before_node_section'
+ }
+ ],
+ 'type' => 'document_root'
+};
+
+$result_texis{'recursive_call_in_rmacro'} = '
+@rmacro rec
+@rec{}
+@end rmacro
+
+
+
+';
+
+
+$result_texts{'recursive_call_in_rmacro'} = '
+
+
+
+';
+
+$result_errors{'recursive_call_in_rmacro'} = [
+ {
+ 'error_line' => 'warning: macro call nested too deeply (set
MAX_MACRO_CALL_NESTING to override; current value 100) (possibly involving @rec)
+',
+ 'file_name' => '',
+ 'line_nr' => 6,
+ 'macro' => 'rec',
+ 'text' => 'macro call nested too deeply (set MAX_MACRO_CALL_NESTING to
override; current value 100)',
+ 'type' => 'warning'
+ }
+];
+
+
+$result_floats{'recursive_call_in_rmacro'} = {};
+
+
+1;
| [Prev in Thread] |
Current Thread |
[Next in Thread] |
- branch master updated: * tp/Texinfo/ParserNonXS.pm (%parser_state_initialization, _next_text) (_process_remaining_on_line): add a value_stack to put expanded @value. Use a sourcemark structure put in source_info to mark the end of @value expansion. Also use such a structure for @macro, replacing 'end_macro'. Check number of values expansion number and error out if there is more than MAX_MACRO_CALL_NESTING. correct MAX_MACRO_CALL_NESTING name in warning message.,
Patrice Dumas <=