Re: Autotest function usage

[Ralf Wildenhues]

Hello Paolo, all,

* Ralf Wildenhues wrote on Sun, Oct 21, 2007 at 02:36:55PM CEST:
> 2007-10-20  Paolo Bonzini  <address@hidden>
>       and Ralf Wildenhues  <address@hidden>
>  
>       * lib/autotest/general.m4 (at_func_test): Remove.
>       (AT_INIT): Pre-extract test groups into separate files.
>       (AT_CLEANUP): Source pre-extracted file instead of calling at_func_test.
>       Remove at-test-source files together with the $at_group_dir.

My testing revealed that this totally trashes performance with borked
shells like AIX sh.  As in: the time from `sh ./testsuite' until the
first test starts is expected to be around half an hour.  Compare with:
less than 20 seconds with bash, and 7 seconds with sh and the
at_func_test way of extracting tests, both on the same system.

I know we generally optimize for decent systems, but a few percent gain
at best, if even at all, seem too little a rationale for such a huge
loss.  Especially since before this, we actually had comparably decent
performance with this shell.

OK to revert?  (Asking this before considering Paolo's other patch;
I haven't tested that one much yet.)

Cheers,
Ralf

> diff --git a/lib/autotest/general.m4 b/lib/autotest/general.m4
> index 29834f1..49026f2 100644
> --- a/lib/autotest/general.m4
> +++ b/lib/autotest/general.m4
> @@ -310,15 +310,6 @@ at_func_diff_devnull ()
>    $at_diff "$at_devnull" "$[1]"
>  }
>  
> -# at_func_test NUMBER
> -# -------------------
> -# Parse out test NUMBER from the tail of this file.
> -at_func_test ()
> -{
> -  sed -n 
> '/address@hidden:@AT_START_'$[1]'$/,/address@hidden:@AT_STOP_'$[1]'$/p' 
> "$at_myself" \
> -       > "$at_test_source"
> -}
> -
>  # at_func_create_debugging_script
>  # -------------------------------
>  # Create the debugging script $at_group_dir/run which will reproduce the
> @@ -413,7 +404,7 @@ at_status_file=$at_suite_dir/at-status
>  at_stdout=$at_suite_dir/at-stdout
>  at_stder1=$at_suite_dir/at-stder1
>  at_stderr=$at_suite_dir/at-stderr
> -# The file containing the function to run a test group.
> +# The stem for files containing a test group.
>  at_test_source=$at_suite_dir/at-test-source
>  # The file containing dates.
>  at_times_file=$at_suite_dir/at-times
> @@ -853,6 +844,31 @@ else
>    at_diff=diff
>  fi
>  
> +[{
> +  echo 'BEGIN {'
> +  for at_group in $at_groups; do
> +    at_group_normalized=$at_group
> +    ]_AT_NORMALIZE_TEST_GROUP_NUMBER(at_group_normalized)[
> +    echo "  outfile[\"$at_group\"] = 
> \"$at_test_source-$at_group_normalized\""
> +  done
> +  echo '}
> +emit == 0 && /address@hidden:@AT_START_/ {
> +  test = substr($ 0, 11);
> +  if (outfile[test]) emit = 1
> +}
> +emit != 0 && /address@hidden:@AT_STOP_/ {
> +  if (outfile[test]) close (outfile[test])
> +  emit = 0
> +}
> +emit != 0 { print > outfile[test] }
> +'
> +}] > "$at_test_source.awk"
> +
> +# Extract test group that will be run from the tail of this file
> +if awk -f "$at_test_source.awk" "$at_myself"; then :; else
> +  AS_ECHO(["$as_me: unable to parse test groups"]) >&2
> +  exit 1
> +fi
>  
>  m4_text_box([Driver loop.])
>  for at_group in $at_groups
> @@ -888,10 +904,7 @@ do
>      at_tee_pipe='cat >> "$at_group_log"'
>    fi
>  
> -  if at_func_test $at_group && . "$at_test_source"; then :; else
> -    AS_ECHO(["$as_me: unable to parse test group: $at_group"]) >&2
> -    at_failed=:
> -  fi
> +  . "$at_test_source-$at_group_normalized"
>  
>    # Be sure to come back to the suite directory, in particular
>    # since below we might `rm' the group directory we are in currently.
> @@ -954,9 +967,12 @@ _ATEOF
>        # Cleanup the group directory, unless the user wants the files.
>        if $at_debug_p ; then
>       at_func_create_debugging_script
> -      elif test -d "$at_group_dir"; then
> -     find "$at_group_dir" -type d ! -perm -700 -exec chmod u+rwx \{\} \;
> -     rm -fr "$at_group_dir"
> +      else
> +     if test -d "$at_group_dir"; then
> +       find "$at_group_dir" -type d ! -perm -700 -exec chmod u+rwx \{\} \;
> +       rm -fr "$at_group_dir"
> +        fi
> +     rm -f "$at_test_source-$at_group_normalized"
>        fi
>        ;;
>      *)