Re: require_ulimit_v_(): tests skipped with enable-single-binary

[Pádraig Brady]

On 09/09/15 00:18, Bernhard Voelker wrote:
> All tests using 'require_ulimit_v_' are skipped when configuring with
> either --enable-single-binary=shebangs or ...=symlinks.
> 
> 
> + require_ulimit_v_
> + local ulimit_works=yes
> + local vm
> + case $(printenv LD_PRELOAD) in
> ++ printenv LD_PRELOAD
> + vm=10000
> + ulimit -v 10000
> + date
> /tmp/coreutils-8.24.16-907f3/build2/src/coreutils: error while loading shared 
> libraries: libcap.so.2: failed to map
> segment from shared object: Cannot allocate memory
> + ulimit_works=no
> + ulimit -v 20
> + date
> ../init.cfg: line 148: 29355 Killed                  date
> + test no = no
> + skip_ 'this shell lacks ulimit support'
> 
> 
> The reason is that the multicall 'coreutils' binary requires
> much more memory due to shared libraries being loaded, and the
> size of the 'date' binary (~290KiB) compared to the multicall
> binary (~5MiB), of course.
> 
> Adapting the above limit in the case the single-binary option is
> active doesn't help much, as the actual tests would still use hard
> coded values for 'ulimit -v' which are too low, of course.
> 
> One solution could be that 'require_ulimit_v_' determines the
> basic memory the actual test program (cp, dd, etc.) needs, and
> that the test then uses that value plus a certain number as actual
> memory limit.  Something along the following (completely untested!)
> lines:
> 
>   require_ulimit_v() {
>     ...
>     local v
>     for v in $( seq 1000 1000 50000 ); do
>       ( ulimit -v $v && "$@" ) && { echo $v; return 0; }
>     done

5000 looks like a base on Fedora 22 x86_64 here.
Maybe this would be a better method:

    for v in $( seq 5000 5000 50000 ); do
      if ( ulimit -v $v && "$@" ); then
        prev_v=$v
        for v in $( seq $(($prev_v-1000)) -1000 1000 ); do
          ( ulimit -v $v && "$@" ) || { echo $prev_v; return 0; }
          prev_v=$v
        done
      fi
    done
    return 1

>     ...
>     skip_ "can not determine basic ulimit"
>     ...
>   }
> 
> and in the tests:
> 
>   base=$( require_ulimit_v_ dd if=/dev/zero of=/dev/null bs=1 count=1 ) \
>     || skip_ "ulimit not working"
> 
>   ( ulimit -v $(expr $base + 30000) && dd ... ) || fail=1
>   ...
> 
> WDYT?  Is this worth the trouble?


I mentioned this as a TODO previously, so +1 from me.
Having this adaptive is better anyway, as then all
these tests are more adaptable to disparate and changing systems.

Given the number of tests this impacts I can't think
of anything better than the above method.

  $ git grep -l ulimit tests/
  tests/cp/link-heap.sh
  tests/dd/no-allocate.sh
  tests/misc/csplit-heap.sh
  tests/misc/cut-huge-range.sh
  tests/misc/head-c.sh
  tests/misc/printf-surprise.sh
  tests/misc/sort-continue.sh
  tests/misc/sort-merge-fdlimit.sh
  tests/misc/sort-spinlock-abuse.sh
  tests/rm/many-dir-entries-vs-OOM.sh
  tests/split/line-bytes.sh
  tests/split/r-chunk.sh

thanks!
Pádraig.