Re: Invoke a Coreutils routine from another 'main' program

[Assaf Gordon]

Hello,

On 07/21/2015 04:17 PM, Zhoulai wrote:
> Do you mean that, even if I manage to invoke
> /"echo.main/" from an external program, this external program will
> terminate whenever "echo.main"  gets an error?

yes.

> Indeed, that would be quite an issue for us. Actually, we are trying
> to automatically detect possible bugs in CoreUtils with a technique
> derived from our research work. To this end, we need to be able to
> stress-test 'echo.main', meaning that, we need to pass a large set of
> randomly generated inputs that syntactically conforms to the
> signature of the program under test.
<...>
> Do you think there is an easy workaround so that when
> echo.main() terminates with an error, our pseudocode can still be
> running?

Depending on your implementation and how the bugs are automatically detected,
perhaps it would be simpler for you to follow Eric Blake's suggestion,
and invoke the programs using exec (assuming you don't need to access the
same memory of the running program under test).
This approach sound similar to the "american fuzzy lop" program ( 
http://lcamtuf.coredump.cx/afl/ ).

few other not-so-easy alternatives (none of which I personally tested, these 
are just wild guesses):

1. write your tool as a valgrind module 
(http://valgrind.org/docs/manual/manual-writing-tools.html)

2. implement your own 'wrapper' with ptrace ( 
http://www.secretmango.com/jimb/Whitepapers/ptrace/ptrace.html ).

3. create GCC custom instrumentation to add code to detect when 
entering/exiting the program's main() function
   (an old example here: 
http://codingrelic.geekhold.com/2010/09/gcc-function-instrumentation.html)

4. restructure your code as a library, and add your code to each coreutil's 
program's main (for entry) and atexit (for termination). Also add signal 
handlers to catch segfaults.

5. Use a recent GDB with Python support, and write your bug-hunter using GDB's 
API ( https://sourceware.org/gdb/onlinedocs/gdb/Extending-GDB.html ).

6. Use LD_PRELOAD to inject code into the program ( 
https://rafalcieslak.wordpress.com/2013/04/02/dynamic-linker-tricks-using-ld_preload-to-cheat-inject-features-and-investigate-programs/
 ).

remember that there is more than one way to terminate a user-space program,
one could hope most clean terminations on Linux will go through 
exit(2)/exit_group(2) - but this is not guaranteed to be portable.
if you're hunting for bugs and trigger an invalid memory access, then all bets 
are off.

> Also, I just skimmed two of your source code, /echo.c,/ and
> /base64.c/ from CoreUtil8.23, but  do not find the lines that
> triggers termination of the whole process. Would you show me the
> relevant code lines ?

Most coreutils programs terminate by calling 'error' with non-zero first 
parameter:
  http://lingrok.org/xref/coreutils/src/base64.c#266

Other times, an 'error' with zero first parameter is followed by 'usage', which 
then terminates:
  http://lingrok.org/xref/coreutils/src/base64.c#285

Any time you see a function with 'x' prefix (e.g. 'xmalloc'),
it is a gnulib wrapper to the common function which will terminate on error.
Usage example here:
  http://lingrok.org/xref/coreutils/src/seq.c#259
xmalloc implementation here:
  http://git.savannah.gnu.org/cgit/gnulib.git/tree/lib/xmalloc.c

---

As a side note,
since you are looking for bugs, I would also recommend the following:

1. Work with the most recent git repository, don't use the official release 
tarballs - you might be working on a bug that was already fixed.

2. Consider focusing on less-tested programs. Use 'make coverage' to generate 
test-coverage report.
But also remember that some programs are heavily used in the real-world, and 
low-hanging fruits (=bugs) might be hard to find.

HTH,
 - assaf