[Bug binutils/23720] New: Build directory outside of Source Tree - Test Suite not executed correctly if executed with runtest

[bernhard.barbisch at bachmann dot info]

https://sourceware.org/bugzilla/show_bug.cgi?id=23720

            Bug ID: 23720
           Summary: Build directory outside of Source Tree - Test Suite
                    not executed correctly if executed with runtest
           Product: binutils
           Version: 2.32 (HEAD)
            Status: UNCONFIRMED
          Severity: minor
          Priority: P2
         Component: binutils
          Assignee: unassigned at sourceware dot org
          Reporter: bernhard.barbisch at bachmann dot info
  Target Milestone: ---

Description
If the binutils test suite is used to verify a build outside of the source
directory tree using runtest (objdir in site.exp points to the external build
directory), the tests do not use the newly build binaries but fall back to a
already exiting (older) binary.


Steps to reproduce
- build binutils (./configure and make check)
- copy binutils/testsuite to a destination outside of the binutils source tree
- copy the site.exp to the copied testuite directory 
- check site.exp (objdir should point to the build directory)
- execute runtest within the copied testsuite directory

Expected Result:
- tests are executed
- tests are using the newly generated binaries within the build directory

Current Result:
- tests are executed
- tests do not use the newly generated binaries but fall back to the system
binaries

Root cause:
the file testsuites/config/default.exp initializes the paths to the generated
binaries. It locates these binaries within $base_dir which points to the
current working direcotry (of the test execution). Instead of $base_dir the
variable $objdir should be used (which points the directory which contains the
binaries)


Additional info:
- this seems to be an issue only if you try to execute the test suite with
runtest and the objdir path pointing to an external directory. It does not seem
to be an issue if binutils is build in an external build directory and the
tests are executed using 'make check'.
- one can see that the wrong binaries are used if their versions differ (e.g.
system version: 2.30; build version 2.32) because '--version' outputs can be
found in the generated log files
- use case for this scenario: one wants to execute the test suite against a
already built/existing/deployed version of binutils
- this seems to be also an issue for
-- ld
-- gas

-- 
You are receiving this mail because:
You are on the CC list for the bug.