Re: [Bug-tar] Small bug in tar-1.29

[Jack Howarth]

On Tue, Sep 06, 2016 at 08:24:21PM -0400, Jack Howarth wrote:
> On Tue, Sep 06, 2016 at 06:48:23PM -0400, Jack Howarth wrote:
> > On Tue, Sep 06, 2016 at 04:12:12PM -0400, Jack Howarth wrote:
> > > On Tue, Sep 06, 2016 at 08:12:05PM +0100, Ralph Corderoy wrote:
> > > > Hi Jack,
> > > > 
> > > > > > gnutar: Child died with signal 11
> > > > ...
> > > > > stop reason = signal SIGPIPE
> > > > 
> > > > SIGPIPE is 13.  We want 11, SIGSEGV.  Perhaps a "c" or two for continue
> > > > will have the debugger carry on until the SEGV bites.
> > > 
> > > $ ulimit -c unlimited
> > > $ gnutar -zcvf test_data.tar.gz test_data 
> > > test_data/
> > > test_data/.noe.tbl.swp
> > > test_data/dihedral.tbl
> > > test_data/file.nam
> > > test_data/io.mc
> > > test_data/noe.tbl
> > > gnutar: test_data.tar.gz: Cannot write: Broken pipe
> > > gnutar: Child died with signal 11
> > > gnutar: Error is not recoverable: exiting now
> > > $ lldb -c /cores/core.21601 `which gnutar`
> > > (lldb) target create "/opt/local/bin/gnutar" --core "/cores/core.21601"
> > > warning: (x86_64) /cores/core.21601 load command 84 LC_SEGMENT_64 has a 
> > > fileoff + filesize (0x296fe000) that extends beyond the end of the file 
> > > (0x296fd000), the segment will be truncated to match
> > > Core file '/cores/core.21601' (x86_64) was loaded.
> > > (lldb) bt
> > > * thread #1: tid = 0x0000, 0x00007fff94ebb6d7 
> > > libdispatch.dylib`_dispatch_queue_push_queue + 463, stop reason = signal 
> > > SIGSTOP
> > >   * frame #0: 0x00007fff94ebb6d7 
> > > libdispatch.dylib`_dispatch_queue_push_queue + 463
> > >     frame #1: 0x00007fff7478c4d0 
> > > libdispatch.dylib`_OS_dispatch_mach_vtable + 48
> > >     frame #2: 0x00007fff94eb9b06 
> > > libdispatch.dylib`_dispatch_queue_wakeup_with_qos_slow + 126
> > >     frame #3: 0x00007fff94ec013f 
> > > libdispatch.dylib`_dispatch_mach_msg_send + 1952
> > >     frame #4: 0x00007fff94ebf8dc libdispatch.dylib`dispatch_mach_send + 
> > > 262
> > >     frame #5: 0x00007fff89448fc9 
> > > libxpc.dylib`xpc_connection_send_message_with_reply + 131
> > >     frame #6: 0x00007fff8e31ebdf 
> > > CoreFoundation`__66-[CFPrefsSearchListSource 
> > > generationCountFromListOfSources:count:]_block_invoke_2 + 143
> > >     frame #7: 0x00007fff8e31ea0d 
> > > CoreFoundation`_CFPrefsWithDaemonConnection + 381
> > >     frame #8: 0x00007fff8e31db96 
> > > CoreFoundation`__66-[CFPrefsSearchListSource 
> > > generationCountFromListOfSources:count:]_block_invoke + 150
> > >     frame #9: 0x00007fff8e31d933 CoreFoundation`-[CFPrefsSearchListSource 
> > > generationCountFromListOfSources:count:] + 179
> > >     frame #10: 0x00007fff8e31d214 
> > > CoreFoundation`-[CFPrefsSearchListSource alreadylocked_copyDictionary] + 
> > > 324
> > >     frame #11: 0x00007fff8e31ce5c 
> > > CoreFoundation`-[CFPrefsSearchListSource alreadylocked_copyValueForKey:] 
> > > + 60
> > >     frame #12: 0x00007fff8e31cdec 
> > > CoreFoundation`___CFPreferencesCopyAppValueWithContainer_block_invoke + 60
> > >     frame #13: 0x00007fff8e314b10 
> > > CoreFoundation`+[CFPrefsSearchListSource 
> > > withSearchListForIdentifier:container:perform:] + 608
> > >     frame #14: 0x00007fff8e314867 
> > > CoreFoundation`_CFPreferencesCopyAppValueWithContainer + 183
> > >     frame #15: 0x00000001021a6538 
> > > libintl.8.dylib`_nl_language_preferences_default + 70
> > >     frame #16: 0x00000001021a462f libintl.8.dylib`libintl_dcigettext + 667
> > >     frame #17: 0x00000001021491f2 gnutar`sys_child_open_for_compress + 125
> > >     frame #18: 0x000000010213011f gnutar`open_archive + 603
> > >     frame #19: 0x00000001021355ab gnutar`create_archive + 67
> > >     frame #20: 0x000000010214b494 gnutar`main + 3510
> > >     frame #21: 0x00007fff895605ad libdyld.dylib`start + 1
> > >     frame #22: 0x00007fff895605ad libdyld.dylib`start + 1
> > > (lldb)
> > > 
> > 
> > Using the same approach that I took in http://savannah.gnu.org/bugs/?46261 
> > to determine that the threading in the CoreFoundation
> > framework linked through libintl was causing breakage on darwin15 in make 
> > 4.0/4,1...
> > 
> > rebuilding gettext with the additional configure options...
> > 
> > gt_cv_func_CFPreferencesCopyAppValue=no \ 
> > gt_cv_func_CFLocaleCopyCurrent=no \ 
> > gt_cv_func_CFPreferencesCopyAppValue=no \ 
> > gt_cv_func_CFLocaleCopyCurrent=no 
> > 
> > to change the linkage from....
> > 
> > $ otool -L /opt/local/lib/libintl.8.dylib
> > /opt/local/lib/libintl.8.dylib:
> >     /opt/local/lib/libintl.8.dylib (compatibility version 10.0.0, current 
> > version 10.5.0)
> >     /opt/local/lib/libiconv.2.dylib (compatibility version 8.0.0, current 
> > version 8.1.0)
> >     /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current 
> > version 1226.10.1)
> >     
> > /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
> >  (compatibility version 150.0.0, current version 1258.1.0)
> > 
> > $ otool -L /opt/local/lib/libintl.8.dylib
> > /opt/local/lib/libintl.8.dylib:
> >     /opt/local/lib/libintl.8.dylib (compatibility version 10.0.0, current 
> > version 10.5.0)
> >     /opt/local/lib/libiconv.2.dylib (compatibility version 8.0.0, current 
> > version 8.1.0)
> >     /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current 
> > version 1226.10.1)
> > 
> > does indeed suppress the bug in tar 1.29 when LANG is unset. In the case of 
> > make, this was fixed by
> > the addition of pselect() calls to protect the threads before forking...
> > 
> > http://git.savannah.gnu.org/cgit/make.git/commit/?id=85c788572d054bc2c41b84007875edbd37ad3ed5
> > 
> > [SV 46261] Use pselect() for jobserver where supported.
> > * Makefile.am, configure.ac: Check for pselect() and sys/select.h.
> > * main.c (main): Block SIGCHLD if we have pselect() support.
> > * posixos.c (jobserver_acquire): If we support pselect() then use
> > it to query the jobserver pipe, while also listening for SIGCHLD.
> > Also pselect() supports a timeout so avoid alarm() calls.
> > 
> > Perhaps something similar needs to be done for the xpipe() call in tar 1.29?
> 
> I think that 
> http://git.savannah.gnu.org/cgit/tar.git/commit/?id=589ba77faf60e4632771d80e2b25d9e3778d4a3f
> 
> Catch compressor execution errors.
> * src/misc.c (write_fatal_details): Move to buffer.c
> * src/buffer.c (write_fatal_details): Close the archive and wait for
> the compressor program to terminate in order to catch eventual execution
> errors.
> * src/system.c (sys_child_open_for_compress): Ignore SIGPIPE so tar will
> not silently terminate when unable to write to the compressor.
> * tests/comperr.at: New file.
> * tests/Makefile.am: Add comperr.at
> * tests/testsuite.at: Include comperr.at
> 
> seems a likely candidate for code change causing this threading conflict with 
> CoreFoundation.
> In particular, the change...
> 
> diff --git a/src/system.c b/src/system.c
> index 9414233..dc0b2b0 100644
> --- a/src/system.c
> +++ b/src/system.c
> @@ -330,6 +330,7 @@ sys_child_open_for_compress (void)
>    pid_t grandchild_pid;
>    pid_t child_pid;
>  
> +  signal (SIGPIPE, SIG_IGN);
>    xpipe (parent_pipe);
>    child_pid = xfork ();
>  
> Perhaps pselect() could be used to fix this on darwin?

Confirmed with tar from git using...

git checkout d02c81df15e3ac3764c654f1e3bf24b5f45b3ed7
./configure --without-xattrs  CPPFLAGS="-I/sw/include -I/usr/lib" 
LDFLAGS="-L/sw/lib -L/usr/lib"
make

that the failure with tar 1.29git when LANG is unset doesn't exist at....

commit d02c81df15e3ac3764c654f1e3bf24b5f45b3ed7
Author: Sergey Poznyakoff <address@hidden>
Date:   Sun Nov 29 20:51:08 2015 +0200

    Fix extraction from concatenated incremental archives with renamed 
directories.
    
    Complements 15c02c2b.
    
    * src/extract.c (delayed_set_stat): Change type of file_name.
    (delay_set_stat): Allocate file_name member.
    (free_delayed_set_stat): Free file_name.
    (fixup_delayed_set_stat): New function.
    (rename_directory): Call fixup_delayed_set_stat on success.
    
    * tests/incr11.at: New testcase.
    * tests/incr10.at: Improve description.
    * tests/Makefile.am: Add incr11.at
    * tests/testsuite.at: Add incr11.at

but using...

git checkout 589ba77faf60e4632771d80e2b25d9e3778d4a3f
./configure --without-xattrs  CPPFLAGS="-I/sw/include -I/usr/lib" 
LDFLAGS="-L/sw/lib -L/usr/lib"
make 

produces the failures now seen in tar 1.29 when LANG is unset so the offending 
commits are...

commit 589ba77faf60e4632771d80e2b25d9e3778d4a3f
Author: Sergey Poznyakoff <address@hidden>
Date:   Sat Dec 5 16:54:26 2015 +0200

    Catch compressor execution errors.
    
    * src/misc.c (write_fatal_details): Move to buffer.c
    * src/buffer.c (write_fatal_details): Close the archive and wait for
    the compressor program to terminate in order to catch eventual execution
    errors.
    * src/system.c (sys_child_open_for_compress): Ignore SIGPIPE so tar will
    not silently terminate when unable to write to the compressor.
    * tests/comperr.at: New file.
    * tests/Makefile.am: Add comperr.at
    * tests/testsuite.at: Include comperr.at

commit 6ea9e62bb398b2f81e52f671749e585367286820
Author: Sergey Poznyakoff <address@hidden>
Date:   Sat Dec 5 08:48:03 2015 +0200

    Upgrade paxutils

which produces issues with CoreFoundation's threading out of the libintl 
linkage on tar.

> 
> > > 
> > > > 
> > > > -- 
> > > > Cheers, Ralph.
> > > > https://plus.google.com/+RalphCorderoy