bug-bash
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

bash does not wait for called script to finish


From: Seved Torstendahl
Subject: bash does not wait for called script to finish
Date: Wed, 12 Nov 2008 11:46:18 +0100

Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS:  -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' 
-DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-pc-linux-gnu' 
-DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL 
-DHAVE_CONFIG_H   -I.  -I../bash -I../bash/include -I../bash/lib   -g -O2
uname output: Linux jessie 2.6.18-5-amd64 #1 SMP Sat Dec 22 20:43:59 UTC 2007 
x86_64 GNU/Linux
Machine Type: x86_64-pc-linux-gnu

Bash Version: 3.1
Patch Level: 17
Release Status: release

Description:

This error is very hard to reproduce. It happens in the innermost loop 
in nightly jobs. Four jobs run in parallel and tree of them will pass this 
point around 800 times each night, the fourth job only about 100 times. 
The error happens about once every second week, i.e. about once out of 
50000 times. 

The main script contains two for loops and each time a function do_build 
is called. do_build creates a directory /tmp/build.nnnn and 'cd' into it. 
A separate script is called to setup some needed files including a Makefile 
and the 'make info' is performed. 

Normally this works fine but in rare cases do_build continues without waiting 
for the setup script to complete! Then of course the 'make info' fails. 
It has taken some time to understand what happens but now some data are 
collected and sent to me when 'make info' fails. 

The build machine has 4 CPUs and 4GB RAM, see info at the bottom. 

##########################################################################
# function do_build ()

function do_build ()
{
# called with ARG1 = modulename ARG2 = configuration file ARG3 = logname
        cd $SOURCE_AREA
        test -d /tmp/build.$$ && mv /tmp/build.$$ /tmp/build.$$.old
        rm -rf /tmp/build.$$.old
        mkdir /tmp/build.$$
        cd /tmp/build.$$
        echo -n "Building $1 for  $2  in `pwd` on "
        date
        nsetup=$SOURCE_AREA/mksupp/nsetup.sh
        ln -s $SOURCE_AREA/$1 .
        SOURCE_AREA=. $nsetup -q $2
        make info   > $3 2>&1
        if [ $? -ne 0 ]; then 
            lsres=`ls -ld ./* /tmp/b* 2>&1`
            wdnow=`pwd 2>&1`
            pstree=`ps fx`
            mail -s 'make info failed' address@hidden <<EOF
            Module:             $1
            Config file:        $2
            Branch:             $ver
            Build area:         /tmp/build.$$
            pwd:  $wdnow
            pstree:  $pstree
            ls -ld ./* /tmp/b* :
$lsres
EOF
        fi
        ## more job
}
########################################################################


Resulting mail. N.B. ls reports only one file, the newly created symlink,
and the 'ps fx' result (pstree:) shows that the main script build_node.sh, 
pid 23678, has 2 (TWO!) children: pid 23986 running nsetup and pid 24001 
running 'ps fx' and 24001 has thus performed 'make info' which failed.  

To: address@hidden
Subject: make info failed
Message-Id: <address@hidden>
From: address@hidden
Date: Mon, 10 Nov 2008 04:31:33 +0100

            Module:             ark-gpio
            Config file:        
/lhome/iovbuild/source_i386/head/sw/configs/nimbra680_debug.config
            Branch:             head
            Build area:         /tmp/build.23678
            pwd:  /tmp/build.23678
            pstree:    PID TTY      STAT   TIME COMMAND
29554 ?        Ss     0:00 /bin/bash -c /home/iovbuild/tools/nightly/build 
--arch arm -x testing iris head dev_ethernetff > /lhome/iovbuild/test/arm.log 
2>&1 
29562 ?        S      0:00  \_ /bin/bash /home/iovbuild/tools/nightly/build 
--arch arm -x testing iris head dev_ethernetff
 2751 ?        S      0:00      \_ /bin/sh 
/home/iovbuild/tools/nightly/build_node.sh -x --branches head --increment 
nimbra340
 3265 ?        R      0:17          \_ cvs -qf -z 3 get -P sw/adt sw/alarm 
sw/base-files sw/base-files-nimbra300 sw/busybox sw/ccm sw/chm sw/crc sw/dcc 
sw/dcp sw/ddns sw/difinfo sw/dispatcher sw/dle sw/dlsp sw/dlsp_oam sw/dpeek 
sw/dscp-cmd sw/dsync sw/dsyp sw/dtm-if sw/dtm_addr sw/dtm_route sw/dtmd sw/dxl 
sw/dxl-if sw/eqm sw/eth-if sw/ethconf sw/ethtool sw/ets sw/event sw/flow 
sw/glibc sw/gpio sw/inetutils sw/inventory sw/ipconf sw/iptables sw/ism-bus 
sw/ism-dsync sw/ism-dxl sw/ism-eth sw/ism-gpio sw/ism-hal-iad sw/ism-hal-if 
sw/ism-mau sw/ism-ncb sw/ism-switch sw/ism-unknown sw/its sw/kernel sw/lurix 
sw/mibs sw/misc sw/module-init-tools sw/mtd-utils sw/nano sw/nb-syslog 
sw/nc-status sw/ncall sw/ncurses sw/netilib sw/netkit sw/ntp sw/oam sw/ooc 
sw/pest sw/plog-utils sw/pmm sw/protocol-mgr sw/rcmd sw/registry sw/rm sw/rst 
sw/sarq sw/slot sw/slot0-channel sw/snmp sw/snmp-if sw/sqc sw/supd sw/swap 
sw/sysklogd sw/sysvars sw/tim sw/tim-if sw/tnrm sw/ttf sw/twofish sw/tz sw/udev 
sw/web sw/zlib sw/nimbra340-dtm-base sw/nimbra340-os-base sw/nodemgmt 
sw/test-tools sw/configs
23664 ?        Ss     0:00 /bin/bash -c /home/iovbuild/tools/nightly/build 
--arch i386 -x testing iris head dev_ethernetff > /lhome/iovbuild/test/i386.log 
2>&1 
23666 ?        S      0:00  \_ /bin/bash /home/iovbuild/tools/nightly/build 
--arch i386 -x testing iris head dev_ethernetff
23678 ?        S      0:06      \_ /bin/sh 
/home/iovbuild/tools/nightly/build_node.sh -x --branches testing iris head 
dev_ethernetff nimbra680
23986 ?        R      0:00          \_ /bin/bash 
/lhome/iovbuild/source_i386/head/sw/mksupp/nsetup.sh -q 
/lhome/iovbuild/source_i386/head/sw/configs/nimbra680_debug.config
24001 ?        R      0:00          \_ ps fx
            ls -ld ./* /tmp/b* :
lrwxrwxrwx 1 iovbuild iovbuild  44 2008-11-10 04:31 ./ark-gpio -> 
/lhome/iovbuild/source_i386/head/sw/ark-gpio
drwxr-xr-x 6 labbon   users    120 2008-10-21 12:42 /tmp/b_tst
drwxr-xr-x 4 seved    users    260 2008-11-07 15:04 /tmp/build.11287
drwxr-xr-x 2 seved    users    120 2008-11-07 15:00 /tmp/build.12360
drwxrwxr-x 7 iovbuild iovbuild 300 2008-10-31 01:33 /tmp/build.13412
drwxrwxr-x 7 iovbuild iovbuild 300 2008-10-31 01:14 /tmp/build.14263
drwxrwxr-x 7 iovbuild iovbuild 300 2008-10-31 01:24 /tmp/build.1488
drwxr-xr-x 4 seved    users    260 2008-11-07 15:08 /tmp/build.15704
drwxrwxr-x 7 iovbuild iovbuild 300 2008-10-31 01:27 /tmp/build.17242
drwxrwxr-x 7 seved    users    280 2008-11-01 14:43 /tmp/build.1746
drwxrwxr-x 7 seved    users    300 2008-10-31 11:42 /tmp/build.18657
drwxrwxr-x 7 iovbuild iovbuild 300 2008-10-31 01:22 /tmp/build.2178
drwxrwxr-x 2 iovbuild iovbuild  60 2008-11-10 04:31 /tmp/build.23678
drwxr-xr-x 4 seved    users    260 2008-11-07 15:05 /tmp/build.23830
drwxrwxr-x 7 iovbuild iovbuild 300 2008-10-31 01:19 /tmp/build.24058
drwxr-xr-x 2 seved    users    120 2008-11-07 15:02 /tmp/build.27328
drwxrwxr-x 7 iovbuild iovbuild 300 2008-10-31 01:30 /tmp/build.29646
drwxrwxr-x 7 iovbuild iovbuild 300 2008-10-31 01:32 /tmp/build.7778
drwxr-xr-x 4 seved    users    260 2008-11-07 15:08 /tmp/build.9048

#########################################################################################
# cpuinfo and meminfo 

jessie:~> cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 15
model name      : Intel(R) Core(TM)2 Quad CPU    Q6600  @ 2.40GHz
stepping        : 11
cpu MHz         : 2400.092
cache size      : 4096 KB
physical id     : 0
siblings        : 4
core id         : 0
cpu cores       : 4
fpu             : yes
fpu_exception   : yes
cpuid level     : 10
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov 
pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm 
constant_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr lahf_lm
bogomips        : 4804.35
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

processor       : 1
vendor_id       : GenuineIntel
cpu family      : 6
model           : 15
model name      : Intel(R) Core(TM)2 Quad CPU    Q6600  @ 2.40GHz
stepping        : 11
cpu MHz         : 2400.092
cache size      : 4096 KB
physical id     : 0
siblings        : 4
core id         : 1
cpu cores       : 4
fpu             : yes
fpu_exception   : yes
cpuid level     : 10
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov 
pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm 
constant_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr lahf_lm
bogomips        : 4800.18
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

processor       : 2
vendor_id       : GenuineIntel
cpu family      : 6
model           : 15
model name      : Intel(R) Core(TM)2 Quad CPU    Q6600  @ 2.40GHz
stepping        : 11
cpu MHz         : 2400.092
cache size      : 4096 KB
physical id     : 0
siblings        : 4
core id         : 2
cpu cores       : 4
fpu             : yes
fpu_exception   : yes
cpuid level     : 10
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov 
pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm 
constant_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr lahf_lm
bogomips        : 4800.19
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

processor       : 3
vendor_id       : GenuineIntel
cpu family      : 6
model           : 15
model name      : Intel(R) Core(TM)2 Quad CPU    Q6600  @ 2.40GHz
stepping        : 11
cpu MHz         : 2400.092
cache size      : 4096 KB
physical id     : 0
siblings        : 4
core id         : 3
cpu cores       : 4
fpu             : yes
fpu_exception   : yes
cpuid level     : 10
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov 
pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm 
constant_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr lahf_lm
bogomips        : 4800.45
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

jessie:~> cat /proc/meminfo
MemTotal:      8184084 kB
MemFree:         80584 kB
Buffers:       1954152 kB
Cached:         817544 kB
SwapCached:          0 kB
Active:        1278720 kB
Inactive:      1564872 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:      8184084 kB
LowFree:         80584 kB
SwapTotal:     9767512 kB
SwapFree:      6234136 kB
Dirty:             136 kB
Writeback:           0 kB
AnonPages:       72264 kB
Mapped:          20320 kB
Slab:          5190856 kB
PageTables:       5592 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  13859552 kB
Committed_AS:  3837324 kB
VmallocTotal: 34359738367 kB
VmallocUsed:      4976 kB
VmallocChunk: 34359733379 kB
jessie:~>




reply via email to

[Prev in Thread] Current Thread [Next in Thread]