[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: What tricks used in readlink to make it faster than realpath bash lo
From: |
L A Walsh |
Subject: |
Re: What tricks used in readlink to make it faster than realpath bash loadable? |
Date: |
Fri, 21 Dec 2018 00:09:58 -0800 |
User-agent: |
Thunderbird |
On 12/13/2018 2:02 PM, Eric Blake wrote:
On 12/13/18 3:37 PM, Peng Yu wrote:
Hi,
`readlink` is faster than `realpath` for a large number of input
arguments. Note that the former starts slower than the latter. What
tricks is used in readlink to make it faster? Thanks.
Why don't you use strace and/or read the code to compare the two
implementations, instead of asking someone else to do it for you?
It could be that someone might already know without anyone having
to do it.
I see that usually being what people want when they ask a question
like that -- most are not asking someone else to do it Eric, they
are asking if anyone might already know the answer. If not THEN
they might follow your advice and track it down further.
The way you phrase your question implies that they are too lazy
to do it themselves, and that, like you, they are assuming no one
on this list would be knowledgable enough to know the answer off
the top of their head, which may be true, but does sorta hint
at your expectations of other list members. Personally,
I'd think someone might already have been curious about that
and checked it out, maybe not, but you don't know until you've
asked.
You did remind me of a script I wrote sometime back to
show the most used calls in an strace of something running.
(script attached -- combination of shell+perl)
It seems to echo some code unnecessarily, but at the end still gives
the 10 top time users in strace (10 was arbitrarily chosen)
Never bother to do much more with it but it hints at the reason in
this case:
wheretime realpath "-e $(printf '. %.0s' {1..50000}) >/dev/null"
realpath:
0.000345 brk(0) = 0x60b000
0.000089 close(3) = 0
0.000029 mprotect(0x609000, 4096, PROT_READ) = 0
0.000029 read(3, "", 8192) = 0
0.000029 write(2, ""..., 10realpath: ) = 10
0.000026 access("/etc/ld.so.preload", R_OK) = -1
0.000026 openat(AT_FDCWD, "/usr/lib/locale/en_US.
0.000026 openat(AT_FDCWD, "/usr/share/locale/loca
0.000025 openat(AT_FDCWD, "/lib64/libc.so.6", O_R
0.000024 arch_prctl(ARCH_SET_FS, 0x7fc5d29c9500)
wheretime readlink "-e $(printf '. %.0s' {1..50000}) >/dev/null"
readlink:
0.000319 brk(0) = 0x60a000
0.000094 close(3) = 0
0.000035 mprotect(0x608000, 4096, PROT_READ) = 0
0.000031 access("/etc/ld.so.preload", R_OK) = -1
0.000031 read(3, "", 8192) = 0
0.000029 openat(AT_FDCWD, "/etc/ld.so.cache", O_R
0.000028 write(2, ""..., 10readlink: ) = 10
0.000027 openat(AT_FDCWD, "/lib64/libc.so.6", O_R
0.000027 openat(AT_FDCWD, "/usr/lib/locale/en_US.
0.000025 arch_prctl(ARCH_SET_FS, 0x7fecd4f99500)
Looks like realpath makes more calls to malloc/alloc or similar
to allocate memory that creates a small overhead of 26 microseconds,
but more extensive testing might be required to get a better idea.
Of course I didn't do any work -- but I did happen to have a script
lying around since 2014 that did something in this area -- who knows
they might find it helpful, or not...but if they don't ask they don't
find out.
P.s. given its been over 4 years since I wrote this, I doubt I can
add any insight into what I did except by reading it ...
#!/bin/bash -x
# by Linda A. Walsh (c) 2014 -- free for use to nice people!
# (not guaranteed to do anything useful)...
p=$0
cmd="/bin/sleep 1.5"
lines=10
if [[ $# -lt 1 ]]; then
echo "wheretime \"cmd [flags]\" [opt#lines]"
echo " - w/cmd+flags in quotes -- to time cmd & flags"
echo "using default test \"$cmd\" $lines"
else
cmd="$1"
shift
fi
if [[ $# ]]; then
if [[ $1 =~ ^[0-9]+$ ]]; then lines=$1; fi
fi
#testing
#if [[ $p -nt /tmp/timings.txt ||
# /tmp/timings.cmd -nt /tmp/timings.txt ]]; then
rm -f /tmp/timings.{cmd,txt}
#fi
read last < <(if [[ -e /tmp/timings.cmd ]]; then cat /tmp/timings.cmd; else
echo ""; fi)
TIMEFORMAT="%2Rsec %2Uusr %2Ssys (%P%% cpu)"
if [[ $last != $cmd ]]; then
echo "generating timings for $cmd..."
echo "$cmd" >/tmp/timings.cmd
time strace -s 0 -r -qq -f $cmd >& /tmp/timings.txt
fi
prog='
##!/usr/bin/perl;use warnings;use strict;use P;
my ($val, $calls)=(0,{});
my $lines=15;
if ($#ARGV>=0) {
$lines=$ARGV[0] if $ARGV[0] =~ /^\d+$/;
shift;
}
while(<>) {
chomp;
if (m{((?:\[pid\s\d+\])?)\s*([\d\.]+)\s(\S+.*)$}){
my ($pid,$time,$call)=($1, $2, $3);
$call=($pid||"").$call;
if (ref $calls->{$call}) {
$calls->{$call}[0]=$calls->{$call}[0]+$time;
#P "cc1=%s", $calls->{$call}
} else {
$calls->{$call}=[$time, $call];
}
}
}
my @scalls = sort { my ($ca, $cb) = @{$calls}{$a,$b};
$cb->[0] <=> $ca->[0] ||
$ca->[1] cmp $cb->[1] } keys %$calls;
my $i;
for ($i=0; $i<$lines; ++$i) {
my $sc=$scalls[$i];
printf " %11.6f %.40s\n", @{$calls->{$sc}}[0,1];
}
# vim: ts=2 sw=2
'
perl -e "$prog" "$lines" </tmp/timings.txt
# vim: ts=2 sw=2