[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Gnu-arch-users] review/merge request: sftp get/put-file() pipe-lining

From: Scott Parish
Subject: [Gnu-arch-users] review/merge request: sftp get/put-file() pipe-lining
Date: Fri, 19 Mar 2004 05:03:44 +0000
User-agent: Mutt/


While researching something else i was struck by the realization that
file-get/put() for sftp should be pipe-lined, but i had neglected to
realize and do such earlier.

Prior sftp's get had been doing something like this:

while not eof:
  write_socket(sftp_request_read(handle, nbytes))
  # .. lag ..

With this patch it will make a bunch (16 by default) requests at a time
there-by negating the effect of lag (for large file transfers).

Testing between here and my server showed significant improvements for
get() on a 8.2mb patch:

tla get address@hidden/big--big--1.0     0.26s user 0.13s system 0% cpu 3:45.01 
tla/tla get address@hidden/big--big--1.0 0.21s user 0.10s system 0% cpu 1:54.13 

put() wasn't nearly as impressive[1], but still somewhat improved:

tla import      1.34s user 0.20s system 0% cpu 6:42.85 total
tla/tla import  1.30s user 0.16s system 0% cpu 5:42.92 total

There is some room for debate on tuning the number of requests to
pipeline(N): with get() its anticipated that there will be N-1 requests
that will return EOF, so if we are get()ing multiple tiny files this is
pure overhead. On the other hand, if there is a large lag, keeping N
sufficiently large should minimize effects of lag.


1| my dsl connection is not symetrical, so it didn't surprise me to see
   a less impressive improvement on my put() times.

Scott Parish

Attachment: pgpnuB_s4AJGO.pgp
Description: PGP signature

reply via email to

[Prev in Thread] Current Thread [Next in Thread]