[Quilt-dev] Refactor quilt refresh

[Adam Hamsik]

Hi Folks, 

[ Please keep me on CC: I'm not subscribed to quilt-dev ]

I have been playing with quilt for some time and I have found that if I add, 
too many files to patch (around 300), quilt refresh can took 45seconds, even if 
there are no changes in those files which is a real problem to me. Usually I 
add more files than it's needed to my patch and later I do not need to care 
about adding file to patch before making any change. 

I have made this simple patch which adds support for some sort of hash database 
to quilt. It creates QUILT_PC/patch/.cksum file with sha1 sum of all files 
which are in patch dir. Example from my tree

address@hidden src]$ cat .pc/gdb-thread-fixes.diff/.cksum                       
                                                                                
  
4b17c0f94ebcb8500629c828a272389e1e92e860  gnu/dist/gdb6/gdb/i386bsd-nat.c
4c7b95c1c47e7e2a82562ff314720493ed80a1d6  gnu/dist/gdb6/gdb/i386nbsd-nat.c
6872730ebc5d6dac8513eb10beee574abf001e12  gnu/usr.bin/gdb6/arch/i386/defs.mk
aaac869067e3928782484304a84b6c1f1ec0c494  gnu/usr.bin/gdb6/arch/x86_64/defs.mk
0000000000000000000000000000000000000000  gnu/dist/gdb6/gdb/nbsd-thread.c
0000000000000000000000000000000000000000  gnu/dist/gdb6/gdb/nbsd-thread.h

If you can see all files which existed before (add/push) have they original 
hash stored here, those who was not existing have all 0. Now during refresh we 
use shasum -c feature to find all files which sha hash changed and we run 
diff_files only for them. For patch with 385 files this 
address@hidden src]$ export QUILT_USE_SHA1=1
address@hidden src]$ export QUILT_NO_DIFF_TIMESTAMPS=1
address@hidden src]$ q files | wc -l                                            
                                                                                
 
     490
address@hidden src]$ time quilt refresh
shasum: WARNING: 24 of 490 computed checksums did NOT match

real    0m0.260s
user    0m0.153s
sys     0m0.107s

real    0m0.044s
user    0m0.012s
sys     0m0.034s
Refreshed patch tz
    0m0.68s real     0m0.31s user     0m0.36s system

From upper comparison you can see that my patch add very little overhead to 
whole time but it can save a lot see bellow.

address@hidden src]$ unset QUILT_USE_SHA1
address@hidden src]$ export QUILT_NO_DIFF_TIMESTAMPS=1
address@hidden src]$ time q refresh 

real    0m0.073s # <- time if [ -n "$QUILT_USE_SHA1" ]; then # on line 176 in 
refresh script
user    0m0.029s
sys     0m0.044s

real    0m19.101s # <- time for file in "address@hidden" # on line 182 in 
refresh script
user    0m5.027s
sys     0m14.018s
Warning: trailing whitespace in lines 2708,2772,2781 of 
external/cddl/osnet/dist/cmd/ztest/ztest.c
Refreshed patch zt
    0m19.56s real     0m5.21s user     0m14.28s system

address@hidden src]$ q files | wc -l
     490

You can find my patch at [1]. It was made against plain HEAD. Any opinions 
would it be possible to commit it to official repository.

P.S. I would like to add some new commands for manipulation with 'series' files 
(to add list of available files, switch 'series' to different file and add new 
one.) Question is if I made this as a patch would it be committed ?

[1]http://www.netbsd.org/~haad/quilt_shasum_db.diff

Regards

Adam.