Re: java.util.zip speedup

[John Leuner]

> I have taken java version of java.util.zip from CVS and done few 
> optimalizations.
> 
> I have zipped entire classpath CVS tree (5MB packed). My test program 
> goes through all entries and uncompress each to memory array (same every 
> time, preallocated). I have tested validity of changed by adding write 
> function to recreate all files - it works ok (after fixing one existing 
> bug - current version in CVS has problems with unpacking files with 0 
> length).

Did you do any profiling of the code?

> Classpath CVS version needs 3150ms. My optimized version manages to do 
> it in 2125ms (results vary +-3ms in each case). I think this is quite 
> reasonable speedup. But real fun comes with memory. Classpath CVS 
> allocates around 130MB of garbage during processing, my version  fits in 
>   1MB (most of it being zip entries/names). I suppose that most of time 
> difference comes from less gc work (but I have also made few 
> optimalizations for reading entries from disk).

That's quite an improvement. Are you saying that the current version
generates 130MB of garbage, and yours creates 1MB?

Could you send me a diff of your changes?

> I'm quite shocked to see how fast sun gc is - even if all difference 
> comes from memory, we are talking about allocating and collecting 130MB 
> in 1 second - this IS a lot. I suppose that with bigger/more complicated 
> heap, gc time, and thus difference, would increase considerably. Anyway, 
> on less advanced jvms, difference in amount of garbage should make 
> greater impact[1].
> 
> Now, the ugly part. Few of optimalizations (algorithm and IO-related) 
> are free. But biggest part (memory savings) require some caching. This 
> means that around 35kb is allocated first time zip subsystem is used and 
> will never get freed. There is also a need for quite close look at 
> inflater sharing versus security.
> 
> My question - is 35kb of memory taken 'forever' an acceptable price for 
> a lot less garbage on runtime ?

Bearing in mind that the java.util.zip implementation in Classpath is
also available as a seperate library, we should consider the needs of
people who might want to use this in an embedded system where space is
at a premium.

But obviously for modern workstations this first-time allocation is less
important than the speed/memory benefits.

John Leuner

signature.asc
Description: This is a digitally signed message part