Re: [Help-smalltalk] ZLib streams

[Robin Redeker]

On Thu, Aug 23, 2007 at 03:14:20PM +0200, Paolo Bonzini wrote:
> 
> >That worked for my small example. But I've tried to implement it in my 
> >program
> >now and ran into some other problem:
> >
> >  [socket atEnd not] whileTrue: [self handleData: inputStream nextHunk]
> >
> >Seemed to block.
> 
> You cannot know if the stream is atEnd unless a) it closes, b) you have 
> one more byte.  So yes, #atEnd blocks (by design).

Thanks okay. But what I mean was that nextHunk blocked even after it
received a SYNC_FLUSHed packet from the other end. See the rest of the
mail I wrote there.

It's also fine in that loop when atEnd blocks, if it comes back after data
arrived or the stream closed or whatever, but inputStream (being (a
RawInflateStream (which I maybe missed to say, sorry)) needs to give me the
packet it got (and it actually really got a _complete_ packet, which I found
out while debugging). The problem was that atEnd (on the socket) which is
called in fillBuffer blocks _after_ the compressed data was read from the 
socket.
Which resulted in nextHunk not to give me the finished packet it got.

In fact nextHunk returned the packet when I shut down one end of the TCP 
connection
(which resulted in the atEnd in fillBuffer to return).

> I would do something like
> 
>    [
>      socket atEnd ifTrue: [ ^self ]
>      self handleData: inputStream nextHunk
>    ] whileFalse.
> 
> with #handleData: returning true when it finishes reading a packet.

handleData accumulates the data it got and maybe finished parsing none,
one, two, ..., N protocol messages. And I don't see why I would like to
end my read-loop when I got packets from the socket.

Maybe my approach of accumulating the bytes from the network in a buffer
and reparse it when more bytes arrive to detect completed packets is not
practical in smalltalk.

My logic was: "read all packets from the socket until we get a disconnect":

    [inputStream "(or socket)" atEnd not]
        whileTrue: [self handleData: inputStream nextHunk]

> >You may also note that the call to ZlibReadStream>>#processInput:size: 
> >which
> >will result in a call to RawInflateStream>>#processInput:size: will 
> >receive the
> >wrong flag (0 Z_NO_FLUSH or 4 Z_FINISH, and not 2 Z_SYNC_FLUSH) when atEnd
> >finally returns.
> 
> As far as I could see from the zlib source code, Z_SYNC_FLUSH only 
> matters for deflating.  On inflation, it only looks for Z_NO_FLUSH, 
> Z_FINISH, Z_BLOCK.

I don't know about the zlib source code, but I read the manual:
   http://www.zlib.net/manual.html#inflate

   If the parameter flush is set to Z_SYNC_FLUSH, inflate flushes as much output
   as possible to the output buffer.
And:
   However if all decompression is to be performed in a single step (a single
   call of inflate), the parameter flush should be set to Z_FINISH. In this case
   all pending input is processed and all pending output is flushed ;

The manual might be wrong, I don't know.


Robin