Re: lynx-dev Dump_Output_Immediately / Cookie bug not completely fixed (AFAIK)

[Klaus Weide]

On Wed, 16 Dec 1998, Elwin M. Oost wrote:

> Hi,
> 
> I think the bug is not completely fixed yet; here's why:
> 
> I use Lynx to download pages one at a time (using -source). In a redirecting
> session, everything works fine. However, even though I compiled with
> the Persistent-Cookie option and turned it on in lynx.cfg (as well as 
> accept_all_domains etc.), the cookies in a dump_immediately session are not
> saved to disk!
> 
> (Using -trace I noticed the cookies were accepted by Lynx, and the options
> page indeed lists the permanent cookies switch turned on)

Not all dump_output_immediately sessions are equal...

Brian added a call to LYStoreCookies() after mainloop() in main(),
when he recently changed LYCookie.c to work in dump_output_immediately
mode.  As you observed, most of cookie handling works with
dump_output_immediately after those changes.  (This doesn't have
anything to do with my recent patches btw.)  But in some cases the
LYStoreCookies() is never reached.  mainloop() doesn't always return.
mainloop() or some function called from it may exit() directly.

The one exit(0) I am aware of where this happens in the normal (non-error)
case is at the end of HTFWriter_free() in HTFWriter.c.  I have never
understood why that `if (dump_output_immediately)' at the end of
HTFWriter_free is necessary - maybe it isn't.  It's kind of a sneaky
way for lynx to exit, from within a stream method.  You could try to see
what happens if it is removed.  Let the list know of your results.

It depends on various factors whether the dump_output_immediately
output actually goes through HTFWriter: which of the flags -dump,
-source, -mime_header, -post_data was used (or which combination);
some other flags may change the behavior (-base, -preparsed come to mind);
it also can depend on the content-type and possible compression of the
data.  Probably there are other things I forget.  It's definitely
non-obvious...

For some of those circumstances, NOT writing the cookie file may actually
be more appropriate than writing it.  For example with -mime_header,
neither HTTP headers nor META tags are parsed, so there is no possible
way that cookies can be set/changed/discarded by Set-Cookie headers.
The only possible (significant) effect on the contents of the cookie
file could be removal of expired cookies.

This reasoning partially also applies to the -source flag: HTTP headers
are parsed, but META tags are not.  So Set-Cookies embedded in HTML will
be ignored.  (But if you feel, like I do, that embedding HTTP headers in
HTML is an abomination anyway, you may not worry too much about this...)

Maybe it would make more sense if a non-interactive lynx never wrote
the cookie file (even if persistent cookies were enabled), unless
explicitly told to do so with a new flag.

> Hope this helps, please reply through email as I am not on the Lynx-Dev list.

It would help if you would subscribe at least temporarily.  Or at least
check the list archives for responses.

> Best wishes,
> 
>   Elwin (last time using other account address@hidden)

   Klaus