[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
lynx-dev Lynx.trace cookie Re: yahoo lists, every 4th message
|
From: |
Bob |
|
Subject: |
lynx-dev Lynx.trace cookie Re: yahoo lists, every 4th message |
|
Date: |
Mon, 25 Mar 2002 00:30:41 -0500 |
lynx <options> "${BASE}${URL}"
repeated returns an ad page again, using either lynx or curl.
That URL only returns the *REAL* "${BASE}${URL}",
not the ad page, when called again from within the page.
Diffing Lynx.trace files looks to me like a cookie is what
allows the second request for the same URL to be honored
with the real URL being sent by yahoo instead of the ad
page.
Here's what happens when the same URL is called for
via a link within the ad page--according to Lynx.trace.
I searched for the first "HText_pageDisplay finished"
which means lynx has the ad page loaded, then what
follows is a request for the original URL a second time
via a link on the page, but with cookie action so the
page yahoo sends the real URL instead of substituting
an ad page.
Abbreviated version of Lynx.trace for what happens
when the link for URL on fakeURL(ad page) is requested--
LYCookie: Searching for 'groups.yahoo.com:80',
'/group/AkhaWeeklyJournal/message/120'.
Checking cookie 0x8191c10
G=v=6&data=D4eF9sg97u_3y_2XV-WS1JggEat8W9StK9ojrm5Lp-rVclo45Is1kwa_S2NLAfcIKqoyWQ9w8shAn5XQPL60qT-H4N11kZj5p9Onyu-mWh1aYAanW52PtI73_D2BmQDundAwp6yODK94u8ZggRogWrm1UGee3ayC8fzUbRLkwLLuoeLw8cDg-df8_HBhvUEEKhcjH2P3flCG2btWopJHX7bRO_9I6Vws4nmAUYOd-J_gtZUrC-0yw4ehvQDmz3rDo5Bb1_itg4vB77qH_zlS4jrm0c9gY6Jw1aeU_Y_9LHDKNnIY-eWCafv4mex1B9fj-e26_PWOxlxITadhxF3qrD7DIXEBTYHKbmAdmP7Z9lOI2w8GWHhmiEvzh8oOTl-QhXh7GCbSVQWmNiqyZQIhGwswTeQbDvAFbhQx-qlljoStPm1V4S7n-OKLBRQ&n=17
groups.yahoo.com groups.yahoo.com 1
/group/AkhaWeeklyJournal/message/120 /group/AkhaWeeklyJournal 0
Checking cookie 0x8190800 interrupt=1&1017025346&120
groups.yahoo.com groups.yahoo.com 1
/group/AkhaWeeklyJournal/message/120 / 0
HTTP: Sending Cookie2: $Version ="1"
HTTP: Sending Cookie:
G=v=6&data=D4eF9sg97u_3y_2XV-WS1JggEat8W9StK9ojrm5Lp-rVclo45Is1kwa_S2NLAfcIKqoyWQ9w8shAn5XQPL60qT-H4N11kZj5p9Onyu-mWh1aYAanW52PtI73_D2BmQDundAwp6yODK94u8ZggRogWrm1UGee3ayC8fzUbRLkwLLuoeLw8cDg-df8_HBhvUEEKhcjH2P3flCG2btWopJHX7bRO_9I6Vws4nmAUYOd-J_gtZUrC-0yw4ehvQDmz3rDo5Bb1_itg4vB77qH_zlS4jrm0c9gY6Jw1aeU_Y_9LHDKNnIY-eWCafv4mex1B9fj-e26_PWOxlxITadhxF3qrD7DIXEBTYHKbmAdmP7Z9lOI2w8GWHhmiEvzh8oOTl-QhXh7GCbSVQWmNiqyZQIhGwswTeQbDvAFbhQx-qlljoStPm1V4S7n-OKLBRQ&n=17;
interrupt=1&1017025346&120
Composing Authorization for
groups.yahoo.com:80/group/AkhaWeeklyJournal/message/120
HTAASetup_lookup: No template matched `group/AkhaWeeklyJournal/message/120' (so
probably not protected)
HTTP: Not sending authorization (yet).
Writing:
GET /group/AkhaWeeklyJournal/message/120 HTTP/1.0
More context from Lynx.trace--
LYpush[0]:
address:http://groups.yahoo.com/group/AkhaWeeklyJournal/interrupt?st=2&ln=AkhaWeeklyJournal&m=1&done=%2Fgroup%2FAkhaWeeklyJournal%2Fmessage%2F120
title: Yahoo! Groups
getfile: getting http://groups.yahoo.com/group/AkhaWeeklyJournal/message/120
HTParse: aName:`http://groups.yahoo.com/group/AkhaWeeklyJournal/message/120'
relatedName:`'
HTParse: result:groups.yahoo.com
HTParse: aName:`http://groups.yahoo.com/group/AkhaWeeklyJournal/message/120'
relatedName:`'
HTParse: result:
Entered HTAnchor_findAddress
Anchor 0x818e020 with address
`http://groups.yahoo.com/group/AkhaWeeklyJournal/message/120' already exists.
HTAccess: loading document
http://groups.yahoo.com/group/AkhaWeeklyJournal/message/120
HTParse: aName:`http://groups.yahoo.com/group/AkhaWeeklyJournal/message/120'
relatedName:`file:'
HTParse: result:http
HTParse: aName:`http://groups.yahoo.com/group/AkhaWeeklyJournal/message/120'
relatedName:`'
HTParse: result:groups.yahoo.com
HTParse: aName:`http://groups.yahoo.com/group/AkhaWeeklyJournal/message/120'
relatedName:`'
HTParse: result:http
HTParse: aName:`http://groups.yahoo.com/group/AkhaWeeklyJournal/message/120'
relatedName:`'
HTParse: result:groups.yahoo.com
Looking up groups.yahoo.com
HTParseInet: parsing `groups.yahoo.com'.
LYGetHostByName: parsing `groups.yahoo.com'.
CHILD gethostbyname: 0x402bef44 { h_name = 0x8190af8
"groups1.vip.scd.yahoo.com",
h_aliases = 0x81909e0 { 0x8190acd "groups.yahoo.com", 0x0 },
h_addrtype = 2, h_length = 4,
h_addr_list = 0x8190ab0 { 0x8190b14 "66.218.66.240", 0x0 } }
CHILD fill_rehostent: 0x81549e0 { h_name = 0x8154a08
"groups1.vip.scd.yahoo.com",
h_aliases = 0x81549fc { 0x8154a22 "groups.yahoo.com", 0x0 },
h_addrtype = 2, h_length = 4,
h_addr_list = 0x81549f4 { 0x8154a04 "66.218.66.240", 0x0 } }
Read from pipe: 0x81549e0 { h_name = 0x8154a08 "groups1.vip.scd.yahoo.com",
h_aliases = 0x81549fc { 0x8154a22 "groups.yahoo.com", 0x0 },
h_addrtype = 2, h_length = 4,
h_addr_list = 0x81549f4 { 0x8154a04 "66.218.66.240", 0x0 } }
LYGetHostByName: NSL_FORK child 31457 exited, status 0x0.
End of LYGetHostByName: 0x81549e0 { h_name = 0x8154a08
"groups1.vip.scd.yahoo.com",
h_aliases = 0x81549fc { 0x8154a22 "groups.yahoo.com", 0x0 },
h_addrtype = 2, h_length = 4,
h_addr_list = 0x81549f4 { 0x8154a04 "66.218.66.240", 0x0 } }
LYGetHostByName: Resolved name to a hostent.
HTParseInet: Parsed address as port 80, IP address 66.218.66.240
Making HTTP connection to groups.yahoo.com
TCP: Error 115 in `SOCKET_ERRNO' after call to this socket's first connect()
failed.
Operation now in progress
TCP: Error 115 in `SOCKET_ERRNO' after call to this socket's first select()
failed.
Operation now in progress
HTParse: aName:`http://groups.yahoo.com/group/AkhaWeeklyJournal/message/120'
relatedName:`'
HTParse: (ABS)
HTParse: result:/group/AkhaWeeklyJournal/message/120
HTParse: aName:`http://groups.yahoo.com/group/AkhaWeeklyJournal/message/120'
relatedName:`'
HTParse: result:groups.yahoo.com
HTParse: aName:`http://groups.yahoo.com/group/AkhaWeeklyJournal/message/120'
relatedName:`'
HTParse: (ABS)
HTParse: result:/group/AkhaWeeklyJournal/message/120
HTParse: aName:`http://groups.yahoo.com/group/AkhaWeeklyJournal/message/120'
relatedName:`'
HTParse: (ABS)
HTParse: result:group/AkhaWeeklyJournal/message/120
HTParse: aName:`http://groups.yahoo.com/group/AkhaWeeklyJournal/message/120'
relatedName:`'
HTParse: result:groups.yahoo.com
LYCookie: Searching for 'groups.yahoo.com:80',
'/group/AkhaWeeklyJournal/message/120'.
Checking cookie 0x8191c10
G=v=6&data=D4eF9sg97u_3y_2XV-WS1JggEat8W9StK9ojrm5Lp-rVclo45Is1kwa_S2NLAfcIKqoyWQ9w8shAn5XQPL60qT-H4N11kZj5p9Onyu-mWh1aYAanW52PtI73_D2BmQDundAwp6yODK94u8ZggRogWrm1UGee3ayC8fzUbRLkwLLuoeLw8cDg-df8_HBhvUEEKhcjH2P3flCG2btWopJHX7bRO_9I6Vws4nmAUYOd-J_gtZUrC-0yw4ehvQDmz3rDo5Bb1_itg4vB77qH_zlS4jrm0c9gY6Jw1aeU_Y_9LHDKNnIY-eWCafv4mex1B9fj-e26_PWOxlxITadhxF3qrD7DIXEBTYHKbmAdmP7Z9lOI2w8GWHhmiEvzh8oOTl-QhXh7GCbSVQWmNiqyZQIhGwswTeQbDvAFbhQx-qlljoStPm1V4S7n-OKLBRQ&n=17
groups.yahoo.com groups.yahoo.com 1
/group/AkhaWeeklyJournal/message/120 /group/AkhaWeeklyJournal 0
Checking cookie 0x8190800 interrupt=1&1017025346&120
groups.yahoo.com groups.yahoo.com 1
/group/AkhaWeeklyJournal/message/120 / 0
HTTP: Sending Cookie2: $Version ="1"
HTTP: Sending Cookie:
G=v=6&data=D4eF9sg97u_3y_2XV-WS1JggEat8W9StK9ojrm5Lp-rVclo45Is1kwa_S2NLAfcIKqoyWQ9w8shAn5XQPL60qT-H4N11kZj5p9Onyu-mWh1aYAanW52PtI73_D2BmQDundAwp6yODK94u8ZggRogWrm1UGee3ayC8fzUbRLkwLLuoeLw8cDg-df8_HBhvUEEKhcjH2P3flCG2btWopJHX7bRO_9I6Vws4nmAUYOd-J_gtZUrC-0yw4ehvQDmz3rDo5Bb1_itg4vB77qH_zlS4jrm0c9gY6Jw1aeU_Y_9LHDKNnIY-eWCafv4mex1B9fj-e26_PWOxlxITadhxF3qrD7DIXEBTYHKbmAdmP7Z9lOI2w8GWHhmiEvzh8oOTl-QhXh7GCbSVQWmNiqyZQIhGwswTeQbDvAFbhQx-qlljoStPm1V4S7n-OKLBRQ&n=17;
interrupt=1&1017025346&120
Composing Authorization for
groups.yahoo.com:80/group/AkhaWeeklyJournal/message/120
HTAASetup_lookup: No template matched `group/AkhaWeeklyJournal/message/120' (so
probably not protected)
HTTP: Not sending authorization (yet).
Writing:
GET /group/AkhaWeeklyJournal/message/120 HTTP/1.0
Host: groups.yahoo.com
Accept: text/html, text/plain, application/pdf, image/tiff, application/x-cpio,
application/x-gtar, application/x-tar,
application/x-shar, application/x-zip-compressed, application/x-stuffit,
application/mac-binhex40,
audio/pn-realaudio-plugin
Accept: application/msword, image/png, audio/x-mpeg, audio/x-realaudio,
audio/midi, audio/x-midi, audio/mod,
audio/x-mod, audio/mpeg2, audio/x-mpeg3, audio/mpeg, audio/x-sidtune,
audio/psid, audio/x-psid, audio/x-basic,
audio/x-pn-realaudio
Accept: audio/vnd.rn-realaudio, application/smil, text/vnd.rn-realtext,
video/vnd.rn-realvideo, image/vnd.rn-realflash,
application/x-shockwave-flash2-preview, application/sdp, application/x-sdp,
application/vnd.rn-realmedia,
image/vnd.rn-realpix
Accept: audio/wav, audio/x-wav, audio/x-pn-wav, audio/x-pn-windows-acm,
audio/basic, audio/x-pn-au, audio/aiff,
audio/x-aiff, audio/x-pn-aiff, video/mpeg, video/mpeg, audio/mpeg,
application/x-troff-man, application/postscript,
application/ghostview
Accept: application/pdf, image/avs, image/bie, image/x-ms-bmp, image/cmyk,
image/dcx, image/eps, image/fax, image/fits,
image/gif, image/gray, image/gradation, image/hdf, image/jpeg, image/pjpeg,
image/map, image/miff, image/mono, image/mtv
Accept: image/x-portable-bitmap, image/pcd, image/pcx, image/pdf,
image/x-portable-graymap, image/pict, image/png,
image/x-portable-anymap, image/x-portable-pixmap, image/ps, image/rad,
image/x-rgb, image/rgba, image/rla, image/rle,
image/sgi
Accept: image/sun-raster, image/targa, image/tiff, image/uyvu, image/vid,
image/viff, image/x-xbitmap, image/x-xpixmap,
image/x-xwindowdump, image/yuv, audio/mpeg, audio/midi, audio/basic,
audio/x-aiff, audio/x-gsm, audio/x-wav,
application/x-dvi
Accept: video/mpeg, application/x-ogg, application/x-msdos-program, audio/mpeg,
audio/mpegurl, audio/x-scpls,
audio/x-wav, application/postscript, application/ghostview, image/gif,
image/jpeg, image/tiff, image/x-portable-bitmap
Accept: image/x-portable-graymap, image/x-portable-pixmap, image/x-xbitmap,
image/x-xpixmap, image/bmp, image/x-bmp,
image/x-ms-bmp, image/x-rgb, image/targa, image/fits, image/png, image/pm,
image/png, image/x-ms-bmp,
image/x-cmu-raster, image/g3fax
Accept: image/gif, image/jpeg, image/targa, image/x-portable-bitmap,
image/x-portable-graymap, image/x-portable-pixmap,
image/x-rgb, image/x-xbitmap, image/x-xpixmap, text/*, application/pdf,
audio/mpeg, audio/mpeg, text/*,
application/msword
Accept: application/msword, */*, application/x-debian-package, audio/basic,
text/sgml, video/mpeg,
application/postscript, */*;q=0.01
Accept-Encoding: gzip, compress
Accept-Language: en
User-Agent: Lynx/2.8.3rel.1 libwww-FM/2.14 SSL-MM/1.4.1 OpenSSL/0.9.6
Cookie2: $Version="1"
Cookie:
G=v=6&data=D4eF9sg97u_3y_2XV-WS1JggEat8W9StK9ojrm5Lp-rVclo45Is1kwa_S2NLAfcIKqoyWQ9w8shAn5XQPL60qT-H4N11kZj5p9Onyu-mWh1aYAanW52PtI73_D2BmQDundAwp6yODK94u8ZggRogWrm1UGee3ayC8fzUbRLkwLLuoeLw8cDg-df8_HBhvUEEKhcjH2P3flCG2btWopJHX7bRO_9I6Vws4nmAUYOd-J_gtZUrC-0yw4ehvQDmz3rDo5Bb1_itg4vB77qH_zlS4jrm0c9gY6Jw1aeU_Y_9LHDKNnIY-eWCafv4mex1B9fj-e26_PWOxlxITadhxF3qrD7DIXEBTYHKbmAdmP7Z9lOI2w8GWHhmiEvzh8oOTl-QhXh7GCbSVQWmNiqyZQIhGwswTeQbDvAFbhQx-qlljoStPm1V4S7n-OKLBRQ&n=17;
interrupt=1&1017025346&120
----------------------------------
Sending HTTP request.
HTTP: WRITE delivered OK
HTTP request sent; waiting for response.
HTTP: Trying to read 1535
HTTP: Read 1448
HTTP: Rx: HTTP/1.1 200 OK
HTTP: Scanned 2 fields from line_buffer
--- Talking HTTP1.
HTTP/1.1 200 OK
HTFormat: Constructing stream stack for www/mime to www/present
HTFormat: Looking up presentation for www/mime to www/present
HTFormat: comparing text/* and www/mime for half match
HTFormat: comparing text/* and www/mime for half match
HTFormat: comparing */* and www/mime for half match
StreamStack: found strong subtype wildcard match: */*
StreamStack: found weak wildcard match: www/present
FindPresentation: found exact match: www/mime
StreamStack: found exact match: www/mime
StreamStack: Returning "MIMEParser"
HTMIME: Date: Mon, 25 Mar 2002 03:02:41 GMT
Server: Apache/1.3.3 (Unix)
Cache-control: private
Expires: Mon, 25 Mar 2002 03:36:01 GMT
Connection: close
Content-Type: text/html
<HTML>
<HEAD>
<META content="free email groups, mailing lists, communities, majordomo,
e-mail, bounce handling, mlm software,
listserv, Yahoo! Groups, newletters, announcement, email lists, list hosting"
name=keywords>
<META content="Yahoo! Groups - Free, easy email groups" name=description>
<TITLE>
Yahoo! Groups : AkhaWeeklyJournal Messages :Message 120 of 146 </TITLE>
</HEAD>
<BODY
real URL follows, not ad page substitute
Haven't been able to lynx -source and save cookies, loading
cookies from file when lynx -source again, requesting same
URL again. Yahoo might send real URL instead of ad page
on second request if cookies were stored the first time but
cookies file is empty. Looking for EXP_PERSISTENT_COOKIES
and see if yahoo sends expired cookies in first place to prevent
storing in file. Might hack LYCookies.c to store expired
cookies if so. When the going gets weird.
Is LYGlobalDefs.h a good place to define EXP_PERSISTENT_COOKIES?
; To UNSUBSCRIBE: Send "unsubscribe lynx-dev" to address@hidden
- lynx-dev Lynx.trace cookie Re: yahoo lists, every 4th message,
Bob <=