[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: LYNX-DEV Since Lynx won't, what will?
From: |
Jim Dennis |
Subject: |
Re: LYNX-DEV Since Lynx won't, what will? |
Date: |
Thu, 07 Aug 1997 18:07:09 -0700 |
> Jim Dennis describes his script for doing a mirror type function with
> Lynx and -crawl, etc., in response to my message last week wanting a
> program to do something similar.
Please feel free to let me know if you use my script and find
any bugs in it.
> Jim, I agree that it just doesn't make sense to add this kind of
> functionality to Lynx when there are already tools out there that do
> such things, and do them quite well. (After all, we don't want Lynx
> to have everything, including the kitchen sink, like emacs has done!)
Umm! That's not what I said. I'm the one who originally
posted the request TO ADD the -mirror function.
My script was a feasibility study and prototype to show how
this might be done. I'm hoping to tickle some coder's fingers
into integrating my design into C code. Maybe I'll bite the
bullet and work on it myself (though my C coding skills aren't
yet up to the task -- so it will take me ten times as long as
it would for a "real" programmer).
On the one hand I agree about "freaping creaturism" and on the
other I balance the benefit against the cost. One the one
hand the -crawl and -traversal switches are "features" without
specific benefits attributed to them. (I use -traversal to
detect errors, and obviously I use it as part of my
"lynxmirror" script; I have used -crawl to create or update a
set of "infobot" pages for procmail to avoid ugly little
"dynamic fetch" recipes -- but I don't know what the
authors/contributors of these functions had in mind).
My conclusion (from this feasibility test) is that the amount
of extra code would be minimal and the added functionality
would be considerable. In part I'm thinking in terms of
"marketing" Lynx to a core set of webmasters and sysadmins.
My hope is that a number of webmasters and sysadmins will use
Lynx as their prefered "web automation" tool and will
consequently put furhter pressure on their designers to have
Lynx friendly sites. The benefit I receive from this is that
I can work with my preferred tools for browsing as a side
effect. (I like keyboards and text mode and don't link
electronic rodents and "squinting").
> Go to one of the GNU sites and get a copy of wget. It is an excellent
> program, has lots of options to tailor your usage to exactly what
> you want to do, and works very well for mirroring a site or a portion
> thereof. You CAN tell it to fetch just particular types of files, or
> not to fetch certain types. You can tell it to traverse to a limited
> depth or recurse all the way. You can tell it not to follow links to
> parent directories, to ignore certain directory or file name patterns,
> and much more.
> Scott
I'm sure that wget is an excellent tool. I'll probably *have*
to get a copy of it for other reasons. However it doesn't
serve some of my other purposes and I still think a -mirror
switch is a "good thing" to add to Lynx. If no one agrees
with me then either I'll find the time to climb the C hill and
add it myself (and maybe nobody will include that patch in
there code) or it will never get done.
If I do implement a -mirror set of patches I'll try for the
following behavior: -mirror [path] [URL] will work essentially
like -traversal and my script (except that it wouldn't save
the .dat files unless you also include the -traversal switch.
If you left off the URL I'd expect a set of URL's from <stdin>
which I would assume was a previously created (and presumably
filtered or edited) traverse.dat (the mirror would then only
fetch those times listed and not do a further traversal. If
you didn't include a path it would default to the corrent
directory. If it found filename collisions it would rename
the old files with emacs style "versioning"
Finer granularity of control would not be directly available
with my proposed set of additions. However it could be
written so that the API is available -- so that one could
create a UI for it (beyond the implicit UI of the command line
switches).
One of the other enhancement requests I've seen at browser.org
is the one calling for some sort of embedded scripting
language. I followed that up with the recommendation that we
TCL for this task (since TCL is specifically designed to be
embedded). I've also recommended that we look at ctk (curses
tk) since it's designed to work with TCL and we could
seriously benefit from the layout manager for use in tables
(among other things).
An embedded TCL would be ideal for controlling the mirroring
features. In addition would be neat to see Safe-TCL quietly
succeed in delivering on some the promises that Java has
hyped.
(That's what I like about GNU software -- as close to working
anarchy as I've ever seen).
--
Jim Dennis (800) 938-4078 address@hidden
Proprietor, Starshine Technical Services: http://www.starshine.org
PGP 1024/2ABF03B1 Jim Dennis <address@hidden>
Key fingerprint = 2524E3FEF0922A84 A27BDEDB38EBB95A
;
; To UNSUBSCRIBE: Send a mail message to address@hidden
; with "unsubscribe lynx-dev" (without the
; quotation marks) on a line by itself.
;
- Re: LYNX-DEV Since Lynx won't, what will?, Jim Dennis, 1997/08/07
- Re: LYNX-DEV Since Lynx won't, what will?, Scott McGee (Personal), 1997/08/07
- Re: LYNX-DEV Since Lynx won't, what will?,
Jim Dennis <=
- Re: LYNX-DEV Since Lynx won't, what will?, David Woolley, 1997/08/09
- Re: LYNX-DEV Since Lynx won't, what will?, Jonathan Sergent, 1997/08/20
- Re: LYNX-DEV Since Lynx won't, what will?, Larry W. Virden, x2487, 1997/08/20
- Re: LYNX-DEV Since Lynx won't, what will?, Foteos Macrides, 1997/08/09
- Re: LYNX-DEV Since Lynx won't, what will?, Jim Dennis, 1997/08/10