[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Bug-wget] LIFO instead of FIFO in directory downloads

From: Robins Tharakan
Subject: [Bug-wget] LIFO instead of FIFO in directory downloads
Date: Thu, 13 Nov 2008 22:33:18 +0530


I have an odd problem, something about which although I have a solution, I thought I'd discuss first before working on a patch.

Short Version:
Is it worth the effort to submit a patch that accepts a command line option to download the contents of a folder in LIFO rather than FIFO format ?

Long Version:
I have to keep a folder in sync with a client's FTP folder, which has new additions to the folder every minute (its a data feed). Now, at any instant, the latest file is of highest significance, and therefore I use wget in recursive mode to fetch this folder (it doesn't have any sub-folders).

The problem is that this is a government server and heavily burdened, which means that downloads sometimes reach 400 bytes/s or even lower or worse, gets stalled ! Now to counter this, even if I run cron jobs of wget every ten minutes, I have rarely ever caught up to get that 'last file' ... until of course, its late in the night.

The solution to my problem is a simple 'LIFO' switch which allows wget to fetch the contents of a folder in the reverse order, and thereby, (at least at the beginning of every cron job would) ensure that I have the latest file on my server, at the required instant.

Now I already am using a hacked LIFO version of wget for myself, but do you think its worth the time to actually give this feature a command line parameter and submit a patch for it ?

Robins Tharakan

reply via email to

[Prev in Thread] Current Thread [Next in Thread]