Re: [Bug-wget] [PATCH] New option: --rename-output: modify output filename with perl

[Ángel González]

On 01/08/13 21:57, Dagobert Michelsen wrote:
>  From a packagers perspective this is a nightmare because this feature
> introduces weak dependencies to programs which need to be in PATH.
> If PATH includes only tools shipped by the system it may be necessary
> to explicitly set the path for each of the tools to have the switches
> you use (for example /usr/bin/sed on Solaris does not have -u) or disable
> them during configure time when the tools are not available. The exact
> locations need to adjustable during configure for each of the tools you
> use. PCRE support adds an additional library dependency to wget.
>
> All this added complexity seems highly overengineered for a feature
> that is not in the core functionality of the tool and that only a
> fraction of the users use. Keep in mind: a good tool is one that does
> a single job right.
PCRE is already an optional wget dependency.
Which is why my first reaction to the perl proposal was "Why not use pcre
for that?" ;)

Although I agree being able to use a custom binary for setting the filename
would be cool.



Micah Cowan wrote:
> The only system shell we might have to
> deal with having a truly different quoting syntax, would be Windows
> command shell, in the event that we port this feature for the Windows
> version (I imagine the implementation for piping to shell processes
> would be necessarily different, so wouldn't be immediately supported, if
> ever).

I would recomment using posix_spawnp(), which is much more easily 
mapeable to
Windows CreateProcess() than fork() + execvp, but otherwise creating a 
pipe in a
subprocess is not really complex in Windows.
The only problem comes from the quoting rules, that are crazy. You can't 
even
call mscrt's execv() and have the child process receive the same 
parameters you
sent! (for parameters containing quotes).


About the how-to-communicate-processes question, I would prefer to keep 
things
simple rather than requiring to parse a http-like protocol for a rename.
My preference would be that the subprocess gets passed the filename as 
parameter
and -if doing something complex- can create the file itself, storing it 
from a passed fd.
If the filter needs extra headers, they can be obtained by passing 
--save-headers to wget
(and optionally stripping them in the filter). Another feature would be 
that the filter
could then store the files somewhere else (such a database).
I would have to check the cases where wget uses the new filename if the 
filter could
handle the full file storage. Although I suppose we could define a 
"none" answer for
"do not use this filename at all" and return the final filename in the 
other cases.

Regards