Re: Eshell's external pipe module interferes with other argument parsing hooks

[Sean Whitton]

[switching debbugs -> emacs-devel for this subthread]

Hello,

On Thu 31 Mar 2022 at 01:58PM -07, Jim Porter wrote:

> Another possibility would be to keep the current behavior (or close to
> it), but to reconstruct the command to pass to `sh' during Eshell's
> rewrite phase. I'm not quite sure if that would actually work, but if it
> did, it would allow other argument parsers to run normally without
> extpipe needing to know what parsers to try. Perhaps if we kept around
> the substring that each argument parser consumed, it would be possible
> to reconstruct the relevant bits for extpipe's purposes?

Well, in your case (2), you don't want the other parsers to get a chance
to run -- that's the whole point.

> More generally though, maybe there are really two different use cases?
>
> 1) Eshell's built-in pipelines are slow because they go through Emacs
> buffers.
>
> 2) It would be convenient to invoke a whole command (or some large part
> of a command) using `sh' syntax.

These are both things that extpipe is meant to make easy, though I'm not
sure how separate they are -- often I want both.

> For (1), Eshell could opportunistically use external pipelines without
> any special syntax. It should be possible to tell just by looking at the
> parsed command form if "foo | bar" connects two external processes on
> the same host, and then perform the appropriate rewrite to connect the
> processes efficiently (e.g. using `sh -c'). This would happen after
> expansion of variables/subcommands, so to the user it would work just
> like any other Eshell command, but it would be faster.

This could just be added to Eshell right now, right?  Definitely useful.

> For (2), we'd need a convenient syntax for forwarding some command
> string to `sh'. Something like your proposed !! or || syntax, or maybe
> something to wrap around part of a command?

Yeah, extpipe's syntax covers most such cases but not quite all of them.

> (Or maybe an even something like an interactive `eshell-externalize'
> function that replaces the selected region with the correct `sh'
> invocation?)

I had something like this previously.  The problem is it makes editing
and re-running the command a fair bit more fiddly.

> And finally, sorry for bringing up these issues months after
> bug#46351.  At the time, I didn't really understand the internals of
> Eshell, so I didn't have anything of substance to say then. Since then
> I've delved a bit *too* deep into Eshell's internals while trying to
> prove to myself that my implementation of Lisp function pipelines is
> sufficiently-flexible. :)

No problem, but could I request that you spend a little more time
editing your messages for length?  And perhaps consider separating out
discussion of significant future possible enhancements from fixing bugs
with the existing code into separate bugs or ML threads, as I've done
with this message.  Thanks in advance :)

-- 
Sean Whitton