bug#28180: [w32] Unicode characters in subprocess (git) arguments changed to space

[Eli Zaretskii]

> From: npostavs@users.sourceforge.net
> Date: Mon, 21 Aug 2017 22:35:24 -0400
> 
> In w32.c there is a comment saying
> 
>    . Running subprocesses in non-ASCII directories and with non-ASCII
>      file arguments is limited to the current codepage [...]
>      This should be fixed, but will also require changes in cmdproxy.
>      The current limitation is not terribly bad anyway, since very
>      few, if any, Windows console programs that are likely to be
>      invoked by Emacs support UTF-16 encoded command lines.
> 
> I believe we're running into this limitation with git: staging a file
> named 好.txt fails from magit[1] (I tried also with vc, same problem).
> A quick way to see the problem is evaluating the call-process form
> below, the output shows that the Chinese character has been transformed
> into a space.

I'd expect that in a non-Chinese locale (which I believe was what you
did), but the OP of the Magit issue has Windows set up for a Chinese
locale, so there has to be some other explanation, because passing
Chinese characters on the command line ought to work in that case.

> Am I correct that this problem is related the w32.c comment?

The comment is accurate, but it can only explain why command-line
arguments with characters outside of the current Windows locale cannot
be safely passed to sub-processes.  Which AFAIU is not the case with
the OP of that Magit issue.

> It's not clear to me what changes are needed in cmdproxy (and other
> places?) to address it.

cmdproxy is not involved in call-process, but it is involved in
shell-command and its ilk.  As it makes no sense to support Unicode in
the former, but not in the latter, if we want to lift this limitation,
we must teach cmdproxy to use "wide" APIs both for receiving
command-line arguments from Emacs and for passing them to programs it
invokes.

As to the "other places", the only problem I'm aware of is that the
encoding of the command-line arguments, when they arrive at w32proc.c,
is not known in advance, so this must be somehow fixed/changed,
otherwise we will be unable to re-encode them in UTF-16.  I believe
the comment in w32.c does mention that.