Re: [PATCH v2] semihosting: add O_BINARY flag in host_open for NT compatibility

[Peter Maydell]

On Mon, 16 Jan 2023 at 15:56, <eiakovlev@linux.microsoft.com> wrote:
>
>
>
> On 1/6/23 7:58 PM, Peter Maydell <peter.maydell@linaro.org> wrote:
> > On Fri, 6 Jan 2023 at 18:22, Evgeny Iakovlev
> > <eiakovlev@linux.microsoft.com> wrote:
> > >
> > >
> > > On 1/6/2023 17:28, Peter Maydell wrote:
> > >> On Fri, 6 Jan 2023 at 15:44, Alex Bennée <alex.bennee@linaro.org> wrote:
> > >>> Peter Maydell <peter.maydell@linaro.org> writes:
> > >> I think the theory when the semihosting API was originally designed
> > >> decades ago was basically "when the guest does fopen(...) this
> > >> should act like it does on the host". So as a bit of portable
> > >> guest code you would say whether you wanted a binary or a text
> > >> file, and the effect would be that if you were running on Windows
> > >> and you output a text file then you'd get \r\n like the user
> > >> probably expected, and if on Linux you get \n.
> >
> > > If SYS_OPEN is supposed to call fopen (i didn't actually know that..)
> > > then it does make more sense for binary/text mode to be propagated from
> > > guest.
> >
> > It's not required to literally call fopen(). It just has to
> > give the specified semantics for when the guest passes it a
> > mode integer, which is defined in terms of the ISO C
> > fopen() string semantics for "r", "rb", "r+", "r+b", etc.
> >
> > > Qemu's implementation calls open(2) though, which is not correct
> > > at all then. Well, as long as qemu does that, there is no
> > > posix-compliant way to tell open(2) if it should use binary or text
> > > mode, there is no notion of that as far as posix (and most
> > > implementations) is concerned.
> >
> > QEMU doesn't have to be pure POSIX compliant: we know what our
> > supported host platforms are and we can freely use extensions
> > they provide. If we want to achieve the semantics that semihosting
> > asks for then we can do that with open(), by passing O_BINARY when
> > the mode integer from the guest corresponds to a string with "b" in it.

> Thanks Peter, i think i see your point. However, if you ask me, i feel like 
> advertising a feature to guest code and only implementing it on 1 platform 
> that supports it just because it has a non-standard POSIX implementation will 
> only confuse the issue further.

Huh? We can implement it, if we want, on *all* hosts that
we support:
 * On Windows hosts, plumb the binary indication from the
   semihosting SYS_OPEN call through to whether we pass O_BINARY
   to open(2)
 * On all other hosts, do nothing: on these hosts, text and
   binary files are identical so there is nothing to do

Note that semihosting is not an API that QEMU has specified:
it's an external one provided by multiple platforms. We do
not "advertise" the existence of the 'binary' flag to SYS_OPEN:
it is part of the pre-existing decades-old specification we
implement.

> Guest code doesn't want to care whether or not an emulator is
> running on Linux or Windows, there is no notion of that leaking
> to guest code. What it cares about is being able to consistently
> use a certain feature in their code.

The trouble here is that we have two different choices
about how to be consistent:

(1) Consistently have guests that use semihosting to open
a file in text mode get the text-mode file that they asked for,
regardless of the host operating system and its definition of
what a text file is
(2) Consistently have guest code produce a binary-identical
output file regardless of host operating system

It is not possible to have both; we have to pick one.

On balance, I agree with Alex that option (2) is probably
better, especially with the file-I/O-via-gdbstub part of it;
but we are genuinely giving up property (1) in the process.

thanks
-- PMM