[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Libcdio-devel] [RFC] New API iso9660_statv2_t as API/ABI compatible
From: |
Pete Batard |
Subject: |
Re: [Libcdio-devel] [RFC] New API iso9660_statv2_t as API/ABI compatible way to read files >= 4 GiB |
Date: |
Mon, 9 Jul 2018 12:24:42 +0100 |
User-agent: |
Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.9.0 |
On 2018.07.09 08:33, Thomas Schmitt wrote:
* Never break the API (source code compatibility) of a library.
Unfortunately my findings are that, after a while, experienced
developers come to their senses and realize that focusing too hard on
having an immutable API hurts *EVERYONE*, and especially the end users.
Even if it took them a while, it seems that the kernel developer have
started realizing this as well (even if, IMO, they are still some way to
full enlightenment): See
https://www.kernel.org/doc/html/latest/process/stable-api-nonsense.html
That's not to say that API breakage is to be taken lightly, because that
is not at all the argument I am trying to make, and I really have to
stress out that I am not using the points I am making below as any kind
of justification to be cavalier about API breakage (though, as far as
I'm concerned, ABI is a different matter). But my view is that we're all
in this together (meaning upstream & downstream developers as well as
end-users) and that we must strive to develop software that will benefit
the whole population, and _not just a few privileged individuals_.
As such, the idea that upstream should _never_ break an API, because
there might be downstream developers who can't/won't (be happy to)
update their software is actually a very dangerous idea to try to
enforce, because, in general, you are breaking the API with the goal of
benefiting the end users (otherwise, it would make very little sense to
do so), which you have to consider is a much larger population than the
downstream developers, and are the group who have the more to benefit.
So, in essence, by withholding API changes because downstream developers
might be unhappy with them, you are effectively *hurting* the end-users,
who won't see changes that are beneficial for them being enacted.
Well, considering that our goal is to make the best software we can,
within the time and resource constraints we have, we need to consider
that we are all in this together, and that, yes, one group may have to
push another group to go through something they would rather avoid, _so
that the whole population benefits_. This may mean avoiding an upstream
library going through a hefty alteration of much of its stable code,
when we can very much minimize these changes (and ensure that we are
unlikely to introduce new issues) even if it comes at the cost of
breaking its ABI.
Also, please please bear in mind that I am very much seeing this as a
two way street (because, we're all in this together), meaning that, when
downstream developers come to upstream because they have found a problem
that is hurting their development process, and ultimately their
end-users, then upstream should be flexible enough to consider doing
work to alleviate these issues, even if they can justify ignoring said
issue as well as the unwanted additional work, altogether.
This is how software development should work: not as groups whose needs
seem to oppose the needs of the other group, bus as groups who realize
that we are all part of a larger group, and that it means that, from
time to time, part of the group has to take on extra work, even if, in
the absolute, they are entitled to make a point they should not have to
do said work, for the benefit of the larger population.
So, in essence, what I am saying is: we need to stop treating downstream
developers & package maintainers like porcelain dolls. Yes,
_occasionally_, we will place additional burdens on them, and we expect
them to come to accept them in their stride. Just like we also expect
them to place some burden "against" upstream from time to time, when
they feel they have grounds to do so, that we should not reject with an
all-to-easy "this is an annoyance that we can make a case against, so we
will simply dismiss it wholesale", especially if we know that this extra
work is going to help downstream (and ultimately the end users).
Also please bear in mind that, as a downstream developer, I very much
used to sit on the other side of the fence, and bark at the idea of any
upstream libraries breaking their APIs.
Then, one day, as I was using a library that had *really* broken its
whole API from head to toe (I believe it was LUFA, an USB stack for
embedded ARM), in such a way that I had to completely re-learn the
interfaces and update almost every function I called in my code, and
once a couple weeks had elapsed, I realized that barking at API breakage
was nonsense. Sure, I was still pretty annoyed that I'd had to rewrite
my code, but then it dawned on me that simply taking these breaking
changes in their stride achieved two very important goals:
1. Ensure that "mistakes" or poor initial design decisions from upstream
were corrected as needed, which ultimately would lead to my own software
being easier to develop and maintain (even if it meant a new major
effort right now) as well as a providing general improvements to the end
users.
2. Ensure that my software had not become "stale" and/or in pass of
becoming an untouched "liability", but instead was being actively
maintained, _as all actively used software should be_, which is the
point I touched on earlier. Again, the weeding out "stale" software is
something that I believe is very beneficial for end users, yet something
that is so difficult to enforce, so much so that we might as well use
API/ABI breakage as a means to enact it (but of course, _with restraint_
and only when we feel have otherwise good reasons to update an API/ABI)
This is quite a necessity for all distro maintainers, who have to
build software which they do not know in depth.
Yet, we're all in this together. So, while I have a lot of sympathy for
distro maintainers (I genuinely do - It's a much more thankless job than
being a software application developer), I will assert that if their
goal is provide the best package for their end users, as well as ensure
that developers of existing and new software using said package is
developed in the best possible way, they will acknowledge that, if the
upstream developers of a package want to push a change that will require
extra work of them, they usually have a very good reason to do so (which
I do argue we do have, in the ABI breakage option), and that, in turn,
they can feel assured that we are going to turn a sympathetic ear to
their requests when they have some (such as, as I have also been
proposing, not introducing a breaking ABI change in a library right
after we went through an API change, but instead giving them enough
breathing space and trying to wait until we can group this change with
something else).
It is also a courtesy towards the developers of applications who
would else be forced to change their code without own motivation.
There is courtesy. And then there's treating one group of people as
being "more equal" than the others, whilst ignoring the drawbacks of
doing so, for the end users.
I only have one objective in mind here, which is to ensure end users
always get the best software they possibly can. And as I have no
explained may times over, achieving that goal may mean having to nudge
developers, who might still fail to see the bigger scheme, into
participating in that goal, even if they would prefer not to.
* Try hard to keep the ABI usable for old programs which have been
compiled to use an older version of the dynamic library.
This is a courtesy towards the users of applications. They can carry
them to more modern versions of their operating system without the
need for recompilation (which might be not trivial).
Strong disagreement on this one.
If you do that, then users of the application find that they can't
process the blackarch-linux ISO at all (even worse, what they will find
is that the software they use seems to process it just fine, but
silently "forgot" to copy a couple GB of data over - ouch!!).
The courtesy here, is to ensure that end-users don't get in a situation
where their old application corrupts data without the slightest warning.
And that is what is likely to happen if we spend a lot of work ensuring
that nobody realizes that, yes, they do need to start caring about
multiextent in their code. Heck, the first thing I did when I realize
that libcdio was missing multiextent support, resulting in some of my
users getting truncated data when transfering an ISO to USB was uttering
something like "Wow! As a stable ISO-9660 library that has existed for
years, this is a major letdown if it doesn't even warn you that it's
just skipping over large amount of crucial data..."
So I'd rather annoy quite a few users with finding that their old
application doesn't work, even if they aren't going to process
multiextent images, than give my blessing to a change that will result
in even a single person losing valuable data when they copy a unique
multiextent backup they created from one medium to another and get a
notice from their application that the copy was A-OK, without realizing
that they might just as well have issued "rm -rf
./my_valuable_backup.tar.gz"...
* Design the API/ABI for being expandable without conflicting with the
first two rules.
Pious goal. But realistically unachievable in my book.
This ideal is quite common in the plumbing layers of GNU/Linux.
Yet, as I pointed out, the kernel driver maintainers seem to have a very
different opinion. So I would say, that, no, contrary to what you
assert, this ideal is not common in the actual plumbing layers of
GNU/Linux, and that some developers are rallying to the idea that we are
all in this together, for the benefit of end-users, and that, as a
result, while we should of course strive not to treat API/ABI breakage
in a cavalier fashion, API/ABI breakage is something we need to stop
pretending is hurtful, whereas it helps make software better for end users.
I blame the short breath of GNU/Linux GUI developers on that porcellain
fragility.
Perhaps. But if that's the case, I don't believe it is "right" for any
developer to bring the whole boat down for users because they don't want
to plug the holes they are supposed to be responsible for. Nor should we
have to build a very costly second hull around our boat (which is what I
would assert we're proposing to do here), so that we let the people who
don't want to plug holes have their way...
We're all in this together.
Regards,
/Pete
- Re: [Libcdio-devel] [RFC] New API iso9660_statv2_t as API/ABI compatible way to read files >= 4 GiB, (continued)
- Re: [Libcdio-devel] [RFC] New API iso9660_statv2_t as API/ABI compatible way to read files >= 4 GiB, Thomas Schmitt, 2018/07/10
- Re: [Libcdio-devel] [RFC] New API iso9660_statv2_t as API/ABI compatible way to read files >= 4 GiB, Rocky Bernstein, 2018/07/10
- Re: [Libcdio-devel] [RFC] New API iso9660_statv2_t as API/ABI compatible way to read files >= 4 GiB, Pete Batard, 2018/07/10
- Re: [Libcdio-devel] [RFC] New API iso9660_statv2_t as API/ABI compatible way to read files >= 4 GiB, Thomas Schmitt, 2018/07/10
- Re: [Libcdio-devel] [RFC] New API iso9660_statv2_t as API/ABI compatible way to read files >= 4 GiB, Thomas Schmitt, 2018/07/11
- Re: [Libcdio-devel] [RFC] New API iso9660_statv2_t as API/ABI compatible way to read files >= 4 GiB, Thomas Schmitt, 2018/07/13
- Re: [Libcdio-devel] [RFC] New API iso9660_statv2_t as API/ABI compatible way to read files >= 4 GiB,
Pete Batard <=
- Re: [Libcdio-devel] [RFC] New API iso9660_statv2_t as API/ABI compatible way to read files >= 4 GiB, Thomas Schmitt, 2018/07/09
- Re: [Libcdio-devel] [RFC] New API iso9660_statv2_t as API/ABI compatible way to read files >= 4 GiB, Rocky Bernstein, 2018/07/09
- Re: [Libcdio-devel] [RFC] New API iso9660_statv2_t as API/ABI compatible way to read files >= 4 GiB, Thomas Schmitt, 2018/07/10