[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Libcdio-devel] [RFC] New API iso9660_statv2_t as API/ABI compatible

From: Pete Batard
Subject: Re: [Libcdio-devel] [RFC] New API iso9660_statv2_t as API/ABI compatible way to read files >= 4 GiB
Date: Mon, 9 Jul 2018 12:24:42 +0100
User-agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.9.0

On 2018.07.09 08:33, Thomas Schmitt wrote:
* Never break the API (source code compatibility) of a library.

Unfortunately my findings are that, after a while, experienced developers come to their senses and realize that focusing too hard on having an immutable API hurts *EVERYONE*, and especially the end users.

Even if it took them a while, it seems that the kernel developer have started realizing this as well (even if, IMO, they are still some way to full enlightenment): See

That's not to say that API breakage is to be taken lightly, because that is not at all the argument I am trying to make, and I really have to stress out that I am not using the points I am making below as any kind of justification to be cavalier about API breakage (though, as far as I'm concerned, ABI is a different matter). But my view is that we're all in this together (meaning upstream & downstream developers as well as end-users) and that we must strive to develop software that will benefit the whole population, and _not just a few privileged individuals_.

As such, the idea that upstream should _never_ break an API, because there might be downstream developers who can't/won't (be happy to) update their software is actually a very dangerous idea to try to enforce, because, in general, you are breaking the API with the goal of benefiting the end users (otherwise, it would make very little sense to do so), which you have to consider is a much larger population than the downstream developers, and are the group who have the more to benefit.

So, in essence, by withholding API changes because downstream developers might be unhappy with them, you are effectively *hurting* the end-users, who won't see changes that are beneficial for them being enacted.

Well, considering that our goal is to make the best software we can, within the time and resource constraints we have, we need to consider that we are all in this together, and that, yes, one group may have to push another group to go through something they would rather avoid, _so that the whole population benefits_. This may mean avoiding an upstream library going through a hefty alteration of much of its stable code, when we can very much minimize these changes (and ensure that we are unlikely to introduce new issues) even if it comes at the cost of breaking its ABI.

Also, please please bear in mind that I am very much seeing this as a two way street (because, we're all in this together), meaning that, when downstream developers come to upstream because they have found a problem that is hurting their development process, and ultimately their end-users, then upstream should be flexible enough to consider doing work to alleviate these issues, even if they can justify ignoring said issue as well as the unwanted additional work, altogether.

This is how software development should work: not as groups whose needs seem to oppose the needs of the other group, bus as groups who realize that we are all part of a larger group, and that it means that, from time to time, part of the group has to take on extra work, even if, in the absolute, they are entitled to make a point they should not have to do said work, for the benefit of the larger population.

So, in essence, what I am saying is: we need to stop treating downstream developers & package maintainers like porcelain dolls. Yes, _occasionally_, we will place additional burdens on them, and we expect them to come to accept them in their stride. Just like we also expect them to place some burden "against" upstream from time to time, when they feel they have grounds to do so, that we should not reject with an all-to-easy "this is an annoyance that we can make a case against, so we will simply dismiss it wholesale", especially if we know that this extra work is going to help downstream (and ultimately the end users).

Also please bear in mind that, as a downstream developer, I very much used to sit on the other side of the fence, and bark at the idea of any upstream libraries breaking their APIs.

Then, one day, as I was using a library that had *really* broken its whole API from head to toe (I believe it was LUFA, an USB stack for embedded ARM), in such a way that I had to completely re-learn the interfaces and update almost every function I called in my code, and once a couple weeks had elapsed, I realized that barking at API breakage was nonsense. Sure, I was still pretty annoyed that I'd had to rewrite my code, but then it dawned on me that simply taking these breaking changes in their stride achieved two very important goals:

1. Ensure that "mistakes" or poor initial design decisions from upstream were corrected as needed, which ultimately would lead to my own software being easier to develop and maintain (even if it meant a new major effort right now) as well as a providing general improvements to the end users.

2. Ensure that my software had not become "stale" and/or in pass of becoming an untouched "liability", but instead was being actively maintained, _as all actively used software should be_, which is the point I touched on earlier. Again, the weeding out "stale" software is something that I believe is very beneficial for end users, yet something that is so difficult to enforce, so much so that we might as well use API/ABI breakage as a means to enact it (but of course, _with restraint_ and only when we feel have otherwise good reasons to update an API/ABI)

   This is quite a necessity for all distro maintainers, who have to
   build software which they do not know in depth.

Yet, we're all in this together. So, while I have a lot of sympathy for distro maintainers (I genuinely do - It's a much more thankless job than being a software application developer), I will assert that if their goal is provide the best package for their end users, as well as ensure that developers of existing and new software using said package is developed in the best possible way, they will acknowledge that, if the upstream developers of a package want to push a change that will require extra work of them, they usually have a very good reason to do so (which I do argue we do have, in the ABI breakage option), and that, in turn, they can feel assured that we are going to turn a sympathetic ear to their requests when they have some (such as, as I have also been proposing, not introducing a breaking ABI change in a library right after we went through an API change, but instead giving them enough breathing space and trying to wait until we can group this change with something else).

   It is also a courtesy towards the developers of applications who
   would else be forced to change their code without own motivation.

There is courtesy. And then there's treating one group of people as being "more equal" than the others, whilst ignoring the drawbacks of doing so, for the end users.

I only have one objective in mind here, which is to ensure end users always get the best software they possibly can. And as I have no explained may times over, achieving that goal may mean having to nudge developers, who might still fail to see the bigger scheme, into participating in that goal, even if they would prefer not to.

* Try hard to keep the ABI usable for old programs which have been
   compiled to use an older version of the dynamic library.

   This is a courtesy towards the users of applications. They can carry
   them to more modern versions of their operating system without the
   need for recompilation (which might be not trivial).

Strong disagreement on this one.

If you do that, then users of the application find that they can't process the blackarch-linux ISO at all (even worse, what they will find is that the software they use seems to process it just fine, but silently "forgot" to copy a couple GB of data over - ouch!!).

The courtesy here, is to ensure that end-users don't get in a situation where their old application corrupts data without the slightest warning. And that is what is likely to happen if we spend a lot of work ensuring that nobody realizes that, yes, they do need to start caring about multiextent in their code. Heck, the first thing I did when I realize that libcdio was missing multiextent support, resulting in some of my users getting truncated data when transfering an ISO to USB was uttering something like "Wow! As a stable ISO-9660 library that has existed for years, this is a major letdown if it doesn't even warn you that it's just skipping over large amount of crucial data..."

So I'd rather annoy quite a few users with finding that their old application doesn't work, even if they aren't going to process multiextent images, than give my blessing to a change that will result in even a single person losing valuable data when they copy a unique multiextent backup they created from one medium to another and get a notice from their application that the copy was A-OK, without realizing that they might just as well have issued "rm -rf ./my_valuable_backup.tar.gz"...

* Design the API/ABI for being expandable without conflicting with the
   first two rules.

Pious goal. But realistically unachievable in my book.

This ideal is quite common in the plumbing layers of GNU/Linux.

Yet, as I pointed out, the kernel driver maintainers seem to have a very different opinion. So I would say, that, no, contrary to what you assert, this ideal is not common in the actual plumbing layers of GNU/Linux, and that some developers are rallying to the idea that we are all in this together, for the benefit of end-users, and that, as a result, while we should of course strive not to treat API/ABI breakage in a cavalier fashion, API/ABI breakage is something we need to stop pretending is hurtful, whereas it helps make software better for end users.

I blame the short breath of GNU/Linux GUI developers on that porcellain

Perhaps. But if that's the case, I don't believe it is "right" for any developer to bring the whole boat down for users because they don't want to plug the holes they are supposed to be responsible for. Nor should we have to build a very costly second hull around our boat (which is what I would assert we're proposing to do here), so that we let the people who don't want to plug holes have their way...
We're all in this together.



reply via email to

[Prev in Thread] Current Thread [Next in Thread]