fab-user
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: 'fabric1' and 'fabric2' packages


From: za3k
Subject: Re: 'fabric1' and 'fabric2' packages
Date: Tue, 26 May 2020 16:49:29 -0700
User-agent: Roundcube Webmail/1.3.9

+mailing list, accidentally replied to Jeff/bitprophet directly

On 2020-05-25 12:40, Jeff Forcier wrote:
In rough order:

- First, please understand I'm still kinda buried so I can't take any
serious action on my end this week. Hopefully by June.
  - FYI, my schedule at the dayjob is "open source Fridays" so most of
my output will be that day of the week.
- I think it's worth asking the distros if they would allow a Python 2
Fabric 1 option, for those users who are still stuck supporting Py2,
but like you say I would not be surprised if they've already closed
that door.
- The next big thing is evaluating 'fabric3' to see where they forked
/ how much they added besides py3 compat. Could be anything, I've no
idea. Suppose it helps that Fabric 1 dev has been nearly nil since 2
came out, but if they added a lot of features or bugfixes that's a lot
for me to review.
- Dropping Python 2: not anytime terrifically soon, but depends how
the rest of things go. The userbase for this type of tool skews
conservative and supporting 2+3 isn't /that/ painful, still, it mostly
just means fewer Py3-only toys.
  - A discussion about when to drop older Py3 so we can use, e.g.
f-strings or newer typing/async features, is its own thing...
- Pip can absolutely handle updating both major release lines at once,
that's never been a problem.
  - Bugs in support tools (eg my changelog software) can make it more
of a pain, but those are surmountable problems.
- Test suite for Fab 2 is great (should be 90+% cov), but one of the
many reasons 1.x got pseudo abandoned is its test suite is ... nowhere
near that good. In part because the old module-globals design
precludes sane testing of all the moving parts.
  - This is why I resisted doing an in place rewrite or Py3 upgrade -
there was a lower guarantee that the suite would catch issues.
Starting over and being test-driven was incredibly compelling.
- Re: compat layer: ideally it'll be possible to write something that
looks like Fabric 1 but implemented on top of Fabric 2, but it'll
unlikely ever be 100% so the aim should probably be to help users
slowly migrate / hit the highest value API members.
- Re: 3.x, 4.x etc - I didn't mean we'd put out such releases so
frequently, just that they would not be the full rewrite that 2.0 was.
  - We're talking things like "subsystem X has gotten far too creaky
and is super hard to maintain with all the shit we've added, it needs
a rewrite" or "we found that the original API design for Y module is
too frustrating for too many users and need to rejigger the way you
call it".
  - As you say, the "right" way to handle a lot of this is to
add-and-deprecate, but eventually you do need to cut the chaff so it's
not a support burden. Feeling out the right length of time for this
intermediate period is the tricky part. Off the cuff I'd look at how
Django does it as they figured this out long ago. (Not that Fabric is
anywhere near that size, obviously, I couldn't have even pretended to
solo maintenance otherwise...)
- Resentful upgrades: to be fair, I have no control over the distros,
and I took pains on /my/ end to communicate and ensure that running
just the old version, or running the new one side by side w/ it, would
function.
  - But that brings us back to the classic gulf between pip users and
distro-package users - such an old, frustrating story for all
involved, for all libraries (except the ones who have the time to also
be their own distro maintainers :D)
  - And I still apologize for the frustration - I was hoping v2 would
get to where it was an obvious upgrade to v1, far sooner than it did.
I can't have anticipated being burned out, but I should still have at
least attempted some kind of "in place upgrade". Hindsight...

Depending how this week goes I hope to get a blog post up on Fri or
the weekend (still dealing with a ton of just-moved bullshit)
expounding on my need for help from folks. Mostly in a triage sense (I
basically have Github Issues PTSD now), but certainly this sort of
assistance re: other aspects of maintenance that I haven't had time
for, is also more than welcome.

Best,
Jeff

On Sun, May 24, 2020 at 7:39 PM <address@hidden> wrote:

Thanks for the info, you're encouraging me. FYI my timeline is
something
like "this week", I don't see any big chunks of coding needed.

1. I see a couple options at this point. I'm going to ask you to
tell me
which one is best, because it's your project and because I don't
know
enough.
- I write a package for fabric1 today, using python2, even though
it's
end-of-life, and then pursue one of the below after it's done. This
is
the only one where I think distro maintainers could possibly say no
(because it's a new python2-based package), but I'm still willing to
try
it.
- I start on python3 support for 1.x immediately. I can do this one,
I
understand all the steps. Next, pip release. Then, I make packages.
- You or I talk to fabric3 folks to merge their work. Next, pip
release.
Then, I make packages. I don't know fabric3 licensing, are the
people on
board, what version of 1.x version it's forked off of. But yeah,
could
save testing time if it's forked off the latest 1.x and they're
happy to
have their changes merged.
- (Longer term, after feature parity) You or I could write a 1.x API

compat layer for 2.x. Pip release, no new distro package. A compat
layer
is a good idea I hadn't thought of. I don't know enough about 1.x vs
2.x
to know how much this is just some wrappers vs how much there's some

fundamental incompatibilities. This shouldn't be step 1, because
feature
compatibility and a compat layer are both big projects, so it would
massively delay having packages.

2. I get that you wanted 1.x to keep python2 support+stability when
you
did the 2.x release. Want to drop python2 support for 1.x now that
the
official stance is that python2 is end-of-life or keep support? Will

make quite a bit of difference while adding python3 support.
3. Can pip support pushing new stuff to both 1.x and 2.x,
technically?
4. How's the test suite? Adding python3 support is likely to break a
lot
of stuff.

On 2020-05-23 12:12, Jeff Forcier wrote:
I'm still digging myself out of a couple large 2019/2020 related
personal holes (and that was all before The Circumstances
happened),
but this sort of thing has been on my mind lately (blog post
forthcoming).
oof, life hits us all sometimes. hope you're doing okay.

Thanks for reaching out about it - lots of folks seem to think
that
just dropping a ticket on github is the only way to communicate
lately, even about major things...:'(
I wouldn't take it as super meaningful. Email is losing popularity,
and
they may not even be aware of the list. Pin something about where to

send what communications to the mailing list to your github issue
tracker. Re-establish the cultural norms you want in the next
generation.

- though once official-fabric 3.x (and 4.x and etc) come out -
ideally
w/o being full rewrites and just being small chunks of API changes
-
that gets more difficult...I really hate running into packages
named eg
'project2 version 5.8.13'.

I'm assuming that the plan is for 3.x to break backwards
compatibility,
based on semantic versioning and the idea that one might want to
maintain 3 package names.

I don't think a roadmap of incrementally changing the API while
breaking
backwards compatibility is a good one. API stability is a pretty big

deal for me as a user who uses this to provision and update all my
machines. I think you want a good API, but as a user having a stable
API
is more important to me. Can I encourage you not to break backwards
compatibility, even if it makes the codebase worse?

I'd have to see the specifics to make better suggestions, but here
are
some generics. One common compromise approach is to have planned
x-version-back support: mark some part of the API as "deprecated"
for a
couple versions but keep it working. (Deprecation = remove it from
documentation, issue warnings at runtime, maybe other things). Then
after X versions with that part of the API deprecated, actually drop

support. This gives a migration path for users with no breaking
steps.
This plan assumes a regular release schedule, adjust appropriately
if
you're going to do stuff in spurts. It's equally valid to keep
deprecated API functions around indefinitely, if there's no big
cost,
and it's great for users. Aside: The step of deprecation that
developers
usually suck at, is to actually tell people what specifically to do
instead of using the deprecated piece.

I think packaging is pretty reversible and OK to punt packaging
decisions to the time of fabric 3.x, 4.x release. Probably what
you'd do
is have a 'fabric1' on version 1.x and a 'fabric2' on version 3.0,
even
if that's a little confusing. If you can come up with a better name
than
"fabric2" it would make things less confusing, but I can only come
up
with "fabric" and "fabric1" which I think is worse. I mean you could

literally rename one project, but this seems like a hella dumb
reason to
do that.

- however in OS packaging land that's very NOT true, so I don't
have
a big problem with attempting to deprecate 'fabric' in favor of
two
explicit 'fabric1'/'fabric2' versions for now (however this kind
of
decision is often more up to the distro's packagers/policymakers
than
us upstream authors anyway)
Heh, I think you may be overestimating the existence of 'distro
packagers/policymakers', in my experience. It's often more like
"whoever
got around to making a package". There may be some group deal going
for
anything pip-based where they tackle a big list uniformly. Life has
really taught me that "99% of life is showing up" and no one ever
does
anything--simply saying you'll do something often means you can
decide
what happens, and nearly every time I've asked a package maintainer
to
make a change, they have.

But... I use Arch, which is small and tries for absolutely vanilla
packages, and I haven't spent a lot of time making packages. So my
experience may not be representative.

- I'd still prefer to get Fabric 2 to feature parity (+ perhaps
even
a compat layer), but things did not go as planned and I don't
begrudge
people wanting to use the 1.x design on Python 3.
Yeah, I was resentful around this two years ago. Being "upgraded" by

other people to something incompatible and not at feature parity,
with
no recourse, sucks. Having 2.x distro packages be an upgrade I feel
was
a mistake. Releasing 2.x under the same pip name (rather than to a
beta
testing group) before feature parity under was probably a mistake
too?
Oops, happens. It would be equally bad to break 2.x for current
users
trying to "undo" things. Let's fix the packaging at least :).
Getting to
feature parity is definitely good too whenever you have time (you've

probably gotten to parity for everything I use while I wasn't
looking,
I'm pretty out of date). But that's a bigger project and sounds like

you're not in a place to do that atm.

- A BIG disclaimer here is that it's conditional on breadth of
divergence; "fabric 1 + python3 compat" is one thing, "a big ol'
fork
with a lot of additional changes" is another. I don't know which
it is
right now, but the whole point of 2.x and not doing py3 on 1.x was
for
stability's sake.
Well my own plan is certainly just stability and a port, because my
whole motivation is to change nothing in my fabfiles, indefinitely
:).
That said, maintaining a 1.x branch/package does make external forks
or
internal divergence easier. Ultimately, the best thing to make sure
there's no development against 1.x, is just make 2.x better than 1.x
so
everyone submits patches against 2.x :).

--
Jeff Forcier
Unix sysadmin; Python engineer
http://bitprophet.org



reply via email to

[Prev in Thread] Current Thread [Next in Thread]