[Gnu-arch-users] State of the Project

[James Blackwell]

Hi guys. So things are pretty confusing. There's confusion about who is in
charge of arch, where releases are made, what our "roadmap is". With this
email, I hope to catch everybody up, and get them onto the same page.
Though there are certainly a lot of complex dynamics happening, I hope to
explain it in a simple way. In this mail, I make all reasonable attempts to
be accurate. However, though certain parts of this email are derived from
perspective, I try to stick with facts as much as possible.

First, who am I, as it pertains to arch? There's a variety of roles I fill,
to which there is universal agreement. As of this writing, I perform
the following: 

  * I manage and host the project's wiki site
  * I manage and host the project's web site
  * I host and manage the arch supermirror
  * I host and provide system administration for the projects bugtracker
  * I review patches from other arch developers. Over the last
    two months, I've overseen the inclusion of dozens, if not
    hundreds of patches. These patches range everywhere from 
    cosmetic changes to bugfixes to minor new features
  * I contribute code to gnuarch. I'm best known for the grab and
    rbrowse commands, though my work is scattered all over the tree
  * I support gnuarch users on the project's irc channel, on the gnuarch
    mailing list, and other places as well. 
  * I have even provided financial support to Tom Lord

A few months ago, arch was having a number of problems. Tom told me that he
had burned out on arch and wasn't receiving donations of the size he wanted
from the sorts of places that he wanted. Sure, there were individuals like
myself helping out, but the large companies were essentially absent. The
result of these two problems is that I think Tom Lord became very
demotivated. He also mentioned more than once in public places that he had
tried to do things like holding development hostage in order to get more
funding. Though he was still thinking about arch, his patch review had
slowed to a crawl. Tom's major public complaint at the time was that the
patches were not of a quality that he could accept.

During this time, I flew out to speak with Tom and to try and get him
involved in arch again. We discussed a variety of issues but the two that
stick with me best are that Tom told me that he was burned out on arch and
it was too much for him. So we discussed someone else taking over arch.
Though Tom hoped that somebody else would take over, he didn't think any of
the potential candidates was quite suitable. However, I was able to achieve
the main purpose of my visit: To make sure he was aware were becoming
frustrated with the development holdup. Towards the end of the visit, he
told me that the message had "been received". He agreed that he would start
reviewing patches in about a month.

After I returned from visiting Tom, I waited out the month. At the end of
the month, there was no apparent change in the development process.
Disheartened that my pep talk had not had the results I had hoped for, I
considered in more detail the problems that were impacting arch
development:

   * Arch development had stagnated. Though Tom was accepting patches 
     at a rate of about one a day, he was infrequently mirroring out
     this information, which prevented other developers from coordinating
     well

   * Arch developers were showing clear signs of frustration that their
     work was sitting in limbo for months at a time without any clear 
     indication that their work had been accepted.

   * Users were becoming increasing frustrated that arch was continuing
     to exhibit bugged behavior that had been recognized 

   * Though Tom had shown some effort in communicating his thoughts 
     effectively, he still hasn't quite gotten to the point where he can
     bring across a vision with mass appeal. Though he's usually capable of
     getting a couple people to understand, he often appears to leave many
     in a confused state (which has been mentioned by several people lately
     on the list).  I don't think that this is necessarily a fault.  I
     believe that Tom's view of the world as through the eyes of an
     engineer is a valuable asset to the project.  Unfortunately, clearly
     communicating thought to the group at large is a necessary skill for a
     project leader.

With this in mind, I considered the qualities and/or responsibilities 
that the arch project leader must either possess or oversee:

  * Strong communication skills, that would allow the project leader to 
    clearly express both short and long term goals for the project.

  * The fortitude to provide a strong presense in the mediums used by
    arch developers and arch users alike. Yes, this is a time
    consuming process, but its necessary in order to properly coordinate
    developers and understanding how the tool is used in the "real world"

  * Dedication to keeping the arch development process moving. To perform
    this one, the project leader must ensure that code submissions are 
    reviewed (and if appropriate, accecepted) on a timely bases

  * New releases of gnuarch must occur on a relatively frequent basis. 
    This minimizes the amount of time that users have to suffer through
    bugs (there's always bugs in releases. Some are from the previous
    release, some go back to arch's conception). Releases are necessary
    for other reasons as well. The health of a project is naively judged
    by the general public by how frequently releases are made.

  * The project leader must ensure that the project remains oriented 
    towards serving the needs of a wide variety of users. Though we want to
    have a tool that can be used very precisely, we also need the tool to
    be simple enough to use so that the engineers of tomorrow don't give up
    before they even start. 

To address this, I approached a small handful of people that I personally
felt were up to the task as project lead. While not quite as good as Tom
when it came to a designer, they more than made up for the difference when
it came to communication.  Unfortunately, I was unable to convince any of
them to step up to the responsibility. Realizing that if I didn't do it,
nobody would, I stepped up for the job.

While I was working on the announcement that I would make, Tom and I had
an argument. In the argument, I lost my temper and stated that henceforth
I would be overseeing development. Much to my surprise, Tom's reaction
was the exact opposite of what I expected. While I had expected something
along the lines of "you can have my gun when you pry it from my cold, dead
fingers", I received a different reply. Though I don't remember the 
exact reply, it was something along the lines of 'I pissed off James into
doing the right thing. I should piss him off more often'

I was thrilled with the prospect that I was going to be able to make sure
that arch would develop at the rate it should without having to fight to
make it happen. For me, the world had indeed come up with roses; Patches
would be reviewed, users would be supported, releases would be made, Tom
would continue to remain involved and forcus on developing tomorrow's
features. It was really an ideal situation in which *everybody* (well,
everybody but Larry McVoy) was going to walk away with what they wanted. 

I was very happy with the situation. Everybody was going to get what they
wanted, so when Tom appeared to have some misgivings about me performing
the releases, I backed off for him. Though I was willing to give 1.3 back
to him, and I was willing to give him some time to personally review
releases that I was going to make, I held my resolve that I would be making
the stable releases.

Though I wasn't an ideal candidate for the job, I can (and have) easily
offset my weaknesses by dependanding upon the strengths of others. I.E., I
leverage the communities resources for the benefit of the community. So I
started diving into patches. The ones I understood I took. The ones I
didn't understand, I brought to others. Sometimes I even made the patch
submitter walk me through the patch step by step. But usually, I can
tell when I'm in over my head, and I ask for help.

I've been very happy with the results. By leaning upon the experts of any
given subset of tla, by relying on test cases, by relying on user testing
and by relying on my sheer determination, I'm actually quite comfortable
with the releases. 

My first release, 1.2.1 was only a so-so release. However, it did prove
that the model *does* work. Only one moderate bug (with many faces) slipped
into 1.2.1. And with the 1.2.2 candidates, there have been *0* regressions
found. With some practice under my belt, I'm very comfortable stating that
1.2.2 is going to be the first good release of many frequent good releases.

I've done a pretty good job for the community. Unlike under Tom's
management, patches are reviewed much quicker, releases come out much
faster, and the user community is more involved in the development.

Sure. I don't understand libraries as well as Aaron. I don't have Robert
Collins' intrinsic understanding of file-ids. I'll never understand the
problems that frontends deal with quite as well as ddaa does, nor do I have
quite the foresight that Tom Lord does. But that's fine; those people are
there, and when I ask them for a hand, they're there for me. 

The problem with arch is that the lead developer's job isn't just one of
development. Sure, that's *part* of the job, but these days, the real job
is *management*. Somebody needs to be in charge of coordinating the various
project activities so that everybody can work as efficiently as possible. 

As a group, we're doing the best work that we've *ever* done. I recently
checked the divergeance between where Tom left off and I picked up, and the
diff for the last two months of work is *80,000* lines long. Within those
80,000 lines of code is the blood, sweat and tears of over a dozen arch
developers and the valued input of hundreds of endusers.

Yes, I'm aware that Tom is showing signs of dissettlement with the current
process at the current moment. I think he'll eventually get over it, and in
time actually come to *like* it. Tom doesn't have to deal with the hordes
of users, he doesn't have to deal with the patches that aren't quite up to
snuff, he doesn't have to do a lot of stuff he didn't like to do anyways.
The only thing that he looses out on is that arch isn't strictly his
vision; rather the arch vision is the amalgam of lots of people.

That doesn't mean I'm not going to listen to Tom. As the both the "father"
of arch and its lead visionary, Tom's opinion is vital for the health of
arch. I strongly encourage Tom to participate in any aspect of the project
that he can.

As Tom told me when I visited him, the project has grown from his baby into
a community project with a life of its own. Though I'll listen to him
carefully, I can't promise that the project is always going to go in the
direction that he likes. However, I do promise that his professional
opinions will always hold considerable sway as I balance them against
the aggregate opinion of the rest of the community. 

I continue to invite everyone to continue to work together side by side.
_Everyone is welcome_. My only request is that we all remember that
everybody involved is for all practical purposes a volunteer doing their
part to further the goals of free software in general, and arch in
specific. The only requirement is that we all do our best to remain civil,
because without civility, our little arch civilization could fall apart.

This responsibility lays with every one of us in the community, including
myself. I've heard from several people that I have been either too
demanding or too forceful with Tom. Perhaps I have been a bit of both;
however I decided that this pain was necessary in order to solve a
development crisis that goes back for months, if not years. I appologize
for anything that I did that was more than was strictly necessary to solve
these problems.

I've also heard from several members of the community that I have been
unfair to Tom by "plonking" him (plonking is a term that means that all
mail sent to you by an individual is deleted without reading).  I think
that I've acted quite appropriately, considering mails he's sent both to
the list and to me in private. However, out of respect to the community,
I'll honor your wishes and deplonk Tom. I make no promises that I won't
plonk him again; life really is too short for me to devote much time 
to somebody that have the respect for others that I think everybody 
deserves.

-- 
James Blackwell          Try something fun: For the next 24 hours, give
Smile more!              each person you meet a compliment!

GnuPG (ID 06357400) AAE4 8C76 58DA 5902 761D  247A 8A55 DA73 0635 7400

signature.asc
Description: Digital signature