On Open Source Software, Itches, and Documentation


Those people plugged into the Linux user community are probably aware that recently Microsoft has announced that it will be donating some 20,000 lines of code for eventual (they hope) inclusion in the Open Source Linux kernel project.

Setting aside any interest in a debate about altruistic vs. capitalistic motivations that might be at the heart of this effort on Microsoft’s behalf, the terrible thought that the evil empire might infect the seeming purity of Linux has reached enough of a fever-pitch in the Linux community that Linus Torvalds himself weighed in on the ‘controversy’ in an interview for Linux Magazine (one wonders sometimes if ‘Linux Magazine’ should change its logo to just a giant ‘L’ somewhat like Oprah did with the giant ‘O’ on her own magazine 🙂 ).  And, yes, I know that unlike O Magazine being owned by Oprah, Mr. Torvalds doesn’t own Linux Magazine.

In any case, during the interview, Linus Torvalds spoke a bit about his philosophy on Open Source software, why he could care less about getting source contributions from Microsoft (so long as its properly legally and technically vetted) and most interesting to me, about what he viewed as the implicit motivation for anyone contributing to Open Source software:

Linus states that this is how all open source code gets written, developers scratching an itch. The fact that this comes from Microsoft shouldn’t make any difference at all, saying:

“I agree that [MIcrosoft’s contribution is] driven by selfish reasons, but that’s how all open source code gets written! We all “scratch our own itches”. It’s why I started Linux, it’s why I started git, and it’s why I am still involved. It’s the reason for everybody to end up in open source, to some degree.”

The Itch

I’d never really thought at a deep level about trying to distill the wide-ranging motivations people must have for contributing to Open Source software down to a single factor.  Until reading this quote, I believed that the universe of OSS-motivations probably contained some mix of most of the following (in no particular order):

  • those with a desire to express their creativity (in code) but who cannot do so in their regular professional place of business (because they are professionally stifled, working in uninteresting areas, etc.); these people generally work in OSS projects using significantly different technologies or solving significantly different problems than those in their ‘day jobs’
  • those with a strong philosophical belief in the power of free software to empower the individual and the marketplace; these people generally work in OSS projects that are alternatives to established commercial offerings (e.g., OSS operating systems, etc.)
  • those interested in increasing their professional notoriety; the evil side of this is people just interested in a passing contribution for the sake of their resume but the positive side of this is people interested in helping the project as they increase their professional profile
  • those with a problem to solve with software and no way to monetize their solution so they decide “why NOT open-source this?, perhaps it will be more enticing for others to consider adopting if they have the ‘comfort zone’ of access to the source code”; this leads to the 1000s (millions?) of little, tiny, uninteresting, and largely abandoned OSS projects that litter places like SourceForge, CodePlex, and others (you know, the ones with 1-2 check-ins from 3+ years ago and perhaps 5-10 downloads by users)

But Linus’ contention that the root of the motivation for people to engage in Open Source is to scratch an itch got me thinking about what that perspective could do to help explain some of the most frequently observed patterns I see over and over again in Open Source software projects.

Open Source Software Behavior Patterns In Terms of Itches

For many years, there has been lots of talk in software engineering about the idea of ‘personas’, a sort of condensed archetype of aggregated users that people concoct in order give anthropomorphic shape to otherwise abstract behaviors.  In the .NET software world, perhaps the most (in)famous of these is ‘Mort’, whom Microsoft brought to life some years ago to represent either (depending on your point of view): the pragmatic developer interested in getting things done or: the developer without any foresight for whom the ends always justify the means and ease of future maintenance is always someone else’s concern.  To be fair, other design fields routinely use personas too; let’s not infer that there’s anything unique or even forward-looking about software engineering’s use of personas to guide design decisions.

After my time spent with various levels of involvement in various different OSS projects, I’d like to propose the following personas to describe the archetypes which I have observed thus far.  Rather than giving them names like ‘Mort’, ‘Elvis’, or ‘Einstein’ to which each reader may bring their own different connotations, I will instead use characterizations to describe them.

Persona 1: The Creator

da vinci the creation

The Creator is a frequent contributor to the OSS project, but almost exclusively in the area of new features.  Rarely if ever does The Creator contribute a bug-fix, a patch, or an optimization of an existing feature except where that optimization is needed to lay the groundwork for the new feature the Creator really wants to implement.  The Creator’s ‘itch’ is part creative outlet and part self-aggrandizement; “I just implemented this really cool new feature in project XYZ” makes a more interesting blog post than “I just fixed twenty bugs in project XYZ”.  Like all OSS contributors, The Creator works on the parts of the OSS project that interest him/her…its just that they aren’t interested in any of the existing features, only new ones.

Persona 2: Mr. Fix-It

Mr. Fix-It is also a frequent contributor to the OSS project, but his/her focus is nearly entirely on verifying and fixing bugs, applying patches submitted by those without commit permissions, and managing the content in whatever bug-tracker system the OSS project uses.  Unlike The Creator whose ‘itch’ is at least partially (often entirely justified) self-promotion, Mr. Fix-It is perfectly willing to toil away in (relative) obscurity, his/her efforts overshadowed almost completely by the others on the project team working on The Next Big Thing in the project.  Mr. Fix-It’s ‘itch’ is to be able to contribute in even a small (but sustainable) way to the OSS project.  In healthy OSS projects, the value of Mr. Fix-It is never taken for granted by the rest of the project team who understands all too well that Mr. Fix-It performs an essential role in the long-term health of the project and their efforts in this area free the remainder of the project team to advance the new features of the project while the health of the existing code base is maintained by Mr. Fix-It.

Persona 3: The Demander

Unlike some of the other personas, The Demander isn’t a contributor to the project.  In fact, just the reverse: The Demander is that user of the software that will never contribute anything to the project but is extremely vocal about the OSS project’s shortcomings, the direction it should take for new features, and provides frequent comparisons between the current state of the OSS project and one or more commercial software offerings that might be trying to solve the same problem as the OSS project.  The Demander’s ‘itch’ is to get something for nothing as they view the OSS world as their way to get software without having to pay anything for it.  Despite their disinterest (or perhaps even technical inability) to contribute anything to the project, they labor under the misconception that the OSS project team somehow has an obligation to address their every need.  Interestingly, while The Demander is often the loudest voice on the OSS project’s discussion forums, The Demander cannot usually be bothered to even formalize their request (demand!) in the OSS project’s issue-tracking system even at times when The Demander might actually have a useful suggestion for the project.  If its a bug, The Demander cannot be bothered to provide adequate examples of the conditions that cause the bug, making it impossible for the team to even consider resolving it since its impossible to repro the bug with such limited data on it.

Persona 4: The Drive-By Shooter

Despite the negative connotation of this persona’s name, The Drive-By Shooter plays an important role in any healthy OSS project.  The Drive-By Shooter will never join the project team, but by virtue of their use of the software they have identified a bug, developed a patch, and submitted it to the project team.  Afterwards, it may be some time (and perhaps never!) that The Drive-By Shooter will return to engage visibly with the OSS project.  The Drive-By Shooter’s ‘itch’ is entirely pragmatic: to get the fix for their blocking bug into the OSS project’s trunk so that they can return to their own work.  The Drive-By Shooter has correctly deduced that the quickest path to resolving their bug is to submit a patch rather than waiting for Mr. Fix-It to consider it a priority to address on the project’s own schedule.

Persona 5: The Last Boy Scout

In an era punctuated largely by the self-interested (or at least the self-motivated), The Last Boy Scout is an anomaly: an individual willing to step up and do whatever needs to be done for the project.  Sometimes this involves fixing bugs, other times this involves implementing new features, still other times it might require implementing a build-server or issue-tracker.  The Last Boy Scout is willing to play the role of almost any other persona, but only on an as-needed basis and only for a short period of time.  This persona could perhaps also be considered akin to a good ‘Utility Player’ in baseball, willing and able to play multiple positions depending on the team’s needs at any given time.  The Last Boy Scout’s ‘itch’ that needs scratching is usually about some personal attachment to ensuring that the project continues to live and stay healthy; often this is because The Last Boy Scout either has a significant amount of personal time invested in the project to-date or feels passionately about the moral importance of the project (e.g., as a viable alternative to some similar commercial offering).  In an ideal world, perhaps everyone involved in OSS projects would be this persona, willing to step up and do whatever needs to be done to move the work along, but the real world is usually far from this ideal.

Documentation Scratches No Itch

We all know that one of the (often accurate) complaints about OSS projects is that of poor documentation.  What documentation might (formally) exist is often out of date and the informal documentation is scattered around 1000s of blogs on the Internet in an incoherent mess that is only accessible to OSS project adopters should they be lucky enough to accidentally enter the right search terms into their favorite search engine.  Until I read Linus’ evaluation of the intrinsic motivations for people to participate in OSS projects being ‘scratching their own itches’, I could never really assign a clear explanation in my own mind for why this is the case.  But now I have a context for explaining this phenomenon: there’s simply no OSS persona for whom documentation scratches an itch.

To my mind, the primary OSS participants for whom documentation ‘scratches an itch’ tend to be newer users who have a need for documentation to enable them to effectively learn to use features of the OSS project.  But by virtue of the fact that they need the documentation, they are also uniquely unqualified to write it.  And for the very people qualified to write the documentation (project committers, long-term users, etc.), having documentation scratches no itch – they already know how to use the software so they don’t need the documentation.  So one way to think about the dearth of documentation for OSS projects is to consider it in terms of a so-called “Scratch-Itch Gap”: those with the itch lack the ability to scratch it and those who could scratch it lack the itch in the first place.

I will set aside for the moment whether I completely believe in Linus’ characterization of all OSS efforts being inherently self-beneficial at their core, but I have to admit that this perspective does adequately explain away the dearth of documentation available about most OSS projects.  Typically (though of course not always) lacking any way to monetize new adopters, OSS projects lack any extrinsic motivation to provide high-quality documentation to ease increased adoption (since increased adoption isn’t but a peripheral concern to most OSS contributors).  And the Scratch-Itch Gap means that there’s not going to be (in most cases) any intrinsic motivation for creating documentation.

A ‘No Docs, no Patch’ Policy…?

That leaves us involved in OSS in quite a quandary: without the monetary motivation of more users –> more revenue, the simple motivator of better documentation –> more users isn’t on its face sufficient for anyone to spend the (usually significant) time to construct high-quality (or even low-quality but comprehensive) documentation.  Sure, there’s always the moderate ego-boost that comes from having some OSS project you’re involved in adopted by more and more people, but the ego-scratching itch is only going to work for a small number of OSS projects and eventually it too becomes ineffective as a motivator for even the most determined OSS contributor.

Many OSS projects have a pretty valuable set of rules around submitting patches (or even bug reports): for a patch/bug report to be accepted it must….

  • be accompanied by a failing unit test that can be used to repro the bug in the un-patched existing build of the software
  • if a patch is submitted, contain a test that passes after the patch is applied

The smartest OSS projects enforce this very same set of requirements on new feature implementations as well (essentially for any change to the codebase, this set of requirements must be satisfied, whether a patch from outside the project that Mr. Fix-It will have to verify and apply or a new feature introduced by The Creator).

So here’s an idea I have been kicking around: what if another part of this entirely logical set of requirements for changes to the codebase also included the need to provide a corresponding contribution to the documentation for a project?  Just as “you have to provide a corresponding test-case” is considered an acceptable bar to set, would it be possible to elevate the bar just slightly more and add “you have to provide a corresponding new section to the documentation in the case of a feature or a corresponding change (patch) to the documentation in the case of a patch”?

Right now, most projects have the following set of requirements for accepting submissions:

Patch Feature
Breaking Test against existing version Code Implementing Feature
Code implementing the patch Unit Test for Feature’s code
Passing test for the patched code  

I’m suggesting this change (slightly) to the following:

Patch Feature
Breaking Test against existing version Code Implementing Feature
Code implementing the patch Unit Test for Feature’s code
Passing test for the patched code New Section for Docs for the Feature
Patch to docs related to the patched code  

The common outcry would probably be “raising the bar like this will reduce the number of Drive-By Shooter types that will be willing to contribute to the project”.  I would probably have to agree, but consider that the existing bar (you must provide a unit test) already results in a barrier to contributions, but as developers we are willing to consider that an acceptable barrier to erect to ensure the health of the codebase.

What would the net effect be on OSS projects in general if a corresponding requirement for documentation was introduced to help ensure the health of the documentation?  Fewer contributions?  Probably.  But (eventually) more comprehensive documentation?  Almost certainly.

The question we have to ask is: if such a thing were put in place, would the various personas’ existing ‘itches’ be strong enough motivation to overcome this new barrier?  I wonder…