Background
Those people plugged into the Linux user community are probably aware that recently Microsoft has announced that it will be donating some 20,000 lines of code for eventual (they hope) inclusion in the Open Source Linux kernel project.
Setting aside any interest in a debate about altruistic vs. capitalistic motivations that might be at the heart of this effort on Microsoft’s behalf, the terrible thought that the evil empire might infect the seeming purity of Linux has reached enough of a fever-pitch in the Linux community that Linus Torvalds himself weighed in on the ‘controversy’ in an interview for Linux Magazine (one wonders sometimes if ‘Linux Magazine’ should change its logo to just a giant ‘L’ somewhat like Oprah did with the giant ‘O’ on her own magazine 🙂 ). And, yes, I know that unlike O Magazine being owned by Oprah, Mr. Torvalds doesn’t own Linux Magazine.
In any case, during the interview, Linus Torvalds spoke a bit about his philosophy on Open Source software, why he could care less about getting source contributions from Microsoft (so long as its properly legally and technically vetted) and most interesting to me, about what he viewed as the implicit motivation for anyone contributing to Open Source software:
Linus states that this is how all open source code gets written, developers scratching an itch. The fact that this comes from Microsoft shouldn’t make any difference at all, saying:
“I agree that [MIcrosoft’s contribution is] driven by selfish reasons, but that’s how all open source code gets written! We all “scratch our own itches”. It’s why I started Linux, it’s why I started git, and it’s why I am still involved. It’s the reason for everybody to end up in open source, to some degree.”
The Itch
I’d never really thought at a deep level about trying to distill the wide-ranging motivations people must have for contributing to Open Source software down to a single factor. Until reading this quote, I believed that the universe of OSS-motivations probably contained some mix of most of the following (in no particular order):
- those with a desire to express their creativity (in code) but who cannot do so in their regular professional place of business (because they are professionally stifled, working in uninteresting areas, etc.); these people generally work in OSS projects using significantly different technologies or solving significantly different problems than those in their ‘day jobs’
- those with a strong philosophical belief in the power of free software to empower the individual and the marketplace; these people generally work in OSS projects that are alternatives to established commercial offerings (e.g., OSS operating systems, etc.)
- those interested in increasing their professional notoriety; the evil side of this is people just interested in a passing contribution for the sake of their resume but the positive side of this is people interested in helping the project as they increase their professional profile
- those with a problem to solve with software and no way to monetize their solution so they decide “why NOT open-source this?, perhaps it will be more enticing for others to consider adopting if they have the ‘comfort zone’ of access to the source code”; this leads to the 1000s (millions?) of little, tiny, uninteresting, and largely abandoned OSS projects that litter places like SourceForge, CodePlex, and others (you know, the ones with 1-2 check-ins from 3+ years ago and perhaps 5-10 downloads by users)
But Linus’ contention that the root of the motivation for people to engage in Open Source is to scratch an itch got me thinking about what that perspective could do to help explain some of the most frequently observed patterns I see over and over again in Open Source software projects.
Open Source Software Behavior Patterns In Terms of Itches
For many years, there has been lots of talk in software engineering about the idea of ‘personas’, a sort of condensed archetype of aggregated users that people concoct in order give anthropomorphic shape to otherwise abstract behaviors. In the .NET software world, perhaps the most (in)famous of these is ‘Mort’, whom Microsoft brought to life some years ago to represent either (depending on your point of view): the pragmatic developer interested in getting things done or: the developer without any foresight for whom the ends always justify the means and ease of future maintenance is always someone else’s concern. To be fair, other design fields routinely use personas too; let’s not infer that there’s anything unique or even forward-looking about software engineering’s use of personas to guide design decisions.
After my time spent with various levels of involvement in various different OSS projects, I’d like to propose the following personas to describe the archetypes which I have observed thus far. Rather than giving them names like ‘Mort’, ‘Elvis’, or ‘Einstein’ to which each reader may bring their own different connotations, I will instead use characterizations to describe them.
Persona 1: The Creator
The Creator is a frequent contributor to the OSS project, but almost exclusively in the area of new features. Rarely if ever does The Creator contribute a bug-fix, a patch, or an optimization of an existing feature except where that optimization is needed to lay the groundwork for the new feature the Creator really wants to implement. The Creator’s ‘itch’ is part creative outlet and part self-aggrandizement; “I just implemented this really cool new feature in project XYZ” makes a more interesting blog post than “I just fixed twenty bugs in project XYZ”. Like all OSS contributors, The Creator works on the parts of the OSS project that interest him/her…its just that they aren’t interested in any of the existing features, only new ones.
Persona 2: Mr. Fix-It
Mr. Fix-It is also a frequent contributor to the OSS project, but his/her focus is nearly entirely on verifying and fixing bugs, applying patches submitted by those without commit permissions, and managing the content in whatever bug-tracker system the OSS project uses. Unlike The Creator whose ‘itch’ is at least partially (often entirely justified) self-promotion, Mr. Fix-It is perfectly willing to toil away in (relative) obscurity, his/her efforts overshadowed almost completely by the others on the project team working on The Next Big Thing in the project. Mr. Fix-It’s ‘itch’ is to be able to contribute in even a small (but sustainable) way to the OSS project. In healthy OSS projects, the value of Mr. Fix-It is never taken for granted by the rest of the project team who understands all too well that Mr. Fix-It performs an essential role in the long-term health of the project and their efforts in this area free the remainder of the project team to advance the new features of the project while the health of the existing code base is maintained by Mr. Fix-It.
Persona 3: The Demander
Unlike some of the other personas, The Demander isn’t a contributor to the project. In fact, just the reverse: The Demander is that user of the software that will never contribute anything to the project but is extremely vocal about the OSS project’s shortcomings, the direction it should take for new features, and provides frequent comparisons between the current state of the OSS project and one or more commercial software offerings that might be trying to solve the same problem as the OSS project. The Demander’s ‘itch’ is to get something for nothing as they view the OSS world as their way to get software without having to pay anything for it. Despite their disinterest (or perhaps even technical inability) to contribute anything to the project, they labor under the misconception that the OSS project team somehow has an obligation to address their every need. Interestingly, while The Demander is often the loudest voice on the OSS project’s discussion forums, The Demander cannot usually be bothered to even formalize their request (demand!) in the OSS project’s issue-tracking system even at times when The Demander might actually have a useful suggestion for the project. If its a bug, The Demander cannot be bothered to provide adequate examples of the conditions that cause the bug, making it impossible for the team to even consider resolving it since its impossible to repro the bug with such limited data on it.
Persona 4: The Drive-By Shooter
Despite the negative connotation of this persona’s name, The Drive-By Shooter plays an important role in any healthy OSS project. The Drive-By Shooter will never join the project team, but by virtue of their use of the software they have identified a bug, developed a patch, and submitted it to the project team. Afterwards, it may be some time (and perhaps never!) that The Drive-By Shooter will return to engage visibly with the OSS project. The Drive-By Shooter’s ‘itch’ is entirely pragmatic: to get the fix for their blocking bug into the OSS project’s trunk so that they can return to their own work. The Drive-By Shooter has correctly deduced that the quickest path to resolving their bug is to submit a patch rather than waiting for Mr. Fix-It to consider it a priority to address on the project’s own schedule.
Persona 5: The Last Boy Scout
In an era punctuated largely by the self-interested (or at least the self-motivated), The Last Boy Scout is an anomaly: an individual willing to step up and do whatever needs to be done for the project. Sometimes this involves fixing bugs, other times this involves implementing new features, still other times it might require implementing a build-server or issue-tracker. The Last Boy Scout is willing to play the role of almost any other persona, but only on an as-needed basis and only for a short period of time. This persona could perhaps also be considered akin to a good ‘Utility Player’ in baseball, willing and able to play multiple positions depending on the team’s needs at any given time. The Last Boy Scout’s ‘itch’ that needs scratching is usually about some personal attachment to ensuring that the project continues to live and stay healthy; often this is because The Last Boy Scout either has a significant amount of personal time invested in the project to-date or feels passionately about the moral importance of the project (e.g., as a viable alternative to some similar commercial offering). In an ideal world, perhaps everyone involved in OSS projects would be this persona, willing to step up and do whatever needs to be done to move the work along, but the real world is usually far from this ideal.
Documentation Scratches No Itch
We all know that one of the (often accurate) complaints about OSS projects is that of poor documentation. What documentation might (formally) exist is often out of date and the informal documentation is scattered around 1000s of blogs on the Internet in an incoherent mess that is only accessible to OSS project adopters should they be lucky enough to accidentally enter the right search terms into their favorite search engine. Until I read Linus’ evaluation of the intrinsic motivations for people to participate in OSS projects being ‘scratching their own itches’, I could never really assign a clear explanation in my own mind for why this is the case. But now I have a context for explaining this phenomenon: there’s simply no OSS persona for whom documentation scratches an itch.
To my mind, the primary OSS participants for whom documentation ‘scratches an itch’ tend to be newer users who have a need for documentation to enable them to effectively learn to use features of the OSS project. But by virtue of the fact that they need the documentation, they are also uniquely unqualified to write it. And for the very people qualified to write the documentation (project committers, long-term users, etc.), having documentation scratches no itch – they already know how to use the software so they don’t need the documentation. So one way to think about the dearth of documentation for OSS projects is to consider it in terms of a so-called “Scratch-Itch Gap”: those with the itch lack the ability to scratch it and those who could scratch it lack the itch in the first place.
I will set aside for the moment whether I completely believe in Linus’ characterization of all OSS efforts being inherently self-beneficial at their core, but I have to admit that this perspective does adequately explain away the dearth of documentation available about most OSS projects. Typically (though of course not always) lacking any way to monetize new adopters, OSS projects lack any extrinsic motivation to provide high-quality documentation to ease increased adoption (since increased adoption isn’t but a peripheral concern to most OSS contributors). And the Scratch-Itch Gap means that there’s not going to be (in most cases) any intrinsic motivation for creating documentation.
A ‘No Docs, no Patch’ Policy…?
That leaves us involved in OSS in quite a quandary: without the monetary motivation of more users –> more revenue, the simple motivator of better documentation –> more users isn’t on its face sufficient for anyone to spend the (usually significant) time to construct high-quality (or even low-quality but comprehensive) documentation. Sure, there’s always the moderate ego-boost that comes from having some OSS project you’re involved in adopted by more and more people, but the ego-scratching itch is only going to work for a small number of OSS projects and eventually it too becomes ineffective as a motivator for even the most determined OSS contributor.
Many OSS projects have a pretty valuable set of rules around submitting patches (or even bug reports): for a patch/bug report to be accepted it must….
- be accompanied by a failing unit test that can be used to repro the bug in the un-patched existing build of the software
- if a patch is submitted, contain a test that passes after the patch is applied
The smartest OSS projects enforce this very same set of requirements on new feature implementations as well (essentially for any change to the codebase, this set of requirements must be satisfied, whether a patch from outside the project that Mr. Fix-It will have to verify and apply or a new feature introduced by The Creator).
So here’s an idea I have been kicking around: what if another part of this entirely logical set of requirements for changes to the codebase also included the need to provide a corresponding contribution to the documentation for a project? Just as “you have to provide a corresponding test-case” is considered an acceptable bar to set, would it be possible to elevate the bar just slightly more and add “you have to provide a corresponding new section to the documentation in the case of a feature or a corresponding change (patch) to the documentation in the case of a patch”?
Right now, most projects have the following set of requirements for accepting submissions:
| Patch | Feature |
| Breaking Test against existing version | Code Implementing Feature |
| Code implementing the patch | Unit Test for Feature’s code |
| Passing test for the patched code |
I’m suggesting this change (slightly) to the following:
| Patch | Feature |
| Breaking Test against existing version | Code Implementing Feature |
| Code implementing the patch | Unit Test for Feature’s code |
| Passing test for the patched code | New Section for Docs for the Feature |
| Patch to docs related to the patched code |
The common outcry would probably be “raising the bar like this will reduce the number of Drive-By Shooter types that will be willing to contribute to the project”. I would probably have to agree, but consider that the existing bar (you must provide a unit test) already results in a barrier to contributions, but as developers we are willing to consider that an acceptable barrier to erect to ensure the health of the codebase.
What would the net effect be on OSS projects in general if a corresponding requirement for documentation was introduced to help ensure the health of the documentation? Fewer contributions? Probably. But (eventually) more comprehensive documentation? Almost certainly.
The question we have to ask is: if such a thing were put in place, would the various personas’ existing ‘itches’ be strong enough motivation to overcome this new barrier? I wonder…
Interesting post Stephen. Maybe you could take this idea and apply it in a way that causes less friction for developers by instead requiring that the example/quickstart project be updated to demonstrate the new feature. At least that way you are just asking the developer to write some more code.
@Paul:
That’s a really interesting idea (kind of a middle-ground accomodation). I suppose we could consider a sort of hierarchy of documentation that looks something like this:
1) lowest level: code itself is self-documenting, no need for docs of any kind
2) unit tests are the documentation, no written docs needed but the tests ‘explain’ how to use the code
3) the ‘sample app’ you suggest here is the documentation, no written docs needed but the implementation sample app is clear enough to grok what’s going on
4) there’s actual comprehensive written docs provided
I have to say that I have never found any OSS project of significant size where #1 was true, but that I have ‘taught myself’ how to use several complex OSS projects using #2.
The one that comes to mind off the top of my head is Ayende’s Rhino.Tools ‘framework’, for which calling the written documentation ‘sparse’ is an affront to the word ‘sparse’ 🙂 but by reading the unit tests I was able to deduce usage patterns in the code that permitted me to make effective use of the proejct in my own work.
I think your suggestion of “you have to update some kind of ‘reference app’ that demonstrates the new feature/patch” is a pretty good alternative to asking people who may not have good documentation-writing skills to write a chapter or two in the manual.
One thing I’m thinking as I write this response now is that my suggesting fails to account for the fact that writing code and writing docs don’t just differ in how much most developer WANT to perform each task, but also differ in how much most developers CAN do each task. By asking for ‘documentation in code’ we might be acknowledging this and responding to it.
Nice suggestion and thanks for the comment.
-Steve B.
Don’t expect contributions from programmers who don’t write code like open source contributors do. This oversight is the single, most destructive force in ensuring that the open source contributor population will remain a small fraction of open source users.
Code usability problems ensure that code users will not want to use the code as much as the project.
I find it incredibly disheartening that this issue simply isn’t a part of the open source contribution conversation. Code usability is the elephant in the room that open source contributors simply can’t bring themselves to face, just as usability in general is a problem that most programmers can’t see into.
I think that in OSS there are persons typed 1 + 2 + 5 = 8
[…] Behavior Patterns of Open Source Users in terms of itches and typical state of an OS project […]
@Fabio:
Yes, I agree that the personas are often blended in read life (e.g., most people don’t –thankfully! — fit neatly into one persona or the other exclusively but are in fact an amalgam of mutliple personas…not multiple personalities, but multiple personas 🙂 )
You yourself, for example, almost certainly fit into that suggested 8th persona 😉
-Steve B.
@Scott:
Intersesting comment (as always!) but I’m not sure I entirely agree with your estimation that code-usability (I am assuming you mean more code-learnability…?) is a factor that’s due quite the primacy you assign to it. Certanly there are different coding ‘styles’ and ‘approaches’. Certainly there are better and worse projects from a code-learnability/approachability standpoint (which in turn does obviously play a part in contributor-friction).
But I’m not sure that its possible to conclude that there is a vast untapped collection of potential OSS contributors just itching to volunteer their spare time/effort/energy if only they could find an OSS project whose codebase they could understand adequately.
I think that in any endeavor driven by volunteerism (and certainly OSS projects without any tie to commercial ventures like 99.9999% of OSS projects aren’t any different in this regard), the ratio of volunteers to people interested in benefiting from the efforts of volunteers is always very low. While code-approachability might help mitigate that ratio somewhat, I don’t think its the primary governor of contributors by any stretch.
Rather, I tend to think that the very reason that most people go looking for an OSS project to *use* is that the project does something for them that they could not do for themselves (usually due to insufficient technical skills and/or inadequate spare time of their own) and so this leads to a situation where the very people who should be most interested in contributing something back to the OSS projects they use are either technically unable or too time-starved to do so in any significant way.
Admittedly only anecdotally, my experience is that when someone raises an issue and is met with “great idea, we’re accepting patches” from the OSS project team, invariably the responses are more along the lines of “I would have no idea how to do that, I’m just a user of project XYZ”, “I’d love to do that, maybe in 6 months when I get a break from my daily responsibilities”, and other similar replies. Rarely if ever has anyone responded with “I tried the other night, but couldn’t for the life of me figure out where in the codebase to even make this change due to the confusing way the project is designed”.
Since I’m not aware of any comprehensive study on this phenomenon (“The Barriers to Patch Submission in OSS Projects” by The Gartner Group 🙂 , etc.), I will have to admit that its obviously entirely possible that this class of potential contributor does exist out there, silently TRYING to write a patch and then deciding for themselves that the code is too unapproachably complex for them to understand and never providing that feedback so OSS contributors could be aware of it, but I also admit I’m skeptical that this is truely the case.
Have you any examples (anecdotally or otherwise) of situations you could share that exemplify this scenario to the degree of primacy you seem to assign to it as a barrier to OSS contributions?
In my own experience, when I have wanted to contribute a patch to an OSS project the code for which I have only passing familiarity, the OSS project’s team (usually via forums or otherwise) has always been quite helpful about guiding my efforts as I navigate an unfamiliar (and often sizable) codebase.
-Steve B.
[…] On Open Source Software, Itches, and Documentation – Steve Bohlen […]
If you require docs for new features or patches, when is that doc good enough?
Just a quickstart, or MSDN style? Something in between? When is the documentation provided of good enough quality to be accepted? Opinions on this vary between contributors and between patch reviewers.
I’m afraid that if you require documentation, that you just get awful unusable documentation.
@Alwin:
That’s certainly possible, but I would guess that it would be possible for a projects’ contributor teams to review documentation submissions just the same way they presently review code patches submitted.
As an (obvious) example, even when a project has a requirement that all patches be submitted with accompanying tests (and even set a high minimum coverage % for those tests), just because a patch meets this standard ‘on paper’ doesn’t thus mean the patch is accepted if/when the quality of the tests (or the patch, of course!) is poor. The project team should be able to make the same quality judgements about documentation submitted too, I would think.
That said, you bring up an interesting point about the reasonableness of the requirement in the face of unclear standards — a team would have to demonstrate (probably at first with clear stated standards and then later by saying “new docs have to be as good as existing docs”) what is meant by “docs we will accept” for this to have a chance of working else I think you’re right that the result would be “minimally-compliant doc submissions” benfiting nobody.
THanks for the feedback~!