I just gave a talk at last week’s ALT.NET NYC user group introducing Domain Driven Design (DDD) as a pattern language and included in there was a slide expressing my opinion “Once: its a fluke, twice: its a coincidence, three times: its a pattern” intended to posit the idea that patterns evolve from our observations of repeat phenomenon rather than being invented by anyone.   I tend to think the same is true for patterns in life, business, and the software profession as well and I just recently had a confluence of experiences that tell me its time to identify a new pattern:

The present pace of change in the software development industry precludes anyone from being able to ever become deeply knowledgeable in more than a tiny handful of areas.

The Red Queen Hypothesis

Like most patterns, this one has its roots in other previously-observed phenomenon.  In 1973, the Red Queen Hypothesis was identified as a way to consider and understand the effects of a constant (or even increasing) rate of change in a structured system.  A reference to Lewis Carroll’s ‘Alice in Wonderland’, at one point the Red Queen says to Alice…

“it takes all the running you can do, to keep in the same place. If you want to get somewhere else, you must run at least twice as fast as that!”.

As a metaphor for the software development industry, I’m sure that we are all sadly well-aware of the applicability of this problem to our professional world: as new technologies, new languages, new paradigms, and new techniques are introduced into what can best be described holistically as the software engineering body-of-knowledge (SWEBOK), we must all run in place on the learning treadmill or else risk becoming the next generation’s COBOL experts — desperately (and often highly) valued in a (relatively) few limited situations, but experts in a narrow skill-set that lacks broad applicability elsewhere in our industry.

Learning Fatigue

This situation is so prevalent that a term has begun to enter our professional lexicon to help describe it: Learning Fatigue.

To most of us that were drawn to the software engineering profession at least partly because of the rapid pace of change rather than out of a desire to satisfy a high-school guidance counselor that suggested we pursue a CS degree because “that’s who all the companies are hiring right now”, the concept of Learning Fatigue would seem to be at odds with a significant part of what interested us about the software engineering field in the first place.

When a new technology, platform, technique, idea, principle, pattern, or approach is introduced/released/suggested, my curiosity is piqued and my knee-jerk reaction is “I should go learn about that — it seems interesting and might help me be a better software development professional if I were to familiarize myself with that thing.”

So how do we explain the often very real phenomenon of Learning Fatigue if learning new things is a significant aspect of what drives us as professionals?

The Cost of Constant Change

Speaking personally for myself, I think its because the idea of constantly learning new things is at odds with another of the values that tends to drive many software development professionals: the desire to understand how things work at a deep and fundamental level.  One of the reasons that I got into software development was out of a genuine interest in understanding how technology works — an unquenchable thirst to know things…and to know them deeply rather than superficially; to be an expert (to the degree anyone can claim that) in the things with which I must deal to do my job.

Today, the pace at which even just the one (relatively) small corner of the software development universe represented by Microsoft and .NET technologies continues to spit out new things that I really should remain deeply aware of is simply too great for me to absorb any longer.  Epitomized by the release of the W*F technologies released with .NET 3.0 (Windows Presentation Foundation, Windows Communication Foundation, and yes — even the now-nearly-abandoned step-child that was Windows Workflow Foundation), this single release of technology represented an increase in API surface-area many orders of magnitude larger than all of what had existed in .NET to that point.  Although I can’t seem to find the citation to offer up right now, the factoid that stuck with me from the release of WPF was this one:

The entire surface area (classes, methods, etc.) exposed by WPF was something like three times the number of all of the classes and methods that had existed previously in the entire .NET 2.0 framework Base Class Libraries.

Given the massive numbers of people that Microsoft has grinding out all this framework and technology for me to use, its simply impossible for me to deeply know everything about everything — even in the (relatively) tiny corner of the software development universe represented by Microsoft and its developer and platform technologies.

Instead, I find myself having to start to make uncomfortable compromises.  Instead of knowing many things deeply, I have to be satisfied with knowing many things at a much shallower level and only a (relatively) small subset of them at a deep (and eventually expert) level.  And I find this to be extremely at odds with what the software development professional in me tells me I should be doing.

The software development professional in me knows that to know a thing only superficially is largely useless knowledge that won’t empower me to use it in the best way a professional software developer should be able to do.  And so I am likely to create a work product of low quality, poor performance or efficiency, low maintainability, or just plain full of bugs due to my lack of knowledge in an area.  Which in turn makes having professional pride in my work a very challenging goal to achieve indeed.

This tension between what a person can know and what a person feels they need to know is the real root of the Learning Fatigue phenomenon.  Its not that we are fatigued from too much learning — instead its that we are simply too stressed out from trying to learn everything that we end up learning nothing in meaningful way.  And that fights with our fundamental value system in a way that is very difficult to resolve.

And this is made worse if you are in a situation like myself where a significant part of your job involves integrating Microsoft technologies and platforms with those of other vendors — even if you could solve the problem with the Microsoft-provided technologies and platforms, there’s a whole other set of parallel technologies with which you would need to remain current.

The Fallacy of JIT-Learning

The popular response I tend to hear offered in response to this challenge by one developer after another is…

I am a Just-In-Time Learner.  I will wait until I have a project that needs these skills and knowledge, and then I will go get them.

Sorry, but I have to call ‘Bullshit’ on this one.

Without understanding a technology in sufficient detail that can only come from actually experimenting with its use, I would submit that you are committing professional software development malpractice to make a recommendation for its use in any solution you are attempting to develop.  Only by knowing a thing at a deep and experiential level can one hope to be able to make the right choice about its use.

My favorite example of this is the ill-fated (and apparently now-acknowledged by Microsoft to be so bad as to need to be completely rewritten from the ground-up in .NET 4.0) Windows Workflow Foundation.  Unless and until you actually spent the time trying to dig into that technology in a serious way such that you would begin to actually understand its limitations, there is nearly no way on earth that you would have the knowledge to make a recommendation either for or against its use in your solution.

But the fallacy of JIT-learning seems to suggest that somehow one could make this decision and then learn about the technology you just recommended???

The Siren Call of Superficial Understanding

The real trouble with JIT-learning is that it encourages people to do only the most cursory examination of a thing, consider that they ‘know enough’ about it to get by, and then move on to the next thing.  Do this with enough things for long enough, and you can easily start to consider that your cursory, superficial examination of a thing is actually the status quo ante and a perfectly acceptable way for you to get through the otherwise overwhelming crush of new things about which one needs learn in order to remain current in their professional software development career.

This is the seductive siren call of superficial understanding — eventually it can replace your feeling of a need to learn any new things at a deep level and so you cease to do so, becoming completely satisfied that there is no longer a need to learn anything at a deep level.  But because this is now your SOP, you slowly reach a point where you fail to realize this has happened to you.

For an example of what can happen to otherwise competent software development professionals when they fall prey to this siren-call, I would encourage you to listen closely to some recent StackOverflow podcasts here and here and then to read this post by Robert Martin that also attempts to point out the absurdity of substituting superficial understanding for actual expertise and experience.