I just gave a talk at last week’s ALT.NET NYC user group introducing Domain Driven Design (DDD) as a pattern language and included in there was a slide expressing my opinion “Once: its a fluke, twice: its a coincidence, three times: its a pattern” intended to posit the idea that patterns evolve from our observations of repeat phenomenon rather than being invented by anyone. I tend to think the same is true for patterns in life, business, and the software profession as well and I just recently had a confluence of experiences that tell me its time to identify a new pattern:
The present pace of change in the software development industry precludes anyone from being able to ever become deeply knowledgeable in more than a tiny handful of areas.
The Red Queen Hypothesis
Like most patterns, this one has its roots in other previously-observed phenomenon. In 1973, the Red Queen Hypothesis was identified as a way to consider and understand the effects of a constant (or even increasing) rate of change in a structured system. A reference to Lewis Carroll’s ‘Alice in Wonderland’, at one point the Red Queen says to Alice…
“it takes all the running you can do, to keep in the same place. If you want to get somewhere else, you must run at least twice as fast as that!”.
As a metaphor for the software development industry, I’m sure that we are all sadly well-aware of the applicability of this problem to our professional world: as new technologies, new languages, new paradigms, and new techniques are introduced into what can best be described holistically as the software engineering body-of-knowledge (SWEBOK), we must all run in place on the learning treadmill or else risk becoming the next generation’s COBOL experts — desperately (and often highly) valued in a (relatively) few limited situations, but experts in a narrow skill-set that lacks broad applicability elsewhere in our industry.
Learning Fatigue
This situation is so prevalent that a term has begun to enter our professional lexicon to help describe it: Learning Fatigue.
To most of us that were drawn to the software engineering profession at least partly because of the rapid pace of change rather than out of a desire to satisfy a high-school guidance counselor that suggested we pursue a CS degree because “that’s who all the companies are hiring right now”, the concept of Learning Fatigue would seem to be at odds with a significant part of what interested us about the software engineering field in the first place.
When a new technology, platform, technique, idea, principle, pattern, or approach is introduced/released/suggested, my curiosity is piqued and my knee-jerk reaction is “I should go learn about that — it seems interesting and might help me be a better software development professional if I were to familiarize myself with that thing.”
So how do we explain the often very real phenomenon of Learning Fatigue if learning new things is a significant aspect of what drives us as professionals?
The Cost of Constant Change
Speaking personally for myself, I think its because the idea of constantly learning new things is at odds with another of the values that tends to drive many software development professionals: the desire to understand how things work at a deep and fundamental level. One of the reasons that I got into software development was out of a genuine interest in understanding how technology works — an unquenchable thirst to know things…and to know them deeply rather than superficially; to be an expert (to the degree anyone can claim that) in the things with which I must deal to do my job.
Today, the pace at which even just the one (relatively) small corner of the software development universe represented by Microsoft and .NET technologies continues to spit out new things that I really should remain deeply aware of is simply too great for me to absorb any longer. Epitomized by the release of the W*F technologies released with .NET 3.0 (Windows Presentation Foundation, Windows Communication Foundation, and yes — even the now-nearly-abandoned step-child that was Windows Workflow Foundation), this single release of technology represented an increase in API surface-area many orders of magnitude larger than all of what had existed in .NET to that point. Although I can’t seem to find the citation to offer up right now, the factoid that stuck with me from the release of WPF was this one:
The entire surface area (classes, methods, etc.) exposed by WPF was something like three times the number of all of the classes and methods that had existed previously in the entire .NET 2.0 framework Base Class Libraries.
Given the massive numbers of people that Microsoft has grinding out all this framework and technology for me to use, its simply impossible for me to deeply know everything about everything — even in the (relatively) tiny corner of the software development universe represented by Microsoft and its developer and platform technologies.
Instead, I find myself having to start to make uncomfortable compromises. Instead of knowing many things deeply, I have to be satisfied with knowing many things at a much shallower level and only a (relatively) small subset of them at a deep (and eventually expert) level. And I find this to be extremely at odds with what the software development professional in me tells me I should be doing.
The software development professional in me knows that to know a thing only superficially is largely useless knowledge that won’t empower me to use it in the best way a professional software developer should be able to do. And so I am likely to create a work product of low quality, poor performance or efficiency, low maintainability, or just plain full of bugs due to my lack of knowledge in an area. Which in turn makes having professional pride in my work a very challenging goal to achieve indeed.
This tension between what a person can know and what a person feels they need to know is the real root of the Learning Fatigue phenomenon. Its not that we are fatigued from too much learning — instead its that we are simply too stressed out from trying to learn everything that we end up learning nothing in meaningful way. And that fights with our fundamental value system in a way that is very difficult to resolve.
And this is made worse if you are in a situation like myself where a significant part of your job involves integrating Microsoft technologies and platforms with those of other vendors — even if you could solve the problem with the Microsoft-provided technologies and platforms, there’s a whole other set of parallel technologies with which you would need to remain current.
The Fallacy of JIT-Learning
The popular response I tend to hear offered in response to this challenge by one developer after another is…
I am a Just-In-Time Learner. I will wait until I have a project that needs these skills and knowledge, and then I will go get them.
Sorry, but I have to call ‘Bullshit’ on this one.
Without understanding a technology in sufficient detail that can only come from actually experimenting with its use, I would submit that you are committing professional software development malpractice to make a recommendation for its use in any solution you are attempting to develop. Only by knowing a thing at a deep and experiential level can one hope to be able to make the right choice about its use.
My favorite example of this is the ill-fated (and apparently now-acknowledged by Microsoft to be so bad as to need to be completely rewritten from the ground-up in .NET 4.0) Windows Workflow Foundation. Unless and until you actually spent the time trying to dig into that technology in a serious way such that you would begin to actually understand its limitations, there is nearly no way on earth that you would have the knowledge to make a recommendation either for or against its use in your solution.
But the fallacy of JIT-learning seems to suggest that somehow one could make this decision and then learn about the technology you just recommended???
The Siren Call of Superficial Understanding
The real trouble with JIT-learning is that it encourages people to do only the most cursory examination of a thing, consider that they ‘know enough’ about it to get by, and then move on to the next thing. Do this with enough things for long enough, and you can easily start to consider that your cursory, superficial examination of a thing is actually the status quo ante and a perfectly acceptable way for you to get through the otherwise overwhelming crush of new things about which one needs learn in order to remain current in their professional software development career.
This is the seductive siren call of superficial understanding — eventually it can replace your feeling of a need to learn any new things at a deep level and so you cease to do so, becoming completely satisfied that there is no longer a need to learn anything at a deep level. But because this is now your SOP, you slowly reach a point where you fail to realize this has happened to you.
For an example of what can happen to otherwise competent software development professionals when they fall prey to this siren-call, I would encourage you to listen closely to some recent StackOverflow podcasts here and here and then to read this post by Robert Martin that also attempts to point out the absurdity of substituting superficial understanding for actual expertise and experience.
is http://www.autumnofagile.net is dead?… just curious 🙂
@Nisar:
Very much alive. In fact, screencast to be released tomorrow. I’m back on track after a little hiatus for personal + professional complications that are now settling down.
Thanks for the really interesting read.
Be interested to hear ideas about solutions to the problem of too much to learn and only enough time to do JIT (i.e. fairly useless) learning. Should you specialise and learn only a few areas of knowledge that runs the risk of quickly becoming redundant? Or use JIT learning and drill down into areas that seem to offer the most benefits? Or just stop sleeping? 🙂
Great post Steve. I have often thought along the same lines — someone once told me to increase both the breadth and depth of your knowledge. As you point out, these are at odds and striking a balance is difficult. I think by striving to increase breadth you learn at least what you don’t know. So, at a minimum, you are aware of possible alternatives. But, of course, pursing such a solution with shallow knowledge (as you point out, in a JIT fashion) is problematic. The other problem is too choose what narrow areas in which you should strive to become expert. Choose carefully, or you will become the Windows Workflow / Linq to Sql expert. 🙂
[…] The Red Queen, Learning Fatigue, The Cost of Constant Change, the Fallacy of JIT-Learning, and the S… […]
GREAT ARTICLE! Well said in every way. I have been suffering for some time with Learning Fatigue, constantly battling to stay ahead and finally realizing that I must pick what I think will serve my needs best and learn those as much as my time permits. I’m fortunate enough to work at Kennedy Space Center developing some very cool applications for NASA and our department is headed up by a very knowledgeable person who keeps up working on the front edge of technologies that are out now.
Hey Steve
Great post.
However, I’m not sure about the point you’re trying to make. (unless it’s more of a warning)
I don’t think that superficial learning is bad in itself. As you stated in your post it’s impossible to be an expert, or even ‘good’, in all technologies available to us. (unless your last name is Guthrie-Hanselman of course:-) )
IMO, superficial knowledge is way better than no knowledge at all. If you’re faced with problem X and you’ve read the books ‘learn technology Y in 10 hours’ and ‘learn technology Z in 10 minutes’, you’re better of than someone who hasn’t read them.
I guess the point I’m trying to make is that you as a developer should know what you don’t know (yet) and be honest (to yourself) about it.
An example: last summer I canceled my vacation to watch a NHibernate screencast series that some dude put up on the net. Does that make me an expert on NHibernate ? Am I’m getting tired of answering Ayende’s questions ? Of course not, but that ‘superficial knowledge’ gives me a possible alternative solution to my ‘problem’ and with additional study of the resources that I learned about, I might be able to actually implement it in my future projects.
>>
Without understanding a technology in sufficient detail that can only come from actually experimenting with its use, I would submit that you are committing professional software development malpractice to make a recommendation for its use in any solution you are attempting to develop. Only by knowing a thing at a deep and experiential level can one hope to be able to make the right choice about its use.
<<
I agree (although I think that ‘professional software development malpractice’ is a bit harsh) but what do you suggest then ? How does a developer (/small team of developers) make a recommendation about the use of technology X ? How do you achieve a deep and experiential knowledge of a technology without applying it in a real-world scenario ? (If our ancestors were sitting in their caves waiting until they had a deep and profound knowledge on how to kill deer, we’d still be using Cobol these days :-))
One
@One:
>>I think that ‘professional software development malpractice’ is a bit harsh
That’s a literary device called hyperbole 😀
For the record, I think you (and a few other commenters) got the point exactly right: KNOW WHAT YOU KNOW. KNOW WHAT YOU DON’T KNOW. AND KNOW THE DIFFERENCE. Its when we get confused about this that we find ourselves in real trouble.
>>(If our ancestors were sitting in their caves waiting until they had a deep and profound knowledge on how to kill deer, we’d still be using Cobol these days :-))
I love this example — great point. We learn by DOING in the software development world. This is true and I’m certainly not arguing against getting started DOING before having completely mastered a thing.
I think the point is more to manage risk by recognizing it; know thyself (and thy own limitations) and all will be fine (generally).
@David:
Since not sleeping obviously isn’t a viable option (for most of us that aren’t vampires or something), I think the best you can hope for is a good mix of humility (know what you know and don’t know) plus good planning to ensure you keep your skills deep in the areas that make yourself valuable and also taking the time to ensure your AWARENESS of other things remains high.
As an example, I’ve never bothered to write a line of WPF code because I just haven’t had a need to but I ensure that I read reviews, comments, feedback, and experiences from those that have so I can understand its pros and cons at a high level to ensure I can recommend when and when not to consider using it and I figure when I *do* need to write the code, I’ll have to JIT-learn it.
But I’ll recognize that my reading of blog posts about WPF is no substitute for what I’ll learn by actually DOING my first line of code in a WPF app and I will absolutely realize BEFORE I GET STARTED that this area of the proposed solution will have A LOT more risk associated with it since I don’t yet know how to program in WPF.
I’m just arguing against my mistaking my having read blogs about WPF with my actually having done some WPF work and I think this very kind of conflating reading about something with DOING something is what happened to the stack overflow podcast team in re: unit testing, TDD, SOLID principles, and other ideas and practices that they largely miscaracterized based on their having casually read some blogs about the topics.
I find their relative inability to recognize this difference to be reasonably disturbing and that was part of what motivated me to write this post about the more general ‘trap’ that they seem to have fallen into (which, to be fair, is a pretty common occurence when former software engineers hang up their IDEs to become entertainment pundits, magically at once able to consider themselves capable of commenting at-will on all kinds of things about which they clearly lack knowledge, experience, or authority 😀 ).
Thanks for the reply Steve. Completely agree with your point about being aware of vs. having expertise in an area.
Hey Steve … great post .. I think all of us developers who care for developing systems with quality pass through the same “problem” … that is that unfortunately we’ll never know everything ! Again … great post !
Steve –
Long time listener, first time caller…
Where I realized that I had fallen prey to JIT Learning was when I got tired of not really knowing a subject. The problem was that I was casting my net way to wide and trying to learn everything.
I am a new student to SOLID. The challenge that I have given to myself is to not worry about code implementation of the principles, just learn the principle. Try to learn how to restate it. For example, Open Closed Pincipal – Classes should be closed to modification, open to extension. When I try to explain that to some of my collegues, I get a “How they heck can you write code like that.” But then I state the QBQ (Question Behind the Question) of the principle: you seperate the stuff that changes from the stuff that doesn’t.
Thanks for all the Screencast Goodness!!
C
@Chuck:
Thanks for the feedback — and I completely agree with your strategy: learn the values and not the implementations. Over time, the values and principles will hold you in good stead for much if not all of your career whereas the implementation details are the most fleeting of skills.
Appreciate your taking the time to move from listener to caller 🙂
Hi Steve,
Great post as usual. I guess everyone needs to create their own coping mechanism to deal with the Learning Fatigue. Over the years, I found myself reading and trying to learn about principles and methodology -the knowledge that will not be obsolete with the next framework release.
In my opinion, it is less risky and more rewarding to learn SOLID principles, Design Patterns, TDD etc. because they have been around for years and are not likely to be obsolete soon.
Once you build a solid core (http://blog.jpboodhoo.com/BuildASolidCore.aspx) then you’re in a better position to evaluate the technology du jour.
If you’re evaluating a technology and you find out that it is difficult to work with that technology while adhering to your principles, that should be a red flag. Maybe this technology is not as great as it sounds.
My 2 cents.
Thanks,
Alper
[…] they wish to leave IT or stay in IT. Reviews employment trends in IT that may force IT project …Unhandled Exceptions Blog Archive The Red Queen, Learning …… software engineering body-of-knowledge (SWEBOK), we must all run in … Name (required) Mail […]