Automagic Tooling and the Dumbing-Down of Developers

2

The prolific writer Arthur C. Clark once (famously) wrote:

"Any sufficiently advanced technology is indistinguishable from magic."

As software developers, we are presented just about daily with such ‘sufficiently advanced’ technology — the world of what one could know about software engineering techniques, languages, frameworks, databases, compilers, etc., etc., etc. is just so vast now that no one person would ever be able to learn it all in a single lifetime (and since I’m a skeptic about reincarnation, we’ll have to assume that a single lifetime represents a hard upper-bound on the time anyone has to learn anything smile_tongue ).  When faced with such advanced technology (and here ‘advanced’ really means nothing more than ‘that which a person doesn’t yet have the knowledge, expertise, etc. to fully comprehend the details of), everyone will be faced (at repeated points in their career) with ‘magical’ technologies.

The real question I think is not "will I be faced with ‘magical’ technology in my career?", but more "what will I do when faced with it?".

Automatic vs. Automagic

As I see it, you have two choices as a developer when faced with such a thing: just use it and not care about what’s going on under the hood, or take the time to learn about what’s really going on.

The purist in me says that you should always choose the latter rather than the former, but the realist in me understands that this is not always practical or even possible.  The difference between these two responses to ‘magic technologies’ leads me to what I call the difference between ‘Automatic’ and ‘Automagic’ things in your software project.

Automatic: ‘something just happens and the details are abstracted away from me and that makes my life easier but I know how and why this happens so I can effectively troubleshoot if there is a problem with the abstraction’.

Automagic: ‘something just happens and the details are abstracted away from me and that makes my life easier but I have absolutely no idea how or why this happens so I can’t possibly effectively troubleshoot if there is a problem with the abstraction’.

All of this matters because as Joel Spolsky so famously reminded us all some time back, all abstractions are inherently leaky.

Up the Abstraction Hierarchy

The history of computer science can be reasonably thought of as the quest for ever-higher effective abstraction layers.  All modern programming languages are in effect abstractions over assembly language which is in turn eventually an abstraction over binary logic gates of 1s and 0s.  If we didn’t believe in the value of abstractions, we’d all be programming in code that looks like…

0001010101 0111 01011010111101 01101101 1011011 110000110110101010

…and we have to admit that this isn’t very expressive or efficient smile_cry (at least not for me!).

But the question in the presence of all this really has to be: "how many of us could write code in even assembly language, never mind 1s and 0s ?"  Some, to be sure (and I bow at your altar), but the vast majority of otherwise effective programmers can’t (and shouldn’t have to).  This is a layer at the root of their abstractions that they just don’t need to understand to be effective software developers.

And why not?

The value of Abstractions

David Wheeler is often credited with the following quote:

"Any problem in computer science can be solved with one additional layer of indirection."

Taken out of context, this seems somewhat tongue-in-cheek.  But the entire less-frequently mentioned quote is actually more useful (and significantly less flippant):

"Any problem in computer science can be solved with one additional layer of indirection.  But that will usually cause another problem."

This full quote demonstrates the concept that every abstraction introduced actually represents a trade-off  — between the value the abstraction is designed to add and the cost of whatever new problem the abstraction introduces.

In general, we are accepting of abstractions that provide value well above the new pain they add to our lives; we accept that we cannot (easily) get at the underlying 1s and 0s in our code since the abstraction of coding in 3GL is both so rich and so water-tight that it rarely ‘leaks’ (too badly!) on us.  Interestingly, a lot of the failure of wider-spread adoption of 4GL (and 5GL) approaches can probably be traced to the fact the 4GL and 5GL abstractions ‘leaked’ too much for a lot of cases (which isn’t to say that these types of languages have no value, its just to say that their problem domain and their abstraction need to be very closely aligned for them to be appropriate abstractions to introduce into your work).

I hate ‘Automagic’ things: rule of thumb for survival in a highly-abstracted world

Generally, I hate Automagic things in my code.  I would venture to say that most professional developers who take pride in their work and want to genuinely understand why their software either works or doesn’t would say the same thing if asked.  But as we’ve seen, this isn’t really a true statement.  At the root of each layer of abstraction lies another abstraction in the stack and there comes a point of diminishing returns with understanding underlying abstractions unless we all want to say that to be a good developer today means we all have to be silicon-chip designers in our spare time smile_wink.

So what makes an acceptable level of understanding of one’s abstractions?  I think it works like this (as a general rule): learn the guts of the abstraction layer immediately beneath where you are working.  If you’re in .NET and dealing with the CLR, understand the memory model on a PC, learn about the difference between stacks and heaps, understand the threading model in your processor, and make damn sure you understand how that hard drive works (seek times, etc.).

Why does any of this matter?

Because someone asked me the other day about the challenges of bringing a new dev into the team on a project that used NHibernate for O/RM and data access, Castle Windsor for DI container services, Log4Net for logging support, etc., etc.  The question was: can’t this person just start consuming the services exposed by these tools and not really have to understand what’s going on inside them to get started?

The answer (of course) is ‘sure they can’ but that’s only a temporary stop-gap solution because until the person is able to transition from these things being ‘Automagic’ to just plain ‘Automatic’ the person is at a tremendous disadvantage in working on the project because these things are merely a single layer deep in the abstraction hierarchy for the project and they are going to stumble into their leakiness sooner rather than later — its just about guaranteed.