Unit Testing isn’t going to save you time…now (but it won’t cost you either)!

At my current organization, we have been having a number of debates over the past 6-8 months about the relative benefits of writing unit tests for our code.  The opponents (ok, I will give the benefit of the doubt and call them ‘skeptics’ instead) trot out all the usual suspects here to bolster their skepticism:

  • it takes time (the inference here is too much time) to write unit tests
  • unit tests represent a brittle interdependency between two or more classes in my solution — when my production app changes, I have to ‘maintain’ my unit tests to keep them passing
  • I don’t need unit tests, any idiot can look at my code and know its working as intended (ok, this one is tongue-in-cheek, but the thrust is accurate)

Reviewing the non-Unit-Test developer workflow

Putting aside for a moment the debate about test-first (test-driven) -development vs. test-after development work styles, let’s take a look for a minute at a typical development scenario where there are no unit tests involved in the process…

  1. developer creates a new class (lets call it MyClass), writes code to create a public method that does something; doesn’t matter what, let’s call it MyMethod().
  2. developer needs to execute that method to ensure it works as intended but the method can’t be run by itself because its in a class in a class library project (which cannot be set as the start-up project type for the Visual Studio debugger)…oops!
  3. to solve this, the developer creates a stub or throw-away project (a console app if they are smart and efficient, a winforms app if they are just smart, or a webforms app if they are gluttons for punishment and complexity smile_embaressed)  designed to do nothing more than just new-up an instance of this newly-written class (MyClass myClass = new MyClass(); ) and call the newly-written method on it ( myClass.MyMethod();)
  4. using the VS IDE debugger, the developer steps into the body of MyMethod()  and starts inspecting the values of various private member variables at different points in the body of the method
  5. eventually, the developer gets to the return statement in the method and inspects the value about to be returned from the MyMethod() function to ensure its the right value as expected
  6. satisfied that the MyClass.MyMethod() is performing as expected, the developer ends the debugger session and moves on to the next development task in the project

I don’t think there is a non-unit-test-writing-developer out there that would quibble substantially with the preceding overview of how they work.  There just isn’t much choice in the matter and this pattern repeats over and over again through the entire project until all the work is complete.  The only variation has to do with the case where the results of step 4 or 5 don’t come back as intended and the developer has to modify the inner workings of MyMethod() to fix whatever incorrect value is returned from step 5.  But this also is completely typical and repeats hundreds of times during the development of any software project.

Reviewing the Unit-Test developer workflow

Why do I point out the above workflow?  Because its actually eerily-similar to the workflow the same developer would go through if they were writing unit tests!  Let’s evaluate this statement in some detail by looking at the steps this same developer would go through if writing unit tests to support their work…

  1. developer creates a new class (lets call it MyClass), writes code to create a public method that does something; doesn’t matter what, let’s call it MyMethod().
  2. developer needs to execute that method to ensure it works as intended but the method can’t be run by itself because its in a class in a class library project (which cannot be set as the start-up project type for the Visual Studio debugger)…oops!
  3. developer doesn’t need to create a stub or throw-away project because the unit test runner is capable of executing class libraries without an accompanying .exe explicitly written by the developer; developer starts a new test-project and news-up an instance of MyClass inside a new test-method.
  4. the unit-testing developer can skip step 4 (where the non-unit-test-writing developer uses the debugger to step into the inner-workings of the MyMethod() method-body); the unit-test-writing developer doesn’t care about what’s going on inside the method, just that its return value is correct — the intent of the unit test is to confirm the publicly-accessible behavior of the method is correct, not that the internals are doing something in a specific way
  5. the developer doesn’t need the debugger to inspect the return value from the call to MyMethod() since he/she has instead written an Assert.<somthing> statement in the their unit test method that will inspect the return value from MyMethod() and the unit test framework will report back that the Assert passed or failed.
  6. satisfied that the MyClass.MyMethod() is performing as expected, the developer moves on to the next development task in the project

Hey stupid — you’re actually doing unit testing already even if you don’t know it!

The point of investigating these two workflow patterns compared to each other is to realize that even if you aren’t actually writing unit tests, you are still doing all the same work!  Without unit tests, you are doing all the work manually and with unit tests you are asking the computer to do the work.  But the take-away is that either way, you are actually following the exact same process.

So if this is true, why write the unit tests?  Let’s look at some of the commonly-touted benefits of unit tests for a minute…

  • unit tests are repeatable; they can be run again and again throughout the life of the project to ensure that something just changed hasn’t inadvertently broken something else
  • unit tests are a statement about the valid behavior of your software; the method signature public int MyMethod(int) only says that MyMethod() is a function that takes and int and returns an int whereas one or more unit tests can bear out that the input int needs to be non-negative and the return int should be non-zero for example
  • unit tests are a way to convey to any developer on the team the intended behavior of your code; in the preceding example, there are several ways to tell the rest of the team what you had in mind for the MyMethod() function call: you can litter the function itself with comments, you can expect someone to read through the entire method body to completely grok the entire function and determine its restrictions for themselves, or you can write one or more unit tests that not only communicate your intent to the rest of the team, but allow the computer to validate that your intent hasn’t been inadvertently violated by others modifying the internals of MyMethod().

Not codifying your work in Unit Tests is actually less-efficient than writing the tests!

The bottom line is really this: the non-unit-test-writing scenario presented above has the developer going through nearly the same effort as the unit-test-writing developer.

But in the first scenario, all of the developer’s effort to get to a ‘success’ in the development of the MyMethod() function is completely thrown out after the method is written.  For anyone else (or even the same original developer!) to reproduce this effort requires them to manually step back through the debugger all over again to re-investigate whether they may have inadvertently broken the MyMethod() function’s behavior by any changes that they have made to it directly or to other parts of the codebase that indirectly affect MyMethod().  This need to repeat all this overhead leads to ‘developer testing laziness’ where eventually the application is just too large to step through everything in the debugger to ensure an accidental side-effect wasn’t introduced in one area by a change in another.  Instead the developer just ‘knows’ (guesses! … hopes!) that nothing else was adversely effected by the work they just did.

And weeks or more later, who’s to say what even is the proper behavior of MyMethod()?  Is it in the comments within the method that someone may have failed to bother to maintain?  Is it just in the method signature that says ‘thou shalt return an int‘ ?  Without unit tests that can be run to validate that expectations are still being properly met by the codebase, who is actually to say what’s correct?

Definitions of Legacy Code

Everyone reading this (assuming you are a developer with more than 2 months experience in the real world) has at one time or another likely been faced with a ‘legacy application’ that you need to extend or maintain to some degree.  There are several definitions of legacy code that are traditionally bandied-about…

  • code written by anyone but yourself smile_teeth
  • code written in a deprecated language (i.e., COBOL)
  • code written in a ‘current’ language but in a deprecated style (i.e., using C# but as if a procedural programmer got ahold of the framework libraries reference without understanding OO concepts)

But my favorite definition is perhaps the simplest and most-encompassing:

  • any line of code not covered by a unit test! smile_wink

Why is this my favorite definition of legacy code?  Because the main reasons why the preceding definitions are used by so many to identify legacy code are that they all share some of the same characteristics…

  • non-resilient to change; interdependencies abound and a seemingly innocuous change in one small corner of the codebase can have disastrous and unanticipated consequences for the rest of the application
  • no (efficient) way to make a change and discover if you have inadvertently changed something that has a cascading impact on other areas of the codebase; this leads you to have whole sections of your application that are just ‘hands-off’ for making any changes because "nobody knows how it works so we dare not touch it".  Over time, eventually the entirety of your application is like this, leaving no way at all to modify anything and user requests like "can we change the title of that window?" take two weeks to accomplish because the window title was actually the way the app knew how to process requests or something equally bizarre and to change it means touching — and manually retesting — everything in the whole app all over again!)
  • poor (if any) documentation about what’s supposed to be the intended behavior of the various parts of the codebase; sure the compiler can tell us if our changes are syntactically correct, but who knows if the entire app will still compile fine but return the wrong answer to some crucial calculation it makes?

Conclusion

The most important thing about this definition of ‘legacy code’ (any line of code not covered by a unit test) is that its got nothing to do with its age.  Code written last week can look like legacy code even to someone else on the same project.

So, if you write code without unit tests then congratulations to you: you have just written a brand-spanking new line of legacy code that nobody (not even yourself after even a surprisingly trivially-short period of being away from it) will be able to effectively maintain.

But if you do write unit tests, then you have ensured that anyone (including yourself) that comes back to make changes to the code you just wrote will be able to effectively, efficiently, and safely make those changes in a minimum of time and with a maximum of confidence that nothing else has been broken.

And if you’re lucky enough to have been on the one software development project that never needed to have its code edited, never needed to have its capabilities extended, and never needed to have a bug fixed, then please drop me a note — we’d LOVE to hire you at any salary level because that kind of perfection is priceless in this industry smile_omg.