To the unintiated, the title is: The Safety Net - Test-Driven Development In Real Life For the Win. Warning - This post contains some of my thoughts about *practical* TDD and the reasons for it. This may not all encompass "dogmatic" TDD practice. To me, TDD is all-important first as a safety net. Before you find TDD soluble enough to understand how it helps you drive design, you have to understand that you're putting it there to save your code from you, your QA group, and most of all, changes.
When I think about TDD, I think about it as an applied methodology and try to determine what real-world scenarios it can improve. Often I hear from people that they don't "get" TDD, that it seems like a lot of extra work, or that they can't seem to find a use for it IRL. Often, I can see where this kind of thing comes from. Much of what is available out there puts the concepts of TDD in a vacuum and segregates it from the problems it can solve, or doesn't give concrete examples of the benefits of the methodology.
Let's take a step toward changing that. I want to talk about something I had today, and basically recap (and expand upon) a conversation I had with one of my developers regarding unit tests.
The Scenario
A piece of functionality released a few months ago is discovered recently to be buggy. As a matter of fact, recent reports lead us to think that the actual code was never released into the wild at all, as it appears to have completely disappeared. The human testers tested and approved this function (which amounted to an if statement checking a boolean field on a domain entity - pretty simple change). The code was released live. We didn't hear anything for a couple of months, then we start getting a flood of emails telling us that it clearly isn't working. The testers go back and test, it works for them. The developer goes and looks in the code - the check is there. However, clearly demonstrated reports from users show that the expected functionality is in fact not there.
The Root Problem
We dive into it for a little bit today to find the problem. From a human tester perspective, we are finally able to reproduce the problem by testing some scenarios that they never tested in the first place. From a code perspective, once we removed the blinders that were causing us to look strictly at the line of code implemented, it was easy to spot the problem.
public bool CheckAllowedFunction()
{ try
{ bool securityCriteriaOne = SomeDomainFunctionThatCouldFail();
bool securityCriteriaTwo = TheAddedCheckFunction();
if(securityCriteriaOne && securityCriteriaTwo)
return true;
else
return false;
}
catch
{ bool securityCriteriaOne = SomeOtherDomainFunctionForFailover();
if(securityCriteriaOne)
return true;
else
return false;
}
}
Do you see it? What happened is when the code was first written, securityCriteriaOne was the only check. A year later, when we implement the new request, we add securityCriteriaTwo to satisfy the request, and the check for securityCriteriaTwo was never added to the catch clause (this is a very simplified example, and not entirely representative of the code involved, but it is the basic gist with all the business-sugar stripped out).
Ok, so this is a simple enough mistake. You're flying through, you implement something that you've already decided in your head is one simple line of code (developer bias) and you miss a step. Maybe you hit F5 and run it real quick with your favorite bit of test data in the test system, it works (because your bias led you to test for pass rather than for fail). You build it and ship it to QA. They test. They try to break it, because they hate you. But maybe your QA group isn't strong enough to understand and think through possible breaks. Maybe they have their own set of data they test with, and maybe they got the same bias from the spec that you did and subconsciously tested for pass. Maybe, as in this case, the bug is such that by sheer luck (or Murphy's Law) it looks like it works live long enough that you forget about it. Maybe SomeDomainFunctionThatCouldFail(); is only gonna throw an exception on very rare circumstances, or because of some variance in data (as was the case here). Bottom line, it gets missed.
The Cost
Ignoring any potential business costs for now, because those things are intangible until they happen, let's focus on the cost of this bug to development.
I lost an hour of time when I should have been doing something else. My developer lost an hour of time when he should have been doing something else. The bug was critical enough that he had to interrupt what he was doing for a very important iteration currently. The QA guys lost time testing the current iteration to go back and test (under my direction) a plethora of scenarios. The DBA lost time modifying and creating various data scenarios in the database as I had different scenarios tested. My other developer lost time because well, it was exciting, and we interrupted his work, and it was too interesting not to watch. The VP of IT lost time because he had to stop to get a full understanding of what was going on so he could report up. Pretty much an entire IT department lost an hour plus of time JUST TODAY, not including previous days of trying and *not* being able to identify the bug, just for this simple one line of code.
Clearly, that is unacceptable.
Human QA is Unreliable
Look, I love my testers, and they do a good job. They aren't really my testers though. They're tech support guys that do testing when they aren't answering calls or fixing servers. They aren't trained as testers. They don't have programming experience. They simply don't have all the experience needed to create and execute rigorous test scenarios covering wide ranges of data and usages. They are learning and improving every day, but to expect them to be QA rock stars would be unfair to them and delusional at best.
And I think I have it good. Many of you may not have QA people at all. I've been there more times than I can count. Maybe you test your own code. Maybe nobody tests it (been there too).
Not all of us is lucky enough to have real, dedicated, experienced software QA people testing our apps. And even those of us that do still have bugs. Hello Microsoft? You think they don't have QA people that are probably better coders than me? You're wrong.
Humans are fallable. So are their tests. You need to be in a position of comfort with the robustness of your code before it ever sees another human. You need to know that you caught and handled that Null Reference because John McTestalot might not get it this time. You need the unit-test safety net.
How Does TDD Prevent This Scenario
Well, it's not magic. However, mindful application of test-driven practices will prevent problems like this from happening. In our current iteration we are putting out a lot of functionality. We're releasing to QA multiple times a day, and they are coming back with a lot of bugs. These bugs are, with rare exception, very simple errors and things that got overlooked. Off by one. Bounds checking. Null reference exceptions. Basic stuff.
My guys are writing tests. Only recently have we gotten to the point where we're getting decent testing going, and not putting it off because we "don't have time". The tests are far from perfect, because they are *very* new to TDD, but they're getting there, and they're learning. After reviewing some of the bugs, I told them to learn from these bugs, and start incorporating more thorough tests, and more extensive test scenarios. A bounds error or a null reference should never make it out of your IDE, let alone into the hands of another human. These are things that are easily caught with good unit tests.
And that was the case here. No tests. In my estimation, the original code would, at the very least, have had tests that hit both the try and the catch senario and got its true/false assertions in all 4 places. When the new requirement came in, the tests would then have been augmented with a new bit of code to test securityCriteriaTwo. Then the code would have been written. Then one of the tests wouldn't have passed, the code would have been added to the catch block, and three months later we don't lose 15 man-hours on one if statement.
5 minutes to write a simple test 3 months ago, or 15 man hours to track down the bug today? Which do you think is better?
Why Tests Drive
It's TDD, not DDT (development-driven testing). The tests are first-class citizens. They should come first. No matter how you accomplish this, tests should be driving your new code, not the other way around.
As you get better and better at TDD you will see how it drives not only your code, but your design. But for now, let's focus on the reasons for it driving your code.
The word today is BIAS. Developer bias is your enemy. As I said before, you may look at a requirement, decide in your head it's easy, implement it quickly, then make a test for the implemented function, then maybe make a test for a fail condition so you feel good, then you're done. Bias kicked your code in the nuts the second you typed your first curly brace. You aren't going to go after the fact and write good tests, because you are biased toward the implemented function. You are interested in seeing it work.
Now reverse it. Let the test drive. You write a test to satisfy the basic criteria of the requirement. You then write exactly what code will make it pass. You then write another test with the intention of making it fail. Then you write exactly what code will make it pass again. Lather, rinse, repeat until you simply cannot think of another way to make it fail. Then you're done. You wrote a bunch more code, sure, but by thinking about the the test over the code, you put out a much more robust function.
Think of it this way: The TEST is the important code. The application code only exists to satisfy the tests. You are trying to implement a full range of solid tests, and then, because you have to, write some code that makes them pass. Putting the test front and center in your consciousness removes the bias toward the application code.
TDD Takes Too Long
I think I've already demonstrated the gains here. 15 man hours of productivity lost today due to less than 1 man hour of time not being spent months ago. Find me a sane manager who will take that trade and I'll help you find a new job because trust me, you don't want to spend another minute with that guy.
But, you say, TDD seems like insurance. It's only useful when it comes up. Sure, this is true in a way, but think of it less like insurance and more like prevention. You are actively preventing disaster, not trying to just cover yourself in case something happens.
But, you say, it takes a long time, and it's a lot of code to write, and I have more test code in my project than I do real code! Yeah, that'll happen sometimes. That's not necessarily a bad thing. As for the time it takes, well, it takes practice. You don't write perfect tests at first, you don't have good scenario coverage at first, and you feel awkward and unnatural writing tests at first. You gotta practice.
Did you always try/catch/finally when you were cutting your teeth in code? Or did it take many many times of unexpected unhandled exceptions before you started to learn? When you put in a try/catch/finally today, does it feel like so much extra code? Does it take any measurable time that you feel like you should be applying to other code? Of course not. Unit tests are the next step. They are your next try/catch/finally. Keep at it, knowing the benefits, and they will become a natural, flowing, and quick(er) part of your code-writing.
And they will be your safety net.
Tags: TDD Software Writing