Tuesday, January 8, 2013

Tools only give you the ability

Recently I read a post on the Alt.net linked-in group Would mocking help prevent these software bugs? talking about the Knight Capital Group Inc.’s disaster, asking the question "if doing more testing and using mocks would have prevented it?"

Doing unit testing with mocking is a tool, not a silver bullet. Unit tests mixed with behavior testing can be very effective at finding these kinds of bugs and Mocks allow you to create and test against foreseen conditions.  While TDD can help expose bugs, unforeseen data conditions can pass though unnoticed, tests only test what they are told to.

Something else to keep in mind is behavior and unit tests are only a small part of the testing tool box that also includes: integration tests, performance tests, automated UI tests, etc.  and that’s what these are tools.  How effective they are depends on the user, a funny quote I once heard is “A tool runs tools, a craftsman uses them”.  You can run your code coverage tool all day long and say “look I have 80% code coverage” but this doesn't mean your code is well tested, it just means it has test that run though it, and can hide untested code. 

One commenter on the article said something to the effect that what they where working on requires to much performance to be testable.   I would like to state this is B.S. you can make testable code that has just as much performance as not testable code. You may have to change how you test the code, or use different design or testing techniques for example Dependency Injection isn't required to make testable code, if it's not fast enough use greedy constructors or lazy load properties. If you need to make a black box class for performance, you simply do integration tests around it.

In the end, saying TDD isn’t of value because it didn’t catch bug xyz is like a carpenter blaming his hammer for hitting his thumb and not the nail.  On the other side it’s just as ridiculous to say “I can build a house” simply because you own a hammer and some nails.