How to Write 3v1L, Untestable Code

July 24th, 2008 · 15 Comments ·

by Miško Hevery, Jonathan Wolter, Russ Ruffer, Brad Cross, and lots of other test infected Googlers

This guide lists principles that will help you write impossible to tests code. Or, avoiding these techniques will help you write code that can be tested.

  • Make Your Own Dependencies – Instantiate objects using new in the middle of methods, don’t pass the object in. This is evil because whenever you new up an object inside a block of code and then use it, anyone that wants to test that code is also forced to use that concrete object you new’ed up. They can’t “inject” a dummy, fake, or other mock in to simplify the behavior or make assertions about what you do to it.
  • Heavy Duty Constructors – Make constructors that do lots of work in them. The more work you do in the constructor, the hard it is to create your object in a test fixture. And if your constructor can construct other things that are hard themselves to construct, that’s even better! You want the transitive dependencies of every constructor to be enormous. Enormous is hard to get under test.
  • Depend on Concrete Classes – Tie things down to concrete classes – avoid interfaces wherever possible. (They let people substitute the concrete classes you’re using for their own classes which would implement the same contract in the interface or abstract class. Testing do-gooders might want to do this to get your code under a test harness – don’t let them!)
  • Conditional Slalom – Always, always, feel good when writing lengthy if branches and switch statements. These increase the number of possible execution paths that tests will need to cover when exercising the code under test. The higher the Cyclomatic complexity, the harder it is to test! When someone suggests to use polymorphism instead of conditionals, laugh at their thoughtfulness towards testing. Make the branching both deep and wide: if you’re not consistently at least 5 conditionals deep, you’re spoon feeding testable code to the TDD zealots.
  • Depend on Large Context Objects – Pass around ginormous context objects (or small ones with hard to construct contents). These will reduce the clarity of methods [myMethod(Context ctx) is less clear than myMethod(User user, Label label)]. For testing, the context objects will need to be created, populated, and passed around.
  • Use Statics – Statics, statics everywhere! They put a great big crunch in testability. They can’t be mocked, and are a smell that you’ve got a method without a home. OO zealots will say that a static method is a sign that one of the parameters should own the method. But you’re being 3v1L!
  • Use More Statics – Statics are a really powerful tool to bring TDD Infected engineers to their knees. Static methods can’t be overridden in a subclass (sometimes subclassing a class and overriding methods is a technique for testing). When you use static methods, they can’t be mocked using mocking libraries (nixing another trick up the pro-testing engineer’s sleeve).
  • Use Global Flags – Why call a method explicitly? Just like L Ron Hubbard, use “mind over matter” to set a flag in one part of your code, in order to cause an effect in a totally different part of your application (it’s even more fun when you do it concurrently in different threads!). The testers will go crazy trying to figure out why all of a sudden a conditional that was evaluating true one minute is all of a sudden evaluating to false.
  • Use Singletons Everywhere – Why pass in a dependency and make it obvious when you can use a singleton? It’s hard to set up a test that requires singletons, and the TDDers will be in for a world of hurt when all their tests depend on each other’s state.
  • Be Defensive – They’re out to Get Your Code! – Defensively assert about the state of parameters passed in methods, constructors, and mid-method. If someone can pass in a null, you’ve left your guard down. You see, there are testing freaks out there that like to instantiate your object, or call a method under test and pass in nulls! Be aggressive in preventing this: rule your code with an iron fist! (And remember: it’s not paranoia if they really are out to get you.)
  • Use Primitives Wherever Possible – Instead of using a “cookie object,” pass in primitives and do all the parsing you need, every time you need a value. Primitives make people work harder by having to parse and massage them to get the data out — where objects are mockable (gasp) and nullable, and encapsulate state (who’d want to do that?)
  • Look for Everything You Need – By Looking for things you are asserting your objects dominance as the object which knows where everything is. This will make the life of tester hard, since he will have to mimic the environment so that your code can get a hold of what it needs. And don’t be afraid of how many objects you need to reach out to to, the more the harder it will be for test to mock them all out in unisin. If you are an InvoiceTaxCalculator, feel free to do things like: invoiceTaxCalculator.getUser().getDbManager().getCaRateTables().getSalesTaxRate(). Cover your ears when some testing weenie tells you about Dependency Injection, Law of Demeter, or not looking for things.
  • Use static initializes – Do as much work as possible when your class is loaded. Testing nuts will be so frustrated when they find out just loading your class causes nasty stuff like network or file access.
  • Couple functional code directly to the external systems it depends on If your product uses external systems such as a databases, file systems or a network, make sure your business logic is coded to reference as many low level implementation details of these systems as possible. This will prevent others from using your code in ways you don’t intend, (like small tests that run in 2 ms instead of 5 minutes).
  • Mix Object Lifecycles – Have long lived objects reference short lived objects. This confuses people as the long lived object references the short lived object still after it’s no longer valid or alive. This is especially insidious, both bad design, and evil, hard to test.
  • Side Effects are the Way to Go Your best bet is to perform a large number of side effect producing operations in your methods. This is especially true for setters. The more non-obvious the side effects better. Peculiar and seemingly irrational side effects are particularly helpful for unit testing. To add another layer of sweet creamy goodness on top, you want to make it possible to initialize your objects in an invalid state, with uninitialized member fields. Once you have achieved this, be sure to make calls on the methods of the uninitialized fields as side effects from your setters in order to cause SEGV’s or NPE’s, depending on your language’s vernacular. Why go to all this trouble? Clean, readable, and testable code that works, that’s why! Side effect free functions are for intellectual weaklings that think a function name should give some kind of an indication of what the function does.
  • Create Utility Classes and Functions/Methods – For instance, you have a String which is a URL you’re passing around (obeying “Use Primitives Wherever Possible”). Create another class with static methods such as isValidUrl(String url). Don’t let the OO police tell you to make that a method on a URL object. And if your static utility methods can call to external services as well, that’s even better!
  • Create Managers and Controllers – all over the place have these Managers and Controllers meddling in the responsibilities of other objects. Don’t bother trying to pull that responsibility into other individual objects. Look at a SomeObjectManager class and you have no idea what it’s going to do.
  • Do Complicated Creation Work in Objects – Whenever someone suggests you to use a Factory to instantiate things, know that you are smarter than them. You’re more intelligent than they must be, because your objects can have multiple responsibilities and be thousands of lines long.
  • Greenlight if-branches and switch statements – Go ahead, don’t feel dirty about nesting if-branches. It’s “more readable” that way. OO cowboys will want to have a whole polymorphic soup of collaborating objects. Say no to the OO-ist. When you nest and branch conditionals, all you need to do is read the code from top to bottom. Like a great novel, one simple linear prose of code. With the OO-overboard paradigm, it’s like a terrible choose-your-own-adventure kid’s book. You’re constantly flipping between classes and juggling patterns and so many more complex concepts. Just if-things out and you’ll be fine.
  • Utils, Utils, Utils! – Code smell? No way – code perfume! Litter about as many util and helper classes as you wish. These folks are helpful, and when you stick them off somewhere, someone else can use them too. That’s code reuse, and good for everyone, right? Be forewarned, the OO-police will say that functionality belongs in some object, as that object’s responsibility. Forget it, you’re way to pragmatic to break things down like they want. You’ve got a product to ship after all!
  • Use “Refactoring” whenever you need to get away with something – This is a word that Test-Driven and OO-goons like. So if you want to do something far reaching, involving new functionality, without tests, just tell them you’re “Refactoring.” It’ll trick them every time. No matter that they think you need to have tests around everything before you can refactor, and that it should never add new functionality. Ignore their hubbub, and do things your own way!

Java Specific

  • Final Methods – Use final classes and methods all the time. They can’t be overridden for testing (-; But don’t bother making fields final, or using value objects (without setters) – just let your objects’ state be changed by anything and anyone. No sense in guaranteeing state, it’d make things too easy.
  • Handcuff your users to Specific Types – Use instanceof as much as possible. This will make Mocking a pain, and show people that you’re in control of the objects allowed.

C++ Specific

  • Use non-virtual methods – Unless you need to override the method in a deep and scary inheritance hierarchy, just make the method non-virtual. If you make it virtual, a TDD zealot may mock your class! An even nicer trick is to keep your destructor non-virtual – then when the testing freaks try to subclass, whooooaoaaaaaa….
  • Never use pure abstract classes – Depending on pure abstract classes is a sure-fire way to let the TDD crazies inject stubs and mocks into your code, making it testable.
  • Macros are your friends – Always use #ifdef PROD and other compile-time switches to make sure the testies can’t get at your really important blocks of code and test them. In fact, this code won’t even run: until it gets to production!

Tags: Advice · Rant · Testability

15 responses so far ↓

  • Pete // Jul 24, 2008 at 9:57 pm

    Not sure how this is a problem – I thought you TDD zealots had version control systems that rejected code check-ins with less than 85% test coverage. :)

  • How to write untestable code « Quero ser Ágil // Aug 7, 2008 at 6:43 am

    [...] 7, 2008 de Rafael Mueller Este artigo, escrito por “Miško Hevery, Jonathan Wolter, Russ Ruffer, Brad Cross, and lots of other test [...]

  • Fabio Nascimento // Aug 7, 2008 at 12:38 pm

    I have several of these errors, but I am improving…

    Thank’s for the article.

    Fabio Nascimento

  • Jan Lohre // Sep 2, 2008 at 11:45 am

    nice article, although I disagree on mainly two points:

    1. Be Defensive – They’re out to Get Your Code!
    I usually am defensive on arguments of my api methods and I would tend not to sacrifice this defensiveness in my production code for tests unless I find myself in a situation unable to test.
    In the end I cannot change that deliberately lateron as it would be a breakage of runtime behaviour.

    2. Final Methods
    Here I disagree even stronger as my api would then allow overriding/subclassing without being designed for inheritance. (For internal classes I would be in the same situation as in 1, when I cannot write a test because of it, only then I would think about removing the final keyword)

    I would even go so far to consider Java having the wrong default (an unfinal keyword would be the way to go)


  • Brendan Miller // Sep 10, 2008 at 4:19 pm

    I agree with your points excep for:

    1. Your aversion to final classes. Use composition not inheritence. Inheritance breaks incapsulation in many ways.

    2. Your aversion to asserting non-null. Null breaks type safety and assertions are needed to validate that the function contract (supply me with arguments of a certain type) is actually being honored. A null string reference is not a reference to a string *at all*. Null references by default are a hold over from C and assembly pointers, they are a misfeature in a strongly typed language like Java or C++ (although this can be fixed with smart pointers in C++). Many strongly typed languages, like ML, use non nullable references by default and require specification of nullability in the type for this very reason.

    3. Your aversion to utility functions is the only thing I think is truly rediculous. Utility functions *increase* testibility not decrease it. Separating functions from a class that aren’t involved in the maintenance of the classes invariant also increases the encapsulation of the class. To test that the class cannot be transformed into an invalid state, you only need to test methods that actually have access to that state.

    Also a separate function that does not have priveledged access to your Url class can be tested for functionality without having to double check that it violated a class invariant, because it *can’t* violate one.

    Big classes with lots of helper methods are common in java, but this is actually very bad. Classes should only have one responsibility, e.g. to manage some resource, or maintain an on a piece of data, etc. Adding lots of utility methods into the class viol

    That said, a utility function (or static method in java) should not be stateful in any way, and by that I also mean it should not use external services that aren’t passed into the function as arguments. However, this it is true of *any* object that it is bad to access global state.

    Utility functions aren’t really an OO idiom, but they are a *good* idiom, and they are not hostile to testing. OO is great, but it isn’t the only thing in our bag of tricks.

  • Johan Haleby // Nov 9, 2008 at 1:39 am

    Many of the things mentioned here such as static methods, final classes and making your own dependencies etc can actually be unit tested in Java. It’s just that the most common mock frameworks fail to do so. We have developed an open source framework to deal with many of these issues called PowerMock that is just an extension to EasyMock. What’s important to understand is that good design and testable design is not always the same thing.

  • Josh Szepietowski // Dec 1, 2008 at 10:00 pm

    Hello, great post but I had issues with a few items. For a little background I am a bit new to TDD/XXDD and unit testing in general and I am in the games industry so my requirements may be a little different, but I feel we are *mostly* on the same page. :)

    * Use Statics – This is an attack on both statics and free functions.. but it violates Meyer’s Principle of least encapsulation: if a method depends only on the public interface of an object it is less binding to have it as a static/free function than it is a member function… object.Method() is a nicer syntax than Method(object), but it attaches less baggage to Object so it is more flexible and orthogonal. Trading this for easier test harnessing is not a good trade. That said.. This is a point you stress strongly so as my experience with unit testing grows I may come to agree with you. :)

    * Be Defensive. This is a flat out assault on the tenants of Design By Contract and I won’t stand for it! DBC and TDD go together like a hand and a glove! If passing NULL is an invalid action (a bug, a contract violation) than YES assert the fuck out of it! This doesn’t mean the method is untestable: test to make sure it asserts on invalid input (do make sure the test framework can catch the asserts for standard fails and anticipate them as a pass condition in some cases)! And then give it valid input to test it succeeds on. I really don’t see what you are getting at here. Do you want people to be able to pass in erroneous data and get back a valid result? Just plain wrong IMHO.

    * Use Primitives Wherever Possible – The less ‘cookie objects’ you introduce, the less dependencies your code has, the more orthogonal and modular it is. Wrapping a primitive in an object for the sake of mocking is just insane to me: what are you going to mock? It generating a given value? This is my 2nd least favorite point (behind the before-mentioned assault on DBC)

    * Side Effects are the Way to Go – This wins my award for the line I agree with most. Side effects are something I see all the time in production code and they ALWAYS end up generating subtle bugs, injecting tendrils of hidden coupling, and making it really unpleasant to refactor things. They can be the result of Singleton overuse, global state flags, or a million other things. They are very very easy to add yourself (often when you need to add a little extra functionality to an existing system and don’t want to muck things up too much). As such they are becoming the design flaw I watch out for most of all: Every class, every method, must have a laser-focus purpose. Any side effects are unacceptable and if it means a major refactoring when a tiny feature is added this is a price I must be willing to pay, because the seemingly cheaper path of just adding a few hidden state variables is far worse in the long run.

    * Create Utility Classes – I take issue with this.. see my prior point about the ‘static methods’ for my argument, because it is the same here. Reduce coupling! Meyer’s! Yaaaaaghh!

    * Use non-virtual – in the games industry we know.. we know. We know that virtual == overhead. We don’t strive for bad design and we pay the piper where it makes sense, but we don’t make things virtual for the sole sake of testability. But being virtual does enable mocking, so perhaps a compromise is a TEST_VIRTUAL macro that says ‘virtual’ when in unit-test land and nothing in other cases, such that unit-test mockings can override it safely? I dunno, maybe that is a bad idea.. In any case if it is a class that you are gonna wanna mock, you are gonna wanna have the class implementing an abstract base class (interface) in the first place I would assume.

    * #ifdef PROD is silly. #ifdef DEBUG is fine. Tests should be running which full debug code active IMO (even if optimizations are turned on) since this helps them do the tag-team of awesomeness with ASSERTs you are using for poor-mans DBC.

  • lance // Dec 15, 2008 at 3:18 am

    I don’t understand how you can in one paragraph be advising the use of the polymorphic soup, and in the next saying avoid util classes? Is your objection to giving classes names with ‘Util’. Or are you talking about those monolithic massive util classes one sometimes sees, with hundreds of not-quite-related methods?

    Say no to the OO-ist. When you nest and branch conditionals, all you need to do is read the code from top to bottom. Like a great novel, one simple linear prose of code. With the OO-overboard paradigm, it’s like a terrible choose-your-own-adventure kid’s book. You’re constantly flipping between classes and juggling patterns and so many more complex concepts. Just if-things out and you’ll be fine.
    ■Utils, Utils, Utils! – Code smell? No way – code perfume! Litter about as many util and helper classes as you wish. These folks are helpful, and when you stick them off somewhere, someone else can use them too. That’s code reuse, and good for everyone, right? Be forewarned, the OO-police will say that functionality belongs in some object, as that object’s responsibility. Forget it, you’re way to pragmatic to break things down like they want. You’ve got a product to ship after all!

  • Jason // Jan 5, 2009 at 3:36 am

    It’s already been said by almost every comment here…. but I just have to say it again! Assertions are a VERY good thing!!! How can you possibly take issue to this? I use them all the time, especially in library and bridge code. They’re EXTREMELY useful in debugging… remember that? Debugging? Fixing bugs and issues??? Also, every language that I know of that has a mechanism for assertions also has a way to disable them at compile time or at run time. Given that fact, it absolutely boggles my mind that you would take a negative stance on them. Like Josh so eloquently states, you want me to ignore bad input data in my library just so that you can test that my library will accept the bad input data you wish to generate during testing, but you can’t be bothered to test for the fact that my library will throw an assertion exception? Puh-leeze! That’s really the kind of nonsense that gives TDD purists a bad name.

  • notepuddle : TDD and Commerce Server 2007: Getting Started // Jan 22, 2009 at 9:23 pm

    [...] Context object pain. See Misko here [...]

  • Houen // Aug 25, 2009 at 10:15 pm

    This is hilarious reading – i will be SOO 3V1L on my next project. Thats right testers – work that overtime!

  • Pablo // Oct 26, 2009 at 7:00 pm

    It’s a really fun and good post. I see many of these items everyday at work, I’ll send it by e-mail tomorrow!


  • Nikos // Aug 18, 2010 at 10:05 am

    funniest thing ive read in ages

  • Start using Unit Testing today | Qubiz blog // Mar 25, 2012 at 11:21 pm

    [...] and an article that approaches the issue from the avoid-untestable-code perspective [...]

  • Osvaldas Grigas // May 31, 2012 at 5:49 am

    Great article overall, but the bit about “Being Defensive” got me scratching my head. As many other commenters, I disagree that asserting method parameters is a bad thing. To me, it’s all about failing fast as opposed to letting the potentially illegal value cause a failure further down the line of processing.

    After all, isn’t Guava’s Preconditions utility meant exactly for these kind of design-by-contract checks?

    And how can it possibly hurt testability? E.g. if you don’t care about some String parameter in your test, why not simply pass “irrelevant-value” instead of null? That would actually increase the readability of your test.

Leave a Comment