Clean Code Talks – Global State and Singletons

posted on November 21st, 2008 ·

by Miško Hevery

Google Tech Talks
November 13, 2008

ABSTRACT

Clean Code Talk Series
Topic: Global State and Singletons

Speaker: Miško Hevery

Video

Slides

→ 10 CommentsTags: Uncategorized

Changing the Developer Community – One Developer at a Time

posted on November 19th, 2008 ·

Recently, I have been giving talks about testability, and good software practices at my friend’s companies. My goal, is to change the developer community one-developer-at-a-time. I want to get to a world, where when a new developer gets hired, the question of testing does not need to be asked. It is understood that as a developer your job is to write software, and unit-tests hand in hand, because it is more fun, faster, and produces a higher quality product. I want to get to this world through my blog, the talks, and general knowledge sharing.

I just wanted to share a letter from my friend about the experience we had at Loopt. Together we can change the community! If you would like me to come and speak at your company about unit-testing and get people motivated and educated don’t hesitate to ask. It will be my pleasure.

Hi Misko,

I never got a chance to thank you for coming to Loopt. It has made a big difference here. People are excited to write unit tests and there are lots of discussions over tools, architecture, etc. That’s all I really wanted – to bring some more attention to the topic. As for me, then I’ve been using most of your techniques while writing code – however, I didn’t know the terminology and couldn’t really explain why I was doing things in certain ways – it was just the way it had to be in order to write unit tests :-) Now that I know the official terms its much easier to identify place where I drifted away - and more importantly, refactor other peoples code to make it testable. I’ve only ever tried to write unit tests to code you write from scratch, and never really had to write the tests after the code was written. At least now I know exactly why large parts of our code is not testable and what we can do to make it better.
So to answer your question from Facebook; My update was intentionally vague as I wanted a screenshot for the posting I just wrote to our blog: http://www.looptblog.com/2008/11/unit-test-tuesdays.html
Feel free to reference it :-)
Later, Heine

→ 2 CommentsTags: Referal

My Unified Theory of Bugs

posted on November 17th, 2008 ·

by Miško Hevery

I think of bugs as being classified into three fundamental kinds of bugs.

  • Logical: Logical bug is the most common and classical “bug.” This is your “if”s, “loop”s, and other logic in your code. It is by far the most common kind of bug in an application. (Think: it does the wrong thing)
  • Wiring: Wiring bug is when two different objects are miswired. For example wiring the first-name to the last-name field. It could also mean that the output of one object is not what the input of the next object expects. (Think: Data gets clobbered in process to where it is needed.)
  • Rendering: Rendering bug is when the output (typical some UI or a report) does not look right. The key here is that it takes a human to determine what “right” is. (Think: it “looks” wrong)

NOTE: A word of caution. Some developers think that since they are building UI everything is a rendering bug! A rendering bug would be that the button text overlaps with the button border. If you click the button and the wrong thing happens than it is either because you wired it wrong (wiring problem) or your logic is wrong (a logical bug). Rendering bugs are rare.

Typical Application Distribution (without Testability in Mind)

The first thing to notice about these three bug types is that the probability is not evenly distributed. Not only is the probability not even, but the cost of finding and fixing them is different. (I am sure you know this from experience). My experience from building web-apps tells me that the Logical bugs are by far the most common, followed by wiring and finally rendering bugs.

Cost of Finding the Bug

Logical bugs are notoriously hard to find. This is because they only show up when the right set of input conditions are present and finding that magical set of inputs or reproducing it tends to be hard. On the other hand wiring bugs are much easier to spot since the wiring of the application is mostly fixed. So if you made a wiring error, it will show up every time you execute that code, for the most part independent of input conditions. Finally, the rendering bugs are the easiest. You simply look at the page and quickly spot that something “looks” off.

Cost of Fixing the Bug

Our experience also tells us how hard it is to fix things. A logical bug is hard to fix, since you need to understand all of the code paths before you know what is wrong and can create a solution. Once the solution is created, it is really hard to be sure that we did not break the existing functionality. Wiring problems are much simpler, since they either manifest themselves with an exception or data in wrong location. Finally rendering bugs are easy since you “look” at the page and immediately know what went wrong and how to fix it. The reason it is easy to fix is that we design our application knowing that rendering will be something which will be constantly changing.

Logical Wiring Rendering
Probability of Occurrence High Medium Low
Difficulty of Discovering Difficult Easy Trivial
Cost of Fixing High Cost Medium Low

How does testability change the distribution?

It turns out that testable code has effect on the distribution of the bugs. Testable code needs:

The result of all of this is that the number of wiring bugs are significantly reduced. (So as a percentage we gain Logical Bugs. However total number of bugs is decreased.)

The interesting thing to notice is that you can get benefit from testable code without writing any tests. Testable code is better code! (When I hear people say that they sacrificed “good” code for testability, I know that they don’t really understand testable-code.)

We Like writing Unit-Tests

Unit-tests give you greatest bang for the buck. A unit test focuses on the most common bugs, hardest to track down and hardest to fix. And a unit-test forces you to write testable code which indirectly helps with wiring bugs. As a result when writing automated tests for your application we want to overwhelmingly focus on unit test. Unit-tests are tests which focus on the logic and focus on one class/method at a time.

  • Unit-tests focus on the logical bugs. Unit tests focus on your “if”s and “loop”s, a Focused unit-test does not directly check the wiring. (and certainly not rendering)
  • Unit-test are focused on a single CUT (class-under-test). This is important, since you want to make sure that unit-tests will not get in the way of future refactoring. Unit-tests should HELP refactoring not PREVENT refactorings. (Again, when I hear people say that tests prevent refactorings, I know that they have not understood what unit-tests are)
  • Unit-tests do not directly prove that wiring is OK. They do so only indirectly by forcing you to write more testable code.
  • Functional tests verify wiring, however there is a trade-off. You “may” have hard time refactoring if you have too many functional test OR, if you mix functional and logical tests.

Managing Your Bugs

I like to think of tests as bug management. (with the goal of bug free) Not all types of errors are equally likley, therefore I pick my battles of which tests I focus on. I find that I love unit-tests. But they need to be focused! Once a test starts testing a lot of classes in a single pass I may enjoy high coverage, but it is really hard to figure out what is going on when the test is red. It also may hinder refactorings. I tend to go very easy on Functional tests. A single test to prove that things are wired together is good enough to me.

I find that a lot of people claim that they write unit-tests, but upon closer inspection it is a mix of functional (wiring) and unit (logic) test.  This happens becuase people wirte tests after code, and therefore the code is not testable. Hard to test code tends to create mockeries. (A mockery is a test which has lots of mocks, and mocks returning other mocks in order to execute the desired code) The result of a mockery is that you prove little. Your test is too high level to assert anything of interest on method level. These tests are too intimate with implementation ( the intimace comes from too many mocked interactions) making any refactorings very painful.

→ 10 CommentsTags: Advice · Testability

Clean Code Talks – Dependency Injection

posted on November 11th, 2008 ·

by Miško Hevery

Google Tech Talks
November 6, 2008

ABSTRACT

Clean Code Talk Series
Topic: Don’t Look For Things!

Speaker: Miško Hevery

Video

Slides

→ 8 CommentsTags: Uncategorized

Clean Code Talks – Unit Testing

posted on November 4th, 2008 ·

by Miško Hevery

Google Tech Talks October, 30 2008 ABSTRACT Clean Code Talks – Unit Testing Speaker: Misko Hevery

Video

Slides

→ 8 CommentsTags: Uncategorized

Pass Around Ginormous Context Objects

posted on October 27th, 2008 ·

This question was submitted by one of the readers, and it shows a common questions people ask when they are coming up to speed when trying to write testable code….

About “Pass[ing] around ginormous context objects” (http://misko.hevery.com/2008/07/24/how-to-write-3v1l-untestable-code/) – this situation comes up for me when [I] trying to get configuration settings into my application. I want objects deeper in the system to have some configurable values which come from outside the system, and I don’t want everyone along the way to know about what those configurable values are. How do you deal with this case? I’m thinking about a collection of console utilities which take settings like number of retries for different actions, timeouts to allow, logging levels, and other things. Some settings will be of interest to many components and others only to one, and it seems like if I decompose them then whomever handles the construction will have to know a lot about what settings classes know about and don’t know about.

// I would hate the class constructing the
// FileCopier to know it is interested in
// all of these!
FileCopier copier = new FileCopier(logger, 
           copierSettings, generalTimeouts);

// I end up doing this:
FileCopier copier = new FileCopier(configuration);
// and inside..
FileCopier(Configuration configuration) {
  X = configuration.GetSetting(“X”, DefaultValue);
  …
}

Lets look into each of these in turn:

  1. “I would hate the class constructing the FileCopier to know it is interested in all of these!” Why is this a problem? as discussed in ‘Breaking the Law of Demeter is Like Looking for a Needle in the Haystack‘ When you ask for things explicitly it makes it easy to pass in test-doubles for these dependencies and hence it makes the code testable. Not only is the code testable it is easy to understand. If I am new to the code-base I look at the constructor and I see that logger, copierSettings and timeouts are things which I may be interested in. On the other hand passing in a Context object it is hard for me to know which parts from the Context object the class-under-test is interested in. It also makes testing harder. Now I have to create Mock Context and override all of the getLogger(), getCoppierSettings() and so on for these methods to return the real test-doubles of interest. But How do I know what needs to be mocked out? I need to read the code, I can’t just look at the constructor and know. But there is more: Suppose I want to reuse FileCopier class in a different project. In that case I need to reuse all of the compile-time dependant classes as well. In this case it is the Context. But context is a kitchen sink and it knows about everything in your application which means I have to resuse those objects as well. So the code with Context objects is not reusable/re-compilable.
  2. The place where our mental model diverged is in this sentence: “I don’t want everyone along the way to know about what those configurable values are.” The fear is that in order to call the constructor the caller will have to know about the parameters, and its caller will have to know and so on. We talked about this in ‘Dependency Injection Myth: Reference Passing‘. However, we are making one more mistake. We are mixing object creation (and look up) with application logic, see: ‘How to Think About the “new” Operator with Respect to Unit Testing‘. The basic mistake people make at this point is that they assume that the object construction graph is same as object instantiation graph. This assumption is mostly true only if you sprinkle new-operators with your application logic. If the new operator is removed from application logic than your application has two phases. Phase (1) wire the objects together in order to instantiate the application. In Phase (1) all we do is call the new operators and pass in the right dependencies. We do not call any methods! In phase (2) we execute the application, here we stay away from the new-operators and call methods so that the objects can collaborate. Once the application is constructed we no longer need to pass the configuration information around. We simply execute methods on FileCopier.
  3. Loging is a Singleton and we have discussed Singletons here, here and here. The root problem is that Singleton (the design pattern) is the global state and global state sucks, from testability, maintainability, and few other -bilities. So should we pass in the Logger through the constructor. Well that depends… If you want to assert that under some conditions a specific log message is produced than you have no choice but to pass it in. On the other hand, if you just want logging, and don’t want to test that it works correctly, than you can do Logger.getLogger(Class.class). This does not make Singletons OK, but it is benign-kind-of-evil. The reason is that with Logger the information only flows one way: Into the Logger. Your application does not read data from the Logger and more importantly does not behave any differently depending on if the Logger is turned-on or turned-off (It behaves different only in the sense that things don’t get printed, but not in the sense that your application now computes a different answer.) Most Singletons do not fall into this rare category where we push data only in, and hence most Singletons are a nightmare to test.

NOTE: when I say Singleton I mean the design anti-pattern with a global instance field as in private static Singleton instance = new Singleton(); This is what makes the Singleton a global. To be contrasted with singleton (lower case ‘s’) as in a single instance of something without a global instance field.

→ 11 CommentsTags: Uncategorized

Testability Explorer: Measuring Testability

posted on October 21st, 2008 ·

Testability Explorer: Using Byte-Code Analysis to Engineer Lasting Social Changes in an Organization’s Software Development Process. (Or How to Get Developers to Write Testable Code)

Presented at 2008 OOPSLA by Miško Hevery a Best Practices Coach @ Google

Abstract

Testability Explorer is an open-source tool that identifies hard-to-test Java code. Testability Explorer provides a repeatable objective metric of “testability.” This metric becomes a key component of engineering a social change within an organization of developers. The Testability Explorer report provides actionable information to developers which can be used as (1) measure of progress towards a goal and (2) a guide to refactoring towards a more testable code-base.
Keywords: unit-testing; testability; refactoring; byte-code analysis; social engineering.

1. Testability Explorer Overview

In order to unit-test a class, it is important that the class can be instantiated in isolation as part of a unit-test. The most common pitfalls of testing are (1) mixing object-graph instantiation with application-logic and (2) relying on global state. The Testability Explorer can point out both of these pitfalls.

1.1 Non-Mockable Cyclomatic Complexity

Cyclomatic complexity is a count of all possible paths through code-base. For example: a main method will have a large cyclomatic complexity since it is a sum of all of the conditionals in the application. To limit the size of the cyclomatic complexity in a test, a common practice is to replace the collaborators of class-under-test with mocks, stubs, or other test doubles.

Let’s define “non-mockable cyclomatic complexity” as what is left when the class-under-test has all of its accessible collaborators replaced with mocks. A code-base where the responsibility of object-creation and application-logic is separated (using Dependency Injection) will have high degree of accessible collaborators; as a result most of its collaborators will easily be replaceable with mocks, leaving only the cyclomatic complexity of the class-under-test behind.

In applications, where the class-under-test is responsible for instantiating its own collaborators, these collaborators will not be accessible to the test and as a result will not be replaceable for mocks. (There is no place to inject test doubles.) In such classes the cyclomatic complexity will be the sum of the class-under-test and its non-mockable collaborators.

The higher the non-mockable cyclomatic complexity the harder it will be to write a unit-test. Each non-mockable conditional translates to a single unit of cost on the Testability Explorer report. The cost of static class initialization and class construction is automatically included for each method, since a class needs to be instantiated before it can be exercised in a test.

1.2 Transitive Global-State

Good unit-tests can be run in parallel and in any order. To achieve this, the tests need to be well isolated. This implies that only the stimulus from the test has an effect on the code execution, in other words, there is no global-state.

Global-state has a transitive property. If a global variable refers to a class than all of the references of that class (and all of its references) are globally accessible as well. Each globally accessible variable, that is not final, results in a cost of ten units on the Testability Explorer.

2. Testability Explorer Report

A chain is only as strong as its weakest link. Therefore the cost of testing a class is equal to the cost of the class’ costliest method. In the same spirit the application’s overall testability is de-fined in terms of a few un-testable classes rather than a large number of testable ones. For this reason when computing the overall score of a project the un-testable classes are weighted heavier than the testable ones.

3. How to Interpret the Report

By default the classes are categorized into three categories: “Excellent” (green) for classes whose cost is below 50; “Good” (yellow) for classes whose cost is below 100; and “Needs work” (red) for all other classes. For convenience the data is presented as both a pie chart and histogram distribution and overall (weighted average) cost shown on a dial.

[-]ClassRepository [ 323 ]
  [-]ClassInfo getClass(String) [ 323 ]
    line 51:
      ClassInfo parseClass(InputStream) [318]
      InputStream inputStreamForClass(String) [2]
  [-]ClassInfo parseClass(InputStream) [318]
    line 77: void accept(ClassVisitor, int) [302]
    line 75: ClassReader(InputStream) [15]

Clicking on the class ClassRepository allows one to drill down into the classes to get more information. For example the above report shows that ClassRepository has a high cost of 318 due to the parseClass(InputStream) method. Looking in closer we see that the cost comes from line 77 and an invocation of the accept() method.

73:ClassInfo parseClass(InputStream is) {
74:  try {
75:    ClassReader reader = new ClassReader(is);
76:    ClassBuilder v = new ClassBuilder (this);
77:    reader.accept(v, 0);
78:    return visitor.getClassInfo();
79:  } catch (IOException e) {
80:    throw new RuntimeException(e);
81:  }
82:}

As you can see from the code the ClassReader can never be replaced for a mock and as a result the cyclomatic complexity of the accept method can not be avoided in a test — resulting in a high testability cost. (Injecting the ClassReader would solve this problem and make the class more test-able.)

4. Social Engineering

In order to produce a lasting change in the behavior of developers it helps to have a measurable number to answer where the project is and where it should be. Such information can provide in-sight into whether or not the project is getting closer or farther from its testability goal.

People respond to what is measured. Integrating the Testability Explorer with the project’s continuous build and publishing the report together with build artifacts communicate the importance of testability to the team. Publishing a graph of overall score over time allows the team to see changes on per check-in basis.

If Testability Explorer is used to identify the areas of code that need to be refactored, than compute the rate of improvement and project expected date of refactoring completion and create a sense of competition among the team members.

It is even possible to set up a unit test for Testability Explorer that will only allow the class whose testability cost is better than some predetermined cost.

→ 4 CommentsTags: Advice · OO · Testability · Testability Explorer

Dependency Injection Myth: Reference Passing

posted on October 21st, 2008 ·

by Miško Hevery

After reading the article on Singletons (the design anti-pattern) and how they are really global variables and dependency injection suggestion to simply pass in the reference to the singleton in a constructor (instead of looking them up in global state), many people incorrectly concluded that now they will have to pass the singleton all over the place. Let me demonstrate the myth with the following example.

Let’s say that you have a LoginPage which uses the UserRepository for authentication. The UserRepository in turn uses Database Singleton to get a hold of the global reference to the database connection, like this:

class UserRepository {
  private static final BY_USERNAME_SQL = "Select ...";

  User loadUser(String user) {
    Database db = Database.getInstance();
    return db.query(BY_USERNAME_SQL, user);
  }
}

class LoginPage {
  UserRepository repo = new UserRepository();

  login(String user, String pwd) {
    User user = repo.loadUser(user);
    if (user == null || user.checkPassword(pwd)) {
      throw new IllegalLoginException();
    }
  }
}

The first thought is that if you follow the advice of dependency injection you will have to pass in the Database into the LoginPage just so you can pass it to the UserRepository. The argument goes that this kind of coding will make the code hard to maintain, and understand. Let’s see what it would look like after we get rid of the global variable Singleton Database look up.

First, lets have a look at the UserRepository.

class UserRepository {
  private static final BY_USERNAME_SQL = "Select ...";
  private final Database db;

  UserRepository(Database db) {
    this.db = db;
  }

  User loadUser(String user) {
    return db.query(BY_USERNAME_SQL, user);
  }
}

Notice how the removal of Singleton global look up has cleaned up the code. This code is now easy to test since in a test we can instantiate a new UserRepository and pass in a fake database connection into the constructor. This improves testability. Before, we had no way to intercept the calls to the Database and hence could never test against a Database fake. Not only did we have no way of intercepting the calls to Database, we did not even know by looking at the API that Database is involved. (see Singletons are Pathological Lairs) I hope everyone would agree that this change of explicitly passing in a Database reference greatly improves the code.

Now lets look what happens to the LoginPage…

class LoginPage {
  UserRepository repo;

  LoginPage(Database db) {
    repo = new UserRepository(db);
  }

  login(String user, String pwd) {
    User user = repo.loadUser(user);
    if (user == null || user.checkPassword(pwd)) {
      throw new IllegalLoginException();
    }
  }
}

Since UserRepository can no longer do a global look-up to get a hold of the Database it musk ask for it in the constructor. Since LoginPage is doing the construction it now needs to ask for the Databse so that it can pass it to the constructor of the UserRepository. The myth we are describing here says that this makes code hard to understand and maintain. Guess what?! The myth is correct! The code as it stands now is hard to maintain and understand. Does that mean that dependency injection is wrong? NO! it means that you only did half of the work! In how to think about the new operator we go into the details why it is important to separate your business logic from the new operators. Notice how the LoginPage violates this. It calls a new on User Repository. The issue here is that LoginPage is breaking a the Law of Demeter. LoginPage is asking for the Database even though it itself has no need for the Database (This greatly hinders testability as explained here). You can tell since LoginPage does not invoke any method on the Database. This code, like the myth suggest, is bad! So how do we fix that?

We fix it by doing more Dependency Injection.

class LoginPage {
  UserRepository repo;

  LoginPage(UserRepository repo) {
    this.repo = repo;
  }

  login(String user, String pwd) {
    User user = repo.loadUser(user);
    if (user == null || user.checkPassword(pwd)) {
      throw new IllegalLoginException();
    }
  }
}

LoginPage needs UserRepository. So instead of trying to construct the UserRepository itself, it should simply ask for the UserRepository in the constructor. The fact that UserRepository needs a reference to Database is not a concern of the LoginPage. Neither is it a concern of LoginPage how to construct a UserRepository. Notice how this LoginPage is now cleaner and easier to test. To test we can simply instantiate a LoginPage and pass in a fake UserRepository with which we can emulate what happens on successful login as well as on unsuccessful login and or exceptions. It also nicely takes care of the concern of this myth. Notice that every object simply knows about the objects it directly interacts with. There is no passing of objects reference just to get them into the right location where they are needed. If you get yourself into this myth then all it means is that you have not fully applied dependency injection.

So here are the two rules of Dependency Injection:

  • Always ask for a reference! (don’t create, or look-up a reference in global space aka Singleton design anti-pattern)
  • If you are asking for something which you are not directly using, than you are violating a Law of Demeter. Which really means that you are either trying to create the object yourself or the parameter should be passed in through the constructor instead of through a method call. (We can go more into this in another blog post)

So where have all the new operators gone you ask? Well we have already answered that question here. And with that I hope we have put the myth to rest!

BTW, for those of you which are wondering why this is a common misconception the reason is that people incorrectly assume that the constructor dependency graph and the call graph are inherently identical (see this post). If you construct your objects in-line (as most developers do, see thinking about the new operator) then yes the two graphs are very similar. However, if you separate the object graph instantiation from the its execution, than the two graphs are independent. This independence is what allows us to inject the dependencies directly where they are needed without passing the reference through the intermediary collaborators.

→ 31 CommentsTags: Advice · OO · Testability

To “new” or not to “new”…

posted on September 30th, 2008 ·

by Miško Hevery

Dependency injection asks us to separate the new operators from the application logic. This separation forces your code to have factories which are responsible for wiring your application together. However, better than writing factories,  we want to use automatic dependency injection such as GUICE to do the wiring for us. But can DI really save us from all of the new operators?

Lets look at two extremes. Say you have a class MusicPlayer which needs to get a hold of AudioDevice. Here we want to use DI and ask for the AudioDevice in the constructor of the MusicPlayer. This will allow us to inject a test friendly AudioDevice which we can use to assert that correct sound is coming out of our MusicPlayer. If we were to use the new operator to instantiate the BuiltInSpeakerAudioDevice we would have hard time testing. So lets call objects such as AudioDevice or MusicPlayer “Injectables.” Injectables are objects which you will ask for in the constructors and expect the DI framework to supply.

Now, lets look at the other extreme. Suppose you have primitive “int” but you want to auto-box it into an “Integer” the simplest thing is to call new Integer(5) and we are done. But if DI is the new “new” why are we calling the new in-line? Will this hurt our testing? Turns out that DI frameworks can’t really give you the Integer you are looking for since it does not know which Integer you are referring to. This is a bit of a toy example so lets look at something more complex.

Lets say the user entered the email address into the log-in box and you need to call new Email(“a@b.com”). Is that OK, or should we ask for the Email in our constructor. Again, the DI framework has no way of supplying you with the Email since it first needs to get a hold of a String where the email is. And there are a lot of Strings to chose from. As you can see there are a lot of objects out there which DI framework will never be able to supply. Lets call these “Newables” since you will be forced to call new on them manually.

First, lets lay down some ground rules. An Injectable class can ask for other Injectables in its constructor. (Sometimes I refer to Injectables as Service Objects, but that term is overloaded.) Injectables tend to have interfaces since chances are we may have to replace them with an implementation friendly to testing. However, Injectable can never ask for a non-Injectable (Newable) in its constructor. This is because DI framework does not know how to produce a Newable. Here are some examples of classes I would expect to get from my DI framework: CreditCardProcessor, MusicPlayer, MailSender, OfflineQueue. Similarly Newables can ask for other Newables in their constructor, but not for Injectables (Sometimes I refer to Newables as Value Object, but again, the term is overloaded). Some examples of Newables are: Email, MailMessage, User, CreditCard, Song. If you keep this distinctions your code will be easy to test and work with. If you break this rule your code will be hard to test.

Lets look at an example of a MusicPlayer and a Song

class Song {
  Song(String name, byte[] content);
}
class MusicPlayer {
  @Injectable
  MusicPlayer(AudioDevice device);
  play(Song song);
}

Notice that Song only asks for objects which are Newables. This makes it very easy to construct a Song in a test. Music player is fully Injectable, and so is its argument the AudioDevice, therefore, it can be gotten from DI framework.

Now lets see what happens if the MusicPlayer breaks the rule and asks for Newable in its constructor.

class Song {
  String name;
  byte[] content;
  Song(String name, byte[] content);
}
class MusicPlayer {
  AudioDevice device;
  Song song;
  @Injectable
  MusicPlayer(AudioDevice device, Song song);
  play();
}

Here the Song is still Newable and it is easy to construct in your test or in your code. The MusicPlayer is the problem. If you ask DI framework for MusicPlayer it will fail, since the DI framework will not know which Song you are referring to. Most people new to DI frameworks rarely make this mistake since it is so easy to see: your code will not run.

Now lets see what happens if the Song breaks the rule and ask for Injectable in its constructor.

class MusicPlayer {
  AudioDevice device;
  @Injectable
  MusicPlayer(AudioDevice device);
}
class Song {
  String name;
  byte[] content;
  MusicPlayer palyer;
  Song(String name, byte[] content, MusicPlayer player);
  play();
}
class SongReader {
  MusicPlayer player
  @Injectable
  SongReader(MusicPlayer player) {
    this.player = player;
  }
  Song read(File file) {
    return new Song(file.getName(),
                    readBytes(file),
                    player);
  }
}

At first the world looks OK. But think about how the Songs will get created. Presumably the songs are stored on a disk and so we will need a SongReader. The SongReader will have to ask for MusicPlayer so that when it calls the new on a Song it can satisfy the dependencies of Song on MusicPlayer. See anything wrong here? Why in the world does SongReader need to know about the MusicPlayer. This is a violation of Law of Demeter. The SongReader does not need to know about MusicPlayer. You can tell since SongReader does not call any method on the MusicPlayer. It only knows about the MusicPlayer because the Song has violated the Newable/Injectable separation. The SongReader pays the price for a mistake in Song. Since the place where the mistake is made and where the pain is felt are not the same this mistake is very subtle and hard to diagnose. It also means that a lot of people make this mistake.

Now from the testing point of view this is a real pain. Suppose you have a SongWriter and you want to verify that it correctly serializes the Song to disk. Why do you have to create a MockMusicPlayer so that you can pass it into a Song so that you can pass it into the SongWritter. Why is MusicPlayer in the picture? Lets look at it from a different angle. Song is something you may want to serialize, and simplest way to do that is to use Java serialization. This will serialize not only the Song but also the MusicPlayer and the AudioDevice. Neither MusicPlayer nor the AudioDevice need to be serialized. As you can see a subtle change makes a whole lot of difference in the easy of testability.

As you can see the code is easiest to work with if we keep these two kinds objects distinct. If you mix them your code will be hard to test.  Newables are objects which are at the end of your application object graph. Newables may depend on other Newables as in CreditCard may depend on Address which may depend on a City but these things are leafs of the application graph. Since they are leafs, and they don’t talk to any external services (external services are Injectables) there is no need to mock them. Nothing behaves more like a String like than a String. Why would I mock User if I can just new User, Why mock any of these: Email, MailMessage, User, CreditCard, Song? Just call new and be done with it.

Now here is something very subtle. It is OK for Newable to know about Injectable. What is not OK is for the Newable to have a field reference to Injectable. In other words it is OK for Song to know about MusicPlayer. For example it is OK for an Injectable MusicPlayer to be passed in through the stack to a Newable Song. This is because the stack passing is independent of DI framework. As in this example:

class Song {
  Song(String name, byte[] content);
  boolean isPlayable(MusicPlayer player);
}

The problem becomes when the Song has a field reference to MusicPlayer. Field references are set through the constructor which will force a Law of Demeter violation for the caller and we will have hard time to test.

→ 24 CommentsTags: Advice · Testability

Application Wiring on Auto-Pilot

posted on September 24th, 2008 ·

by Miško Hevery

We talked about how it is important to separate the new operators from the application logic. This separation forces your code to have factories which are responsible for wiring your application together. By separating this responsibility the tests can always wire together a subset of an application with key components replaced for friendlies making testing easier and more focused. 

But lets look at a sample factory

class CarFactory {
  Car create() {
    return new Car(
          new EngineCompartment(
                new Engine(),
                new ManualTransmission(),
                new PowerSteering(),
                new Battery()
             ),
          new Cabin(
                new Door(new PowerWindow()),
                new Door(new PowerWindow()),
                new PowerSeat(), new PowerSeat()
            ),
         Arrays.asList(
              new Wheel(new Tire(), new Rim()),
              new Wheel(new Tire(), new Rim()),
              new Wheel(new Tire(), new Rim()),
              new Wheel(new Tire(), new Rim())
            )
       );
  }
}

This factory builds a car. The first thing to notice is that all of the new-operators are here (If you were to look inside each of the classes it would be devoid of “new”s). The second, is a complete lack of logic (no loops or conditions). And finally the third, your application behavior is controlled by the way the classes are wired together. If I wanted a automatic-transmission car, all we have to do is to wire the classes differently. The wiring responsibility is in the factory class not with the application logic.

However, why do we need to tell the JVM how to wire these Classes together? Is it not self obvious?  After all look at the constructors of these classes:

Car(EngineCompartment ec, Cabin c, List<Wheel> ws);
EngineCompartment(Engine e, Transmission t, 
              Steering s, Battery b);
Cabin(Door driverDoor, Door pasangerDoor, 
              Seat driverSear, Seat passangerSear);
Engine(float dissplacement, int pistonCount);
Battery(float voltage);
Door(Window window);
PowerWindow() implements Window;
PowerSeat() implements Seat;
Wheel(Tire tire, Rim rim);
...

Imagine you could just ask for things. Lets start simple and look at the Wheel. The constructor of Wheel needs a Tire and Rim. So when we ask for a Wheel it should be self obvious that we want new Wheel(new Tire(), new Rim()). Why do we need to make this explicit in our factory? Lets build a framework from which we can ask for a class and it returns an instance of that class. So in our case if we ask for getInstance(Wheel.class) it returns instance of new Wheel(new Tire(), new Rim()). Now a framework like this is easy to build since all we need to do is look at the constructor and recursively try to instantiate the objects until all recursive constructors are satisfied.

But things are a bit more complicated than that. What if we ask for Cabing as in getInstance(Cabin.class). Well Cabin needs two Doors and two Seats, but Seat is an interface and so we have to make a decision. What subclass of Seat should we instantiate. To help our framework make that decision we will add a bind method such as bind(Seat.class, PowerSeat.class). Great now we call get(Seat.class) the framework returns new PowerSeat(). Similarly, we will have to call bind(Window.class, PowerWindow.class). Now we can call get(Cabin.class) and the framework will return new Cabin(new Door(new PowerWindow()), new Door(new PowerWindow()), new PowerSeat(), new PowerSeat()).

Notice that a closer to the root a class, the class you ask for is, the more work will the framework do for us. So ideally we just want to ask for the root object. In our case Car. Calling get(Car.class) will cause the framework to do all of the work originally in our factory.

As you can see a framework which will call the new operators on your behalf is very useful. This is because you only have to ask for the root object (in our case the Car) and the framework will build the whole object graph on your behalf. This kinds of frameworks are called Automatic Dependency Injection frameworks and there are few of them our there. Namely GUICE, PicoContainer, and Spring

Since I know most about GUICE, The above example can be rewritten in GUICE like this:

class CarModule extends AbstractModule() {
  public void bind() {
    bind(Seat.class, PowerSeat.class);
    bind(Seat.class, PowerSeat.class);
    bind(Transmission.class, ManualTransmission.class);
    bind(new TypeLitteral<List<Wheel>>(){})
      .toProvider(new Provider<List<Wheel>>(){
        @Inject Provider<Wheel> wp;
        List<Wheel> get() {
          return Array.asList(wp.get(), wp.get(), 
                              wp.get(), wp.get());
        }
      });
  }
}
Injector injector;
injector = GUICE.createInjector(new CarModule());
Car car = injector.getInstance(Car.class);

As you can see Automatic Dependency Injection frameworks can do a lot of things for you. Namely, that you don’t have to worry about writing the factories. You simply write your application logic and ask for your dependencies in the constructor and let the framework resolve all of them for you. You move the responsibility of calling the new operator to the framework. Or to put it differently DI-framework is your new “new”. Now DI-frameworks can do lot of other things, which are beyond the scope of this article, such as manage object lifetimes, enforce singletons (in a good way, see: Root Cause of Singletons and Singletons are Pathological Liars) and manage different configurations of your application such as production vs development server.

For a more real life example of a GUICE Module see: CalculatorServerModule.java

→ 11 CommentsTags: Uncategorized