Everyone seems  to think that they are writing OO after all they are using OO languages such as Java, Python or Ruby. But if you exam the code it is often procedural in nature.

Static Methods

Static methods are procedural in nature and they have no place in OO world. I can already hear the screams, so let me explain why, but first we need to agree that global variables and state is evil. If you agree with previous statement than for a static method to do something interesting it needs to have some arguments, otherwise it will always return a constant. Call to a staticMethod() must always return the same thing, if there is no global state. (Time and random, has global state, so that does not count and object instantiation may have different instance but the object graph will be wired the same way.)

This means that for a static method to do something interesting it needs to have arguments. But in that case I will argue that the method simply belongs on one of its arguments. Example: Math.abs(-3) should really be -3.abs(). Now that does not imply that -3 needs to be object, only that the compiler needs to do the magic on my behalf, which BTW, Ruby got right. If you have multiple arguments you should choose the argument with which method interacts the most.

But most justifications for static methods argue that they are “utility methods”. Let’s say that you want to have toCamelCase() method to convert string “my_workspace” to “myWorkspace”. Most developers will solve this as StringUtil.toCamelCase(”my_workspace”). But, again, I am going to argue that the method simply belongs to the String class and should be “my_workspace”.toCamelCase(). But we can’t extend the String class in Java, so we are stuck, but in many other OO languages you can add methods to existing classes.

In the end I am sometimes (handful of times per year) forced to write static methods due to limitation of the language. But that is a rare event since static methods are death to testability. What I do find, is that in most projects static methods are rampant.

Instance Methods

So you got rid of all of your static methods but your codes still is procedural. OO says that code and data live together. So when one looks at code one can judge how OO it is without understanding what the code does, simply by looking at the relationship of data and code.

class Database {
  // some fields declared here
  boolean isDirty(Cache cache, Object obj) {
    for (Object cachedObj : cache.getObjects) {
      if (cachedObj.equals(obj))
        return false;
    }
    return true;
  }
}

The problem here is the method may as well be static! It is in the wrong place, and you can tell this because it does not interact with any of the data in the Database, instead it interacts with the data in cache which it fetches by calling the getObjects() method. My guess is that this method belongs to one of its arguments most likely Cache. If you move it to Cache you well notice that the Cache will no longer need the getObjects() method since the for loop can access the internal state of the Cache directly. Hey, we simplified the code (moved one method, deleted one method) and we have made Demeter happy.

The funny thing about the getter methods is that it usually means that the code where the data is processed is outside of the class which has the data. In other words the code and data are not together.

class Authenticator {
  Ldap ldap;
  Cookie login(User user) {
    if (user.isSuperUser()) {
      if ( ldap.auth(user.getUser(),
             user.getPassword()) )
        return new Cookie(user.getActingAsUser());
    } else (user.isAgent) {
        return new Cookie(user.getActingAsUser());
    } else {
      if ( ldap.auth(user.getUser(),
             user.getPassword()) )
        return new Cookie(user.getUser());
    }
    return null;
  }
}

Now I don’t know if this code is well written or not, but I do know that the login() method has a very high affinity to user. It interacts with the user a lot more than it interacts with its own state. Except it does not interact with user, it uses it as a dumb storage for data. Again, code lives with data is being violated. I believe that the method should be on the object with which it interacts the most, in this case on User. So lets have a look:

class User {
  String user;
  String password;
  boolean isAgent;
  boolean isSuperUser;
  String actingAsUser;

  Cookie login(Ldap ldap) {
    if (isSuperUser) {
      if ( ldap.auth(user, password) )
        return new Cookie(actingAsUser);
    } else (isAgent) {
        return new Cookie(actingAsUser);
    } else {
      if ( ldap.auth(user, password) )
        return new Cookie(user);
    }
    return null;
  }
}

Ok we are making progress, notice how the need for all of the getters has disappeared, (and in this simplified example the need for the Authenticator class disappears) but there is still something wrong. The ifs branch on internal state of the object. My guess is that this code-base is riddled with if (user.isSuperUser()). The issue is that if you add a new flag you have to remember to change all of the ifs which are dispersed all over the code-base. Whenever I see If or switch on a flag I can almost always know that polymorphism is in order.

class User {
  String user;
  String password;

  Cookie login(Ldap ldap) {
    if ( ldap.auth(user, password) )
      return new Cookie(user);
    return null;
  }
}

class SuperUser extends User {
  String actingAsUser;

  Cookie login(Ldap ldap) {
    if ( ldap.auth(user, password) )
      return new Cookie(actingAsUser);
    return null;
  }
}

class AgentUser extends User {
  String actingAsUser;

  Cookie login(Ldap ldap) {
    return new Cookie(actingAsUser);
  }
}

Now that we took advantage of polymorphism, each different kind of user knows how to log in and we can easily add new kind of user type to the system. Also notice how the user no longer has all of the flag fields which were controlling the ifs to give the user different behavior. The ifs and flags have disappeared.

Now this begs the question: should the User know about the Ldap? There are actually two questions in there. 1) should User have a field reference to Ldap? and 2) should User have compile time dependency on Ldap?

Should User have a field reference to Ldap? The answer is no, because you may want to serialize the user to database but you don’t want to serialize the Ldap. See here.

Should User have compile time dependency on Ldap? This is more complicated, but in general the answer depends on wether or not you are planning on reusing the User on a different project, since compile time dependencies are transitive in strongly typed languages. My experience is that everyone always writes code that one day they will reuse it, but that day never comes, and when it does, usually the code is entangled in other ways anyway, so code reuse after the fact just does not happen. (developing a library is different since code reuse is an explicit goal.) My point is that a lot of people pay the price of “what if” but never get any benefit out of it. Therefore don’t worry abut it and make the User depend on Ldap.

You hear people talking about small/medium/large/unit/integration/functional/scenario tests but do most of us really know what is meant by that? Here is how I think about tests.

Unit/Small

Lets start with unit test. The best definition I can find is that it is a test which runs super-fast (under 1 ms) and when it fails you don’t need debugger to figure out what is wrong. Now this has some implications. Under 1 ms means that your test cannot do any I/O. The reason this is important is that you want to run ALL (thousands) of your unit-tests every time you modify anything, preferably on each save. My patience is two seconds max. In two seconds I want to make sure that all of my unit tests ran and nothing broke. This is a great world to be in, since if tests go red you just hit Ctrl-Z few times to undo what you have done and try again. The immediate feedback is addictive. Not needing a debugger implies that the test is localized (hence the word unit, as in single class).

The purpose of the unit-test is to check the conditional logic in your code, your ‘ifs’ and ‘loops’. This is where the majority of your bugs come from (see theory of bugs). Which is why if you do no other testing, unit tests are the best bang for your buck! Unit tests, also make sure that you have testable code. If you have unit-testable code than all other testing levels will be testable as well.

A KeyedMultiStackTest.java is what I would consider great unit test example from Testability Explorer. Notice how each test tells a story. It is not testMethodA, testMethodB, etc, rather each test is a scenario. Notice how at the beginning the test are normal operations you would expect but as you get to the bottom of the file the test become little stranger. It is because those are weird corner cases which I have discovered later. Now the funny thing about KeyedMultiStack.java is that I had to rewrite this class three times. Since I could not get it to work under all of the test cases. One of the test was always failing, until I realized that my algorithm was fundamentally flawed. By this time I had most of the project working and this is a key class for byte-code analysis process. How would you feel about ripping out something so fundamental out of your system and rewriting it from scratch? It took me two days to rewrite it until all of my test passed again. After the rewrite the overall application still worked. This is where you have an AHa! moment, when you realize just how amazing unit-tests are.

Does each class need a unit test? A qualified no. Many classes get tested indirectly when testing something else. Usually simple value objects do not have tests of their own. But don’t confuse not having tests and not having test coverage. All classes/methods should have test coverage. If you TDD, than this is automatic.

Medium/Functional

So you proved that each class works individually, but how do you know that they work together? For this we need to wire related classes together just as they would be in production and exercise some basic execution paths through it. The question here we are trying to answer is not if the ‘ifs’ and ‘loops’ work, (we have already answered that,) but whether the interfaces between classes abide by their contracts. Great example of functional test is MetricComputerTest.java. Notice how the input of each test is an inner class in the test file and the output is ClassCost.java. To get the output several classes have to collaborate together to: parse byte-codes, analyze code paths, and compute costs until the final cost numbers are asserted.

Many of the classes are tested twice. Once directly throughout unit-test as described above, and once indirectly through the functional-tests. If you would remove the unit tests I would still have high confidence that the functional tests would catch most changes which would break things, but I would have no idea where to go to look for a fix, since the mistake can be in any class involved in the execution path. The no debugger needed rule is broken here. When a functional test fails, (and there are no unit tests failing) I am forced to take out my debugger. When I find the problem, I add a unit test retroactively to my unit test to 1) prove to myself that I understand the bug and 2) prevent this bug from happening again. The retroactive unit test is the reason why the unit tests at the end of KeyedMultiStackTest.java file are “strange” for a lack of a better world. They are things which I did not think of when i wrote the unit-test, but discovered when I wrote functional tests, and through lot of hours behind debugger track down to KeyedMultiStack.java class as the culprit.

Now computing metrics is just a small part of what testability explorer does, (it also does reports, and suggestions) but those are not tested in this functional test (there are other functional tests for that). You can think of functional-tests as a set of related classes which form a cohesive functional unit for the overall application. Here are some of the functional areas in testability explorer: java byte-code parsing, java source parsing, c++ parsing, cost analysis, 3 different kinds of reports, and suggestion engine. All of these have unique set of related classes which work together and need to be tested together, but for the most part are independent.

Large/End-to-End/Scenario

We have proved that: ‘ifs’ and ‘loops’ work; and that the contracts are compatible, what else can we test? There is still one class of mistake we can make. You can wire the whole thing wrong. For example, passing in null instead of report, not configuring the location of the jar file for parsing, and so on. These are not logical bugs, but wiring bugs. Luckily, wiring bugs have this nice property that they fail consistently and usually spectacularly with an exception. Here is an example of end-to-end test: TestabilityRunnerTest.java. Notice how these tests exercises the whole application, and do not assert much. What is there to assert? We have already proven that everything works, we just want to make sure that it is wired properly.

Which one are you? I am an engineer. But maybe we should first define the differences between the two. Engineer cares about how the system is put together whereas CS cares about how it works. Do you care about the technology or the algorithm?

The Interview

Since I am on engineer, these are the kind of questions I ask in an interview:

• Imagine you are an evil developer, how do you make code hard to test?
• How would you store a math expression tree, such that you would have an evaluate() method and a toString() method which would print the tree in an infix notation placing the parenthesis only when necessary.

On the other hand a CS asks these “algorithmic” questions:

• Write a merge sort method which merges ‘n’ files into one.
• Write a method to reverse a string in place.

The solution to engineer’s question is a set of classes interacting to define a system. Whereas a solution to a CS question is a single method doing something complex. Now, I don’t know about you but when was the last time when your application consisted of a single method. When was the last time when your system broke down because someone did not know how to write one of these algorithmic methods? When was the last time when your system broke down because the interaction between classes has gone too complex? I believe that most people are asking the wrong questions in the interview.

Algorithms

Now there is a time and place when an algorithm is exactly what the doctor ordered. Video/voice codecs, image compression, revision control, database optimization, and so on. But notice that all of these are nothing without some “application” around it. Video codec is nothing without YouTube around it, and DB engine is nothing without countless utilities, and database drivers which go with it. Chances are, when you are building an application, you will fail because  your class interaction will go out of hand, not because you did not know how to solve some algorithmic problem. Libraries have already been written for the most (80%) of the algorithmic problems out there. So why do you insist that an interview candidate knows how to sort integers. Is that what he/she will be doing all day long? While building your web application?

Engineer vs CS

The thing is you need both, and preferably in one person. It is also true that no one is 100% engineer or 100% CS, there is a blend in all of us. Algorithms require raw brain power, whereas engineering requires prior knowledge (best practices) which you cannot get by thinking about the problem harder. You can only get “engineering” knowledge by learning from others. I have seen too many people focusing too much on brain power alone. We hire young kids and we let them loose on a code-base, and than we wonder why we can’t change anything because everything is tightly coupled. Hint, it is not because they are stupid.

The Reality

The truth is in the ten years or so I have been out of school I have been fighting bad code every step of the way. I can’t remember when was the last time I had to solve an algorithmic problem. The most complex/algorithmic intensive code I have written was Testability Explorer, which is a byte code analysis engine which tries to determine how testable your code is. I have written it in my free time, and the thing which enabled me to write such a complex “algorithm” were unit tests, and lots of them! I had to rely on “best-engineering-practices” to build something which is “algorithm intensive”. In other words it is my engineering background not my CS which made it possible. I think CS, is required but insufficient.

Going Forward

I would love for the industry to reflect on itself and realize that its problems is not lack of CS majors, or a the lack of bright people, but a lack of good old fashion engineers, thetn maybe we may start asking a lot more questions, to which the answer is not a single method, but a collection of classes interacting together. It took a handful of scientist to come up with an atom bomb, but it took countless of engineers to work out everything else, and software is no different. You need a handful of CS, and an army of engineers.

Take your average developer and ask “do you know language/technology X?” None of us will feel any shame in admitting that we do not know X. After all there are so many languages, frameworks and technologies, how could you know them all? But what if X is writing testable code? Somehow we have trouble answering the question “do you know how to write tests?” Everyone says yes, whether or not  we actually know it. It is as if there is some shame in admitting that you don’t know how to write tests.

Now I am not suggesting that people knowingly lie here, it is just that they think there is nothing to it. We think: I know how to write code, I think my code is pretty good, therefore my code is testable!

I personally think that we would do a lot better if we would recognize testability as a skill in its own right. And as such skills are not innate and take years of practice to develop. We could than treat it as any other skill and freely admit that we don’t know it. We could than do something about it. We could offer classes, or other materials to grow our developers, but instead we treat it like breathing. We think that any developer can write testable code.

It took me two years of writing tests first, where I had as much tests as production code, before I started to understand what is the difference between testable and hard to test code. Ask yourself, how long have you been writing tests? What percentage of the code you write is tests?

Here is a question which you can ask to prove my point: “How do you write hard to test code?” I like to ask this question in interviews and most of the time I get silence. Sometimes I get people to say, make things private. Well if visibility is your only problem, I have a RegExp for you which will solve all of your problems. The truth is a lot more complicated, the code is hard to test doe to its structure, not doe to its naming conventions  or visibility. Do you know the answer?

We all start at the same place. When I first heard about testing I immediately thought about writing a framework which will pretend to be a user so that I can put the app through its paces. It is only natural to thing this way. This kind of tests are called end-to-end-tests (or scenario or large tests), and they should be the last kind of tests which you write not the first thing you think of. End-to-end-tests are great for locating wiring bugs but are pretty bad at locating logical bugs. And most of your mistakes are in logical bugs, those are the hard ones to find. I find it a bit amusing that to fight buggy code we write even more complex framework which will pretends to be the user, so now we have even more code to test.

Everyone is in search of some magic test framework, technology, the know-how, which will solve the testing woes. Well I have news for you: there is no such thing. The secret in tests is in writing testable code, not in knowing some magic on testing side. And it certainly is not in some company which will sell you some test automation framework. Let me make this super clear: The secret in testing is in writing testable-code! You need to go after your developers not your test-organization.

Now lets think about this. Most organizations have developers which write code and than a test organization to test it. So let me make sure I understand. There is a group of people which write untestable code and a group which desperately tries to put tests around the untestable code. (Oh and test-group is not allowed to change the production code.) The developers are where the mistakes are made, and testers are the ones who feel the pain. Do you think that the developers have any incentive to change their behavior if they don’t feel the pain of their mistakes? Can the test-organization be effective if they can’t change the production code?

It is so easy to hide behind a “framework” which needs to be built/bought and things will be better. But the root cause is the untestable code, and until we learn to admit that we don’t know how to write testable code, nothing is going to change…

Reader asks:

have a question regarding testing of Active Record style domain models

The problem is: client code of these objects will instantiate them directly, modify some attributes and then invoke a save(), or insert() method directly on the object. Writing a unit test that doesnt talk to the database is difficult since the save/insert methods are inherited from a super class which handles the persistence using an object responsible for database access.

As I see it, the problem is that Active Record stlye domains cross the boundary between what you desribed as Newable/Injectable. The “model” part is newable, but the data access part is injectable (ideally).

The challenge I’ve had testing is finding a suitable point at which to mock (or stub) the database access. I thought of replacing the new CustomerModel() (for example) with a call to a factory that would be provided using dependency injection, such as factory.createCustomerModel() but this feels like a bit of hack, and indeed you’ve warned against this directly in some of your posts.

The other option would be to define something static on the model super class to allow the default database object to be replaced before the test runs the client code. Again though, this doesnt “feel” that right to me.

What would you suggest?

I am assuming that you have read: http://misko.hevery.com/2009/05/05/the-problem-with-active-record/. The issue here is that the Record should be newable (http://misko.hevery.com/2008/09/30/to-new-or-not-to-new/), and as such it should have no reference to service objects. In other words the save method needs to move some place else. A good place for it is to move to is a repository. I really like the Hibernate model as it makes the repository a “watcher” which watches your newable objects and than saves them automatically when the transaction is up.

Here is an example:

CarRepository carRepo = ... inject me...;
Car car = new Car();
car.setAbc(...);
carRepo.save(car);

No problems here. In your test just inject an InMemoryCarRepository. The cool thing is that most of your business code should only deal with car. Only very few places need to know about CarRepository. Here is something else which is cool.

CarRepository carRepo = ... inject me...;
Car car = carRepo.getById(123);

enter business logic...
car.setAbc(...);
exit business logic...

carRepo.commit();

Since CaRepository knows about all of the Cars it has given out calling carRepo.commit() automatically saves all of the cars. Since most of your app modifies the entities most of it does not even need to know about CarRepository. Only classes which need to create or delete a car need to have a reference to a CarRepository.

From testing point of view this is great since most of my unit-test will be testing code from “enter business logic…” until “exit business logic…”, which means most of my tests will not even deal with CarRepository but only have to deal with a Car.

People often ask how can I justify two people when one will do. Will this not, just double my cost? To answer this question I think it is important to discuss what pair-programing is not.

How much of your time do you spend writing code? If you answer 100% than you are lying! Between checking your emails, going to meetings, talking to people, mentoring others, finding bugs and thinking about fixing them, coding is something we do surprisingly little of. I only do coding about 10% of time, but i am probably at an extrema, so lets say you code 25% of time. Even when you are coding, you spend a lot of time testing what you have just coded, and realizing what you have just done does not work. Your end product may be 100 lines of code, but it may take you weeks to get there.

Out of 25% of coding how much do I pair? Not enough, but lets say 25% of the time. Overall you are only talking about little over 10% of my overall time I am pairing which you can say that I only cost 10% more (if you can apply accounting logic to coding). But what does this extra 10% of “cost” buy me?

At most places I have been at when a new developer shows up, they hand him a computer and say, “OK, get up-to speed and build this feature X.” Then most places do not expect anything out of the developer for few months while he “gets-up-to-speed.” Few months? I expect my developers to check in a feature the first week! But you can’t just demand it you have to approach it differently.

Let me tell you about Arthur, the last developer which joined us for a month, and how productive he was. Arthur showed up on Monday at 9am. I sat down with him behind his computer and said, “Let’s get you up-to-speed.” I made sure that he was driving, and I was talking and we started installing all of the software which he needed to develop code. The compilers, the database the IDE, checking out the code from the repository and running it. The key here is that Arthur was doing all of the work and I was just telling him what to do. By noon, he was up and running, thanks to a very intensive hand-holding from me. We went to lunch and talked about the application, not in general terms but in specific terms which Arthur could translate to what he did few minutes ego.

After lunch, I said let’s go and fix a bug. I picked a simple bug and for the most part again i was doing most of the talking but made sure that Arthur had the keyboard and understood what, and why he was doing things. By 3 pm we had a fix. We ran all of the tests, and submitted the code. We watched the build to make sure we did not break anything.

This is pairing, and it is exhausting! By 3 pm I was beat and went home, but Arthur stayed to play some more with the code. Turns out he fixed another bug on his own before he went home. Not bad for first day. Now I could have fixed the bugs myself in 1 hour flat both of them, but then Arthur would not learn anything. I sacrificed my productivity to make Arthur productive in a single day. If I did not it would take Arthur weeks before he would figure out how to set everything up how things worked and enough courage to fix a bug. Yet that is exactly what most companies do. Think about the confidence Arthur had on day one working with us. He was up and ready and he fixed two bugs on day one. WOW!

More importantly is what we did not do. We did not get distracted by checking email. How can you, you have a single monitor, and that means that Arthur would read my email. Arthur did not waste time looking for documentation which is poor or does not exist, he was spoon fed the knowledge in the most efficient way, and he did it all by himself, i was just talking, he was typing. It is amazing how much more focused you are when you have two people working on something.

Next day, we paired some more, but this time we started to build a new feature. Again I was taking Arthur through the implementation making sure that he did all of the work. By the end of the week Arthur has implemented several features and the amount of pairing we did has dropped of significantly, but instead Arthur started pairing with other engineers which had the know how which he needed to get the features done.

At the simplest level pairing is helping people one-on-one, not through emails, let me show you behind a keyboard, instead of telling you in abstract. The amount of pairing and keeping each other on track depends on many things, but you will find that more you pair the more you will learn from each other and the better the code becomes.

But Arthur was a new engineer coming up-to speed, what about existing engineers? Well let me tell you story of Pepper the senior tech lead on a project. Pepper came to me and said, Misko, I don’t like the way we deal with X in our app, what could we do to make it better. Now here is a case where Pepper has all of the know how of his app, and I know next to nothing. So we sat behind the code and I started writing what I would imagine an ideal app would look like, knowing very little about the code. Pepper had to explain why my view was wrong or too simplistic and in the process we would learn from each other and slowly converged on an ideal design, which was neither his nor my idea wholly but a collaboration of our ideas. We started refactoring, sometimes I would drive and sometimes he would. We would take turns painting ourselves in the corner and then backing out. Many times Pepper would work without me, as the refactoring was tedious, but straight forward and two of us were not needed. All together it took us 2 weeks to finish this refactoring. Pepper did the lions share of the work, but we spent about 3 full days behind a single computer together. That is what is pairing.

If you would ask Pepper, he would tell you that pairing helped him focus. He would say that without the pair he would not have had many of the ideas which we ended up having. Most importantly there are now two people which really understand the details of the design. When I or Pepper will pair with someone else, we will share these new ideas with the new developers. As team pairs with each other ideas and knowledge multiply and flow making sure that the code-base remains consistent no matter which place you look. It is as if a single person has written it. This is a great thing which helps with maintenance.

Pairing does not mean that two people always do everything together and checking each others work, that would be boring and a waste of time. Pairing means that people bonce ideas from each other and have discussions, not in abstract, but behind a keyboard, keep each other focused and on track. Result of which is code which is better than the sum of its parts. Pairing is about sharing the knowledge with your team mates so that everyone is on the same page. We will go further if we make sure that all of us pull in the same direction, something which is rare in the industry at large.

by Miško Hevery & Jeremie Lenfant-engelmann

Did you notice that there are a lot of JavaScript testing frameworks out there? Why has the JavaScript community not consolidated on a single JavaScript framework the way Java has on JUnit. My feeling is that all of these frameworks are good at something but none solve the complete package. Here is what I want out of JavaScript unit-test framework:

I want to edit my JavaScript code in my favorite IDE, and when I hit  Ctrl-S, I want all of the tests to execute across all browsers and return the results immediately.

I don’t know of any JavaScript framework out there which will let me do what I want. In order to achieve my goal I need a JavaScript framework with these three features:

Command Line Control

Most JavaScript test runners consist of JavaScript application which runs completely in the browser. What this means in practice is that I have to go to the browser and refresh the page to get my results. But browsers need an HTML file to display, which means that I have to write HTML file which loads all of my production code and my tests before I can run my tests. Now since browsers are sandboxes, the JavaScript tests runner can only report the result of the test run inside the browser window for human consumption only. This implies that 1) I cannot trigger running of the tests by hitting Ctrl-S in my IDE, I have to Alt-tab to Browser, hit refresh and Alt-tab back to the IDE and 2) I cannot display my test result in my IDE, the results are in the browser in human readable form only.

On my continuous build machine I need to be able to run the same tests and somehow get the failures out of the browser and on to the status page. Most JavaScript test runners have a very poor story here,  which makes integrating them into a continuous build very difficult.

What we need, is the ability to control test execution from command line so that I can trigger it from my IDE, or my continuous build machine. And I need test failures to be reported on the command line (not inside the browser where they are unreachable) so that I can display them in IDE or in continuous build status page.

Parallel Execution

Since most JavaScript test runners run fully in the browser I can only run my test on one browser at a time during my development process. In practice this means that you don’t find out about failures in other browser until you have checked in the code to your continuous build machine (if you were able to set it up) and your code executes on all browsers. By that point you have completely forgotten about what you have written and debugging becomes a pain. When I run my test I want to run them on all browser platforms in parallel.

Instant Feedback in IDE

After I hit Ctrl-S on my IDE, my patience for test results is about two seconds before I start to get annoyed. What this means in practice is that you can not wait until the browser launches and runs the tests. The browser needs to be already running. Hitting refresh on your browser manually is very expensive since the browser needs to reload all of the JavaScript code an re-parse it. If you have one HTML file for each TestCase and you have hundred of these TestCases, The browser may be busy for several minutes until it reloads and re-parses the same exact production code once for each TestCase. There is no way you can fit that into the patience of average developer after hitting Ctrl-S.

Introducing JsTestDriver

Jeremie Lenfant-engelmann and I have set out to build a JavaScript test runner which solves exactly these issues so that Ctrl-S causes all of my JavaScript tests to execute in under a second on all browsers. Here is how Jeremie has made this seemingly impossible dream a reality. On startup JsTestDriver captures any number of browsers from any number of platforms and turns them into slaves. As slave the browser has your production code loaded along with all of your test code. As you edit your code and hit Ctrl-S the JsTestDriver reloads only the files which you have modified into the captured browsers slaves, this greatly reduces the amount of network traffic and amount of JavaScript re-parsing which the browser has to do and therefore greatly improves the test execution time. The JsTestDriver than runs all of your test in parallel on all captured browsers. Because JavaScript APIs are non-blocking it is almost impossible for your tests to run slow, since there is nothing to block on, no network traffic and no re-parsing of the JavaScript code. As a result JsTestDriver can easily run hundreds of TestCases per second. Once the tests execute the results are sent over the network to the command which executed the tests either on the  command line ready to be show in you IDE or in your continuous build.

Demo

A reader asks:

My boss knows about TDD but won’t allow us to use it because he thinks that it is just a passing hype that everyone talks about but no serious, big projects actually use it on daily basis. How could I convince him that it is not so?

This is an excellent question that takes me a bit by surprise, as I can’t imagine my boss dictating how I write code. I don’t have an answer, but here is what I think I would do if I were in this situation. (I would also like to encourage other readers to share their thoughts on this topic and offer suggestions in the comments.)

Arguments don’t work

There is no magical argument I can make to convince someone that TDD is a good idea: logic may not be persuasive here, but experience is. I am not arguing here that 1 + 1 is 2, but rather that if you follow these practices, the result is subjectively better. Since there is so much subjectivity in this, people need to experience TDD to understand its benefits. It is very hard to learn TDD by someone trying to learn it from a book. Learning TDD is more like learning to ride a bike: You can read books on how to do it and you can have silly arguments with theorists on how you can’t ride a two-wheeled contraption because it is inherently unstable, but nothing makes you a believer more than actually riding a bike. The first time you ride a bike it is a very clumsy experience, and I would argue that it takes hours of falling and uncertainty until you are better at it than walking: don’t expect TDD to be any different. It is a skill that takes hours of practice to master. Don’t argue with someone over your experience; instead, make him live through the same experience.

Make sure you know what you are talking about

To me, nothing is worse than someone trying to sell something that he doesn’t fully understand. Make sure you are reasonably good at TDD and have built a reasonably sized application before you try to show someone else how to do it. Imagine someone who has only seen others riding bikes trying to teach someone to ride a bike: my guess is that he will only frustrate the person learning. For this reason, build a small app with TDD in your free time. Your first tests will be the hardest, and at times you will think “this is way more trouble than it is worth,” but keep in mind that you struggling because you are learning a new skill. You have been developing code the old way for years: What makes you think that you can pick up TDD overnight? TDD makes you think about tests first, something that you have never done. It will not come easy, but in time it will become second nature to you. It took about one year before I felt that writing code test-first was the natural flow of things.

Create a small project to show your boss how helpful tests are in refactoring

I cannot overstate the importance of learning TDD on a brand new project without deadlines. Start something new in your free time and build it with TDD. Use it as a learning experience. You will: discover all kinds of things about how to best write code, get a feel for telling apart helpful tests from those that get in the way, learn how to think about the problem from a test-first perspective, realize the importance of not getting ahead of your tests, and discover that your code looks nothing like what you envisioned. Code with tests looks different. Try to refactor code written test-first and you will see that some tests are helpful while others get in the way. You first have to live through this experience to become familiar with the kinds of tests that get in the way of refactoring, and to know how to turn them into the kinds of tests that help you refactor. To know both one and the other you have to first build something with TDD.

Since you will be struggling a lot at the beginning, trying to bring TDD to an existing project may only fuel your boss’s feeling that TDD is not useful. Again, TDD is a skill and takes a while to develop.

Tests help design

We all believe that good code is reusable code. For this reason most of us add all kinds of interfaces and abstraction in the name of reusability. But what percent of the code you have written have you actually reused? I would say a tiny fraction, very close to zero. One reason for this is that your code isn’t reusable, even though you believe it to be. Tests help ensure that your code is in fact reusable, because they force you to write classes that can be used both in production and in a test scenario. If you can’t write a test, you don’t have reusable code. If you write test-first, then it is very likely that the code is reusable: after all, you just used it in tests and in production. This is the single reason why test-first produces better design. On the other hand, if you write code as you always have, you will most likely end up with hard to reuse code. If you write tests afterward, then at best the tests are an afterthought hacked on top of the production code; such tests are very unlikely to help us when you are refactoring, since they are likely to produce false negatives.

Hardware folks have figured this out

Did you know that average chip can have as much as 50% of its silicon dedicated to testing? Why is it that every discipline out there takes testing seriously except software? I believe it is because of the presumed inexpensive nature of fixing bugs. A new rev of silicon is extremely expensive. As a result, most companies can only afford to rev it about 5 times. Can you imagine writing code such that you are only allowed to run the production 5 times? Yet hardware folks can’t imagine designing chips any other way. The cost of another software rev seems close to zero, and this is where people get into trouble: We believe that another build costs us (almost) nothing, and this belief becomes our death by a thousand cuts. The more complicated the project, the more likely it is that a fix in one location will cause a bug elsewhere: the project is forced into an endless cycle of regressions. Tests help here, but again, this needs to be experienced.

If you get stuck, spike

Sometimes you just have no idea how to write code test-first; there are a lot of reasons for this. Instead, you revert to good old fashion hacking. That isn’t a problem as long as, after you first code by hacking, you go back and write tests, or preferably, completely re-write the code test-first. you call this “spike”; I do this quite a lot when faced with new technology or environment. Just don’t let this become the way you develop software.

Vote with your feet

If your boss is micro-managing how you do things and you have tried everything to explain your position, you always have the right to vote with your feet. Last time I checked, we software engineers are in high demand, so there is no reason to work for people who are being counterproductive.

What do you think?

What do the readers think about how they could influence others to develop software test-first? Please leave your comments so that we all can learn from each other

When I develop code I write test first and always run my tests. But it can get rather tedious to run the tests manually every so often. Here is a common scenario. Your tests are green and you start doing whole bunch of refactorings which you think are trivial and safe. When you are done you run the tests and it is broken. The problem is that you did ten little things and you don’t know which of the ten things you did broke the code. The solution is to run the test more often, but we just forget.

What I want is to have my IDE run my tests every time I do a change without me doing anything. This way even though I am doing ten little changes the IDE runs the tests ten times. The moment you do something which you thing is safe but it turns out that it makes your tests fail your IDE should tell you. You than simply hit Cntl-Z (undo) and you are back in business.

What I want is for my IDE to run my tests every time I save the code. To do this my tests need to be fast, because my patience after hitting Cntl-S is about two seconds. Anything longer than that and I will get annoyed. If you start running your tests after every save from test zero you will automatically make sure that your test will never become slow, since as soon as your tests start to run slow you will be forced to refactor your tests to make them faster.

Obviously this technique is only useful for true unit-tests since they are fast. I do not run my scenario tests as part of Cntl-S.

I only know how to set this up on Eclipse, so perhaps people with IDE or NetBeans can chime in. I have already set up the project for you here so all you need to do is download and try it out. But here are the steps

1. Create a Project: Eclipse has this cool feature which compiles your code continuously on every save. This is great, as what it really means is that your code is always compiled and you never have to wait for  a compilation step in eclipse.
2. Create a JUnit Text runner as shown here: here
3. Create an ANT build file which will run your text runner from the eclipse compiled folder. This way the ant does not have to continuously compile your code.
4. Tell Eclipse to run your ant target after every compilation cycle. This is known as Builder in Eclipse.
   1. Open project properties: Right click project -> Properties
   2. Create new ant builder step after the Java Builder in eclipse: Builders -> new -> Ant Builder
   3. Give your builder a nice name. Here is what to do on each tab. (I thing eclipse has a bug and you need to hit Apply every time you change tabs)
      • Main: Select the build file; base directory (This specifies which ant filet to use)
      • Targets: Select your ANT target for: “After Clean”, “Manual Build”, and “Auto Build” (most important) (This specifies that after ever build it should also run your ANT target.)
      • Build Options: Check “Specify working set” and select all of your source folders. (This specifies which file changes will trigger this builder, in our case any file change)

Or just grab my repository form here and import the already made project 00_AutoTest and try it out.

Happy test running…