Lately a lot of people have been arguing that dynamic languages (JavaScript, Python, Ruby) are inherently more testable then static languages such as Java. (See Java is Naturally Untestable, Comments) But in my opinion it is a bad idea and a misdirected argument.
- Mutable global state is bad
- Replacing methods at runtime is mutating global state
- It can be done in Java, but I don’t recommend it, it is a bad idea and it misses the point.
First, let me just state that which I hope all readers already know. “Mutating Global State” is a bad idea. Still, not convinced that global state leads to hard to test, maintain and understand code base? Check Wikipedia or search the web for “Global Variables Good” & “Global Variables Bad“. Here is the breakdown of web page hits returned by Google.
Dynamic languages allow you to change method definition at runtime. On one side this makes testing really easy since now I have a lot more seams in my application where I can intercept the calls. At first glance it looks like that this is a good idea. The problem is that code is part of your global state. In static languages the code is not mutable hence it is a constant and therefore not part of global state. In dynamic languages distinction between code and data is blurred and therefore it can be changed. Since code can be mutated and it is globally accessible it is part of the global state.
Here is a list of problems which you encounter when trying to test an application which has a lot of global state. Notice how all of these issues apply to testing code in a dynamic language where we did monkey-patching in order to intercept method calls:
- The order of tests matter (test may communicate through global state)
- Tests can not be run in parallel (global state interactions)
- We need a teardown method for cleanup (code smell of global state)
- New tests can break old tests (by changing global state)
- Tests know too much about the implementation (refactoring code breaks our tests)
So the argument goes like this. Dynamic languages are more testable because through mutable global state we have more seams than static languages. But this misses the point. Why is your code base written in such a bad way that you have to resort to such drastic measures like global state. Yes many design errors can be hacked through global state, but that does not make it a good idea!
In Java we too can replace method definitions at runtime. It is called a ClassLoader. You simply create a new ClassLoader and as you load your code you can replace a method call with another method call. There are even frameworks which make this easy known as Aspect Oriented Programming. But you don’t see anyone suggesting this in Java world as a good approach to testing.
Too me it sounds more like: Dynamic languages encourage bad application design since they provide so much rope that they make it impossible for you to hang yourself instead you drown in it.
Disclaimer: I am actually a big fan of dynamic languages. And this blog by no means should be read as Java is better than <chose you favorite dynamic language>. Instead read this as people who claim that in dynamic languages there is no need for dependency injection and good OO design really don’t understand how to write well designed and testable code. Yes, you can fight static with static, but it does not make it a good idea, its still static cling.
6 responses so far ↓
What do you think about replacing methods on an individual object?
This provides a useful extra seam that me and my colleagues use a lot during unit-testing, and it doesn’t affect global state. The object in question will be out of scope by the end of the test, and a new one constructed for each subsequent test.
@Jonathan,
Great point about instance method overriding, and you are right there is no global state. However, instance method overriding is equivalent to overriding methods in tests in java using anonymous inner classes. Most TDD frown upon that. The other issues is more complex. If you mix object creation with application logic than you can instantiate the object under test and you can override instance method on the object. But if the object-under-test instantiates other objects you don’t have a reference to those objects, and hence you can’t override the instance methods there.
The main point I want to get to is that you can’t leave testability as after the fact. Testability is something you have to bake into the design of your application. You have to think about it from day one. Now, it is true that there are more tricks you can use to tests your application in dynamic languages, but I feel that those tricks are all work around to the core problem of untestable design. But it differently, you can’t not think about testing in dynamic languages.
The other important point is that testing should be done by wiring objects in tests differently than in production. By replacing key objects with test-doubles your test graph of objects can be controlled. Dynamic languages allow you to change code at runtime, but now you are testing by modifying code rather than wiring the code differently. Since you are modifying code it is very easy to get lost as to what is real and what is replaced code. You no longer have clear separation of objects.
I think what is true is this: In dynamic languages you can write tests for badly designed code, something which is very hard to do in Static languages. However, the tests you write will be brittle and will rely on global state. Claiming that it is more testable is simply missing the point that testability is something which needs to be baked into the codebase.
I think the importance of this argument is in dealing with legacy codebases. With dynamic languages, you have a variety of ways by which you can force tests into a system that was never designed for them, and thus gain confidence to refactor; with static languages, it is very much harder to unravel a big untestable codebase.
Of course, if you design for testability from the start, then this point is obviously moot. But that’s not the world everyone lives in… or is thrown into.
The supporters of Lisp and its ilk maintain that code being data, and just as mutable as data is a *good* thing. I’ve programmed in a Lisp-like language with this property and found it to be liberating in some respects to. Dynamic languages like Python and Ruby are not as dynamic as Lisp in the code==data area, but approach the same flexibility.It would be a shame if, rather than try and explore what dynamism brings and generating best practices that suite dynamic development and testing, we instead ran scared and forced what seemed best for development in static languages on them.The software industry generates a lot of very expensive junk and most of that seems to be written in static languages. Maybe exposure to dynamic languahes can be a kick up the backside that leads to better results because if OO on Java is the best we have now, customers will stop paying.- Paddy.
I think the advantage in testability in “dynamic languages” comes from being able to use a REPL or interactive shell. This, combined with good design principles, either OO or functional and referential-transpancy, allows you to test what you have written right after you have written it. Not only does this encourage keeping your functional units small because larger units are harder to test, it’s just more satisfying to work with. You get the satisfaction of completing a function just as soon as you wrote it.
Your point about changing global methods is totally valid, and experienced users of dynamic languages often frown on too much of this practice. I learned this the ugly way when I inherited code from a JavaScript programmer who wrote tons of methods into Object.prototype (*gasp*). However, judicious use of prototypes in JS or things like open classes in Ruby can yield extremely positive results that make life much easier for everyone. I mean, c’mon, doesn’t every Java programmer in the world want a string method that matches against a regex already?
fcamel’s blog » 最近用 Python + TDD 心得（與 Java 做對照） // Jun 15, 2010 at 7:24 pm
[...] Python 並不會因為方便抽換既有物件而較好測試, 《Dynamic Languages are not Inherently More Testable》提到這樣當於更動 global object, glob… [...]
Leave a Comment