Comments on: Cost of Testing

By: a response to – “What would you say is the average percentage of development time devoted to creating the unit test scripts?” | Down Home Country Coding With Scott Selikoff and Jeanne Boyarsky

Sun, 14 Feb 2010 18:11:31 +0000

[...] Misko Hevery comes up with a figure of 10% cost. He calls it a 10% tax and points out the benefits that come a tax. Note that he is writing tests as an integral part of his process and is fluent in doing so. He also actively dispels the myth that testing takes twice as long. [...]

By: Oleg

Oleg — Fri, 25 Dec 2009 16:17:41 +0000

Hi.

Well, 12 LOC/hour is very good! This is above industry average. Of course you should measure LOC of project / hours of project, so your hours include analysis, design, testing etc. etc. Under project I include even one release cycle with a couple of change requests.

We had a project, where we had like 40 LOC/hour during some unhealthy period, but it was cut down to half very quickly when we started actually testing our code )

Oleg

By: Tomasz Jedzierowski (venc)

Tomasz Jedzierowski (venc) — Tue, 22 Dec 2009 22:47:05 +0000

I agree that LOC is a bad metric when it comes to developer productivity. Just 12 LOC/hour is very low and I think that a lot of the time is spent on making the codebase testable by changing the design. This does not necessarily have a huge impact but in my opinion it is important to include such considerations in the calculations of the total cost of TDD. Quite often more testable code means better design therefor surelly not all the time spent on improving the design should be added to the overhead of TDD but it can easily be overdone. The developer which will have to maintain the project will thank you but the manager who pays the bills might not.

[OT] Changed to real name as I see that none commenter so far used a nickname. Great blog by the way

By: misko

misko — Tue, 22 Dec 2009 15:56:37 +0000

@venc,

excellent question. I consider myself a quick typist and fast at creating code, so the slowness cannot be attributed to it. Thinking back at how the code got created, there are times when I have produced a lot of code in short amount of time with tests. So writing tests is not what slows you down. The slow down comes from realizing that your current design is not the right one, and as a result you start refactoring. My experience is that everyone at some point realizes that their design is suboptimal, but most people say it is not worth the trouble and keep on coding more code. With tests you can step back and say, this is not right, and rearange/rewrite things and still be sure that you did not break anything. I think that you cannot look at LOC in the raw form. You need to look at Feature / LOC and that is hard to calculate. So the question should not be why i have only written 12 LOC per hour but why can I get high feature for little LOC.

By: venc

venc — Tue, 22 Dec 2009 15:08:15 +0000

Well…

14k LOC and 1.2k Hours thats about 12 lines of code per hour. What did You spend the rest of the time on? I suppose it was somewhat connected to the test process i.e. it had to do with makeing the design so that it is testable. At least part of the time spent on the design should be included in the time spent on test coverage.

By: matt harrison

matt harrison — Wed, 18 Nov 2009 17:23:12 +0000

This research paper suggests overhead of 15-35% for doing TDD
http://research.microsoft.com/en-us/projects/esm/nagappan_tdd.pdf

Great blog by the way!

By: Costs of Testing, Costs of Quality | Test And Try

Costs of Testing, Costs of Quality | Test And Try — Tue, 20 Oct 2009 09:34:44 +0000

[...] Cost of Testing by Miško Hevery Test Driven Development and the costs of writing code base tests. [...]

By: Mark’s Testblog » Blog Archive » Internals: To test, or not to test? - …for these are testing times, indeed.

Fri, 09 Oct 2009 01:01:12 +0000

[...] Hevery recently posted something interesting on his testing breakdown and, while most of us won’t reach his level of testing efficiency, [...]

By: Ionut G. Stan

Ionut G. Stan — Thu, 08 Oct 2009 00:47:10 +0000

Thanks for your response Misko.

That’s interesting. My first attempts with TDD were by writing tests for the whole class, which obviously didn’t work. I believe there are more people like me, so what I usually advise now is to tackle them method by method. The red, green, refactor thing was a kind of a revelation for me.

By: misko

misko — Wed, 07 Oct 2009 15:23:26 +0000

@Ionut,

I think everyone does TDD in the first way you described. We write a little test and than we implement it, we write another and than we implement that. I have not heard of anyone writing all of the tests at once.

I do sometimes write multiple test but they are on different level. I will write larger functional test and than write a smaller unit test.