Thursday, March 05, 2009

The reality of testing

This morning I watched an interesting you tube video on Django Success Stores.  If you aren't familiar Django is an open source web framework written in and for python.  As a side note AppEngine from Google also has implemented a version of Django for their systems.

In this video without exception every developer on the panel had various levels of apologies to offer for not testing their software.  Sometimes it seems that there is a whole religion devoted to testing, and just like other religions they have done a great job of making competent, productive developers feel guilty about not following their mantra.  Regardless of what that competent developer is working on or for who.

Don't get me wrong unit testing has its place - but there are situations where a rigorous, disciplined testing process is not appropriate.

If you are developing in an environment where you are not held accountable to your delivery dates ( I have had jobs like that ) or you are not delivering software versions frequently (for example if your production deploys happen less frequently than every other month).  Maybe dogmatic unit testing is for you.  However, if you are in a high stress work environment and you ship the code as fast as you can; then unit testing may not be the correct fit for your development process.  If you work in an environment where speed is the most important - and your business owners don't even know what they want but the want it delivered anyway - stop apologizing for not writing tests for every setter and getter in your code base!  I don't think I've ever heard someone who believed in Christianity apologize for not believing in Buddhism or vice versa.  Let's be done with the self recrimination, unit testing is not a silver bullet nor does if fit every software development scenario.

Having said that should we have great bodies of untested code running around?

Obviously not. I once worked with a testing zealot who loved to scoff at the lack tests in our applications.  It's not like these applications don't have any tests.  In fact some had quite a bit the difference was they tested functional paths.  And some of the only had tests that covered 5% of the code and you know what?  I'm ok with that.  Thanks to Java and Maven our applications have lots of libraries and some of these libraries are literally a bunch of simple POJO's basic setters and getters that are shared across the domain.  Should I write test cases to make sure that if you pass in a value you get one back?  No I should not, maybe a testing zealot should - but not me, that's not my religion.

I would suggest that if you are like me and unit testing is not your religion, then stop apologizing first of all!  Then read up on something called "Functional Testing."  Basically functional testing is taking a step back and looking at your application and understanding the functions that are performed - another way to say that is to test how your application is used (yeah we call those use cases).

This is how I love to test.  I not only enjoy writing these kind of tests but they are also critically important to understanding how the application really works.  And if you run into a problem in production if you wrote these kind of use case tests you should have the ability to recreate the scenario fairly easily.  That is where you get the bang for your buck.

I am making the decision write now - I am not apologizing anymore for not writing unit tests anymore.  My employer doesn't give me enough time to write the code without tests.  However I do write functional test that make EJB calls, query databases and do all kinds of things unit tests aren't supposed to do.  It's possible I may end up in a programmer's hell for writing functional tests instead of unit tests but I bet I'll have lots of great company.

What do you think?

-Aaron

No comments: