Real-word TDD

At the recent Entwicklertag conference in Karlsruhe, three colleagues and I gave a tutorial on 'TDD in the real world'. The basic idea behind the approach was that to establish TDD as a programming practice (in actual development projects), you need more than just to understand the TDD cycle and practice it on a few sandbox examples. You also have to make it into a habit, and you have to develop a skill. The exercises at the tutorial were designed to help participants make a start in this direction.

We have now made the materials from this tutorial available online; along with them, I have written a series of articles on the thinking behind our approach. You can find both the materials and the articles at http://andrena.github.com/reality-tdd/

Have fun!

Who gets the time — man, or machine?

I just stumbled over an interesting observation from an old time (about an even older era):

Mathematicians were so conscious of efficiency considerations that they could not imagine wasting any extra computer time for something a programmer could do by himself.[1]

Today, it's the other way round. The situation has changed, and so have the instincts of programmers. The primary task today (in mainstream programming) is to transform a domain requirement into a system's behavior, and that requires thinking on a higher level than algorithms and computations. Those things are assumed to be implemented already (both correctly and efficiently) in compilers and standard libraries. Smallish details of the code are taken over by tools: syntactic correctness is checked by compilers, type checks are done either statically or at runtime by compilers or runtime environments, semantic properties of code passages are verified by unit tests and code contracts.

What all these have in common is that they can be executed in milliseconds by a stupid machine. It's not necessary for the programmer to keep details about the execution flow, data vectors and so on in mind all the time, verifying them every so often silently in the head or empirically in the debugger. This is a waste of time, energy, and attention. Such details are specified and encoded once and then verified periodically in an automated manner. So the motto for programmers today is the exact converse of what Knuth and Trabb Pardo observed about the past: programmers nowadays are becoming once again conscious of efficiency considerations, and thus more hesitant to waste any brainpower for something a computer can do by itself.

If you think about it, that is how it should be. What Knuth describes is an anomaly, a unusual phenomenon. Historically, whenever people started to use some kind of technology, be it the wheel or the steam engine, the goal was to get rid of tasks and chores, not to take them on. Machines are tools, not gods. They shouldn't get the best of your time and abilities, but they're precisely there to take over everything else so that the higher powers of mind are freed from stupid, repetitive work and can be dedicated to pursuing higher-level goals — of which no machine has any understanding (at least, so far).

[1] From Donald E. Knuth and Luis Trabb Pardo, "The Early Development of Programming Languages", in: A History of Computing in the Twentieth Century. Ed. by J. Howlett, N. Metropolis, and Gian-Carlo Rota, New York: Academic Press 1980, 197–274. Didn't look it up myself, quoted from another article.

The olfactory quality of our language

Our language influences what we think, and our thoughts (or attitudes) sometimes guide our choice of words. And as with many things in life, some choices may look good in the beginning, even gain widespread application, but then become questionable on reflection.

One such choice is the term code smell, which typically refers to a code passage that cries out to be refactored, 'cleaned up'.

It's a very cleverly coined term. It associates certain patterns of bad coding (bad, because they make code less clear or more difficult to maintain, extend, or change) with a nasty experience that we all know. The analogy goes even deeper: bad smells in real life, once you have noticed them, are hard to ignore, and usually we have some impulse to do something about them, to eliminate the source of the stink. Likewise a 'code smell' is supposed to trigger action: reworking the code, cleaning up. Nowadays, every programmer knows what a code smell is; that is, everyone understands the notion, even if not necessarily everyone at all times recognizes typical smells when they encounter them — or at any rate, not necessarily all smells are in fact cleaned up. (At least that's what I often see in practice.) But everyone at least understands the idea of what a code smell is.

Cool, so what's the catch?

In a paper a few years ago, Kent Beck has written about "the engineer’s approach to change: find a problem, solve it"[1]. This way of going through the world and looking at things, however, "is biased negatively and it ignores whatever is going well". As Kent writes, this negativity will drain your emotional energy, if you make it into a general approach to thinking about what to do next. In effect, it will make it harder to achieve change. "Change is not aided by the dominant engineering problem/solution metaphor, where the world is seen in terms of problems to be solved."

Now, these insightful words were written about change, that is, about modifying the way we work, for instance when we start adopting the XP practices. What I want to suggest is that they equally apply to 'normal working mode'. When we are in the middle of everyday coding, we're also in danger of getting into a problem-response mode of acting. And when we're using words like 'smells' to describe to ourselves what we are doing, we cement this way of looking at things. (The effect is compounded when we use it in conversation, face-to-face or digitally, with other developers.)

When you are coding, what habitual attitude do you have towards the code base on which you are working? Do you think of it in terms of 'problems' (to be solved), or in a more strongly, colorful language, in terms of 'smells' (to be cleaned up)? Observe yourself during an average day of coding; make a list on a sheet of paper: how often do you change production code, and what exactly is it about that code which triggers action? How do you think and feel about it? The difference here may be subtle, but compare "Hey, these two passage look as if we can generalize and have a single, elegant query object!" with "Damn, there's this stinking code duplication again, I have to eliminate this!" It doesn't just feel bad when you recognize the pattern (in this case, a code duplication) in the code and instinctively classify it as 'smell'. There's also not much satisfaction to be gained after you have acted and changed the code. You've just changed a 'bad thing' to an 'ok thing'. Moreover, this kind of situation is likely to happen repeatedly, all over the day. Thus we establish for ourselves a feedback loop very effective in creating a general mood of frustration day after day.

So, we might have second thoughts about an outlook, sedimented in terminology, that encourages us to reduce stimulus-response patterns to problem-response patterns. (To be sure, even looking at things in terms of stimulus-response is a narrow mindset. But we're making it even more narrow by favoring a notion of triggers to action that is negative.) Such an outlook is very much suggested by speaking of 'smells'.

To widen our perspective: the notion of a 'code smell' is not the only notion that works this way (though it's probably one of the most ingrained). Generally, becoming more aware of the language we use when we think about what we do in our daily work pays off — even if it's just a small step that brings some fresh air and lets us look at a code base without a continuously crooked nose.

[1] http://www.threeriversinstitute.org/AppreciatingYourWayToXP.htm

Automatic test suites in Usus 0.6

For those of you who were at the Eclipse Testing Day, listening to the talk Marc and I gave about test suites in the Eclipse IDE: the new Usus feature we described is now released. It's part of Usus 0.6. If you already have Usus 0.6, you can install it separately from the Usus update site at http://projectusus.googlecode.com/svn/updates.

Here's what it does:

Just as you can simply right-click any Java project in the workspace and run all JUnit tests contained in it, you can now select any two or more projects and run all their tests as well. The only condition is that there is at least one project somewhere in the workspace that is a common dependency root of both (or all) selected projects, i.e. a project that depends, directly or indirectly, on both (or all of them). There is normally at least one such project, namely the one where you'd put your AllTests suite. Only of course with this new feature in Usus, you don't need an AllTests suite any more :-)

(A couple of remarks: If you select two projects where one of them depends on the other,  directly or indirectly, there is no need for an additional project that acts as common dependency root. The depending project itself can play this role.

A Java project in the workspace 'depends' on another project when it has that project on its Java Build Path, as a required project. Among other things, this includes plug-in projects that are declared as required plug-ins when you do Eclipse plug-in development. Thus, if you do plug-in development, you can use this feature on any two plug-in projects as long as they are in the same dependency tree.)

There's a second interesting new thing you can do with JUnit test suites in Usus 0.6. You can also select a single Java project, and choose Run As... All Tests Suite (JUnit). Usus will then collect all JUnit tests in that project and all its dependencies, i.e. walk the entire dependency tree
and create a test suite from all tests found.

Speaking at the Eclipse Testing Day

My colleage at andrena, and fellow blogger over at Crafting Code, Marc and I will give a talk at the Eclipse Testing Day in Darmstadt on Sep 8th. Here's the abstract:

In software projects with sizeable code bases maintaining the suite of all unit tests regularly becomes an issue: developers want to be able to execute the entire collection of tests with a single command, but an AllTests suite that lists the test cases explicitly must be kept manually, which is inefficient and error-prone.
The well-known ClasspathSuite solves this problem elegantly, but it has its weaknesses, too: it becomes slow on large workspaces and executes static code blocks in all classes when loading and analyzing them, which can cause unexpected effects and in some cases make the tests fail.
The Eclipse IDE allows to run all tests in a single Java project or source folder directly from the context menu, but doesn't extend this functionality to the entire workspace. (For good reasons, as we will explain in the talk.) There are approaches to mitigate this issue (e.g. this Plug-in), but they have their own shortcomings. Still, they clearly show the need for a solution in this area.
After discussing the current status of test suite management in Eclipse, we introduce a new approach to address these issues; we'll also demonstrate a prototypical implementation which we are currently developing at Project Usus.

 (If you want to join us at the Testing Day, here's the registration link :-)

How to code with a hedge trimmer

Something you do often in an Eclipse RCP application (because it's a client in some client-server-architecture) is to access a server and do something with a model object.

For instance, you may have a table view that shows a list of customers, represented by objects of type Customer. (Ok, I know it's a lame example.You're invited to use your imagination, if you'd like to see something more colorful ;-)

One of the tasks then is to load all customers from the server and put them as input on the table view. Thus your code would look something like this:

  CustomerService service = getCustomerService();
  List<Customer> customers = service.loadAllCustomers();
  customersTableViewer.setInput( customers );

Code growing out of proportion... 

Of course, it's never quite that easy. Normally, a service call is a long-running operation, so the least you can do is to show a busy indicator to the user. But that requires that you put your code in a Runnable:

  class LoadAllCustomers implements Runnable {
  
    private List<Customer> serverAccessResult;
  
    @Override
    public void run() {
      CustomerService service = getCustomerService();
      serverAccessResult = service.loadAllCustomers();
    }
    
    List<Customer> getServerAccessResult() {
      return serverAccessResult;
    }
  }

and then you can say:

  LoadAllCustomers op = new LoadAllCustomers();
  BusyIndicator.showWhile( op );
  customersTableViewer.setInput( op.getServerAccessResult() );

(Of course, you could do that in an anonymous implementation of Runnable, but bear with me for a moment, you'll see I've used a slightly more elaborate way for didactic reasons ;-)

Sometimes, however, you may expect the server call to run even longer, and therefore you want to shift it into the background entirely, so that the user can continue working until the server access results have arrived:

  class LoadAllCustomers extends Job {
  
    private List<Customer> serverAccessResult;
  
    LoadAllCustomers() {
      super( "Loading customer data" );
    }
  
    @Override
    public IStatus run( IProgressMonitor monitor ) {
      CustomerService service = getCustomerService();
      serverAccessResult = service.loadAllCustomers();
      return Status.OK_STATUS;
    }
    
    List<Customer> getServerAccessResult() {
      return serverAccessResult;
    }
  }

and then:

  Job job = new LoadAllCustomers();
  job.schedule();

... ah, just a second. What now? We could wait for the job to finish by calling join(), but what use would that be? That would destroy the point of putting something into a background job, wouldn't it? So we need a job-changed listener here:

  LoadAllCustomers op = new LoadAllCustomers();
  op.addJobChangeListener( new JobChangeAdapter() {
    @Override
    public void done( IJobChangeEvent event ) {
      LoadAllCustomers loaderJob = ( LoadAllCustomers )event.getJob();
      customersTableViewer.setInput( loaderJob.getServerAccessResult() );
    }
  } );
  op.schedule();

Looks good? No, still no joy: this listener will be called from a thread outside the UI thread, and this will cause an invalid thread access. So this is getting more complicated-looking by the minute:

  LoadAllCustomers op = new LoadAllCustomers();
  op.addJobChangeListener( new IJobChangeListener() {
    @Override
    public void done( IJobChangeEvent event ) {
      LoadAllCustomers loaderJob = ( LoadAllCustomers )event.getJob();
      Display.getDefault().asyncExec( new Runnable() {
        @Override
        public void run() {
          customersTableViewer.setInput( loaderJob.getServerAccessResult() );
        }
      } );
    }
  } );
  op.schedule();

(Of course, not to mention any error handling you also might want to do.)

To summarize: you basically have the same one or two lines of code, but you sometimes need them as a Runnable, sometimes you need them as a Job, and you may or may not have to put the follow-up operation in the UI thread.

... and cutting it back

Here's how I like to organize this a little.

First step: put the server access code for loading all customers and the follow-up code in their own classes.

  class LoadAllCustomers extends Job implements Runnable {
  
    private List<Customer> serverAccessResult;

    LoadAllCustomers() {
      super( "Loading customer data" );
    }
  
    @Override
    public IStatus run( IProgressMonitor monitor ) {
      run(); 
      return Status.OK_STATUS;
    }
  
    @Override
    public void run() {
      CustomerService service = getCustomerService();
      serverAccessResult = service.loadAllCustomers();
    }
    
    List getServerAccessResult() {
      return serverAccessResult;
    }
  }

  
  class SetInputAfterJobDone extends JobChangeAdapter {

    private final Viewer viewer;
    
    SetInputAfterJobDone( Viewer viewer ) {
      this.viewer = viewer;
    }
  
    @Override
    public void done( IJobChangeEvent event ) {
      LoadAllCustomers loaderJob = ( LoadAllCustomers )event.getJob();
      setInputInUIThread( loaderJob.getServerAccessResult() );
    }

    private void setInputInUIThread( Object input ) {
      Display.getDefault().asyncExec( new Runnable() {
        @Override
        public void run() {
          viewer.setInput( input );
        }
      } );
    }  
  }

(You can see now what I meant by 'didactic reason' a few lines above.)

You can now use your server access code both as Runnable or as Job:

  LoadAllCustomers op = new LoadAllCustomers();
  BusyIndicator.showWhile( op );
  customersTableViewer.setInput( op.getServerAccessResult() );

  LoadAllCustomers op = new LoadAllCustomers();
  op.addJobChangeListener( new SetInputAfterJobDone( customersTableViewer ) );
  op.schedule();

Second, I generally allow a follow-up behavior, if we're running as a job, by adding this method to LoadAllCustomers:

  public Job andThen( IJobChangeListener followUpBehavior ) {
    addJobChangeListener( followUpBehavior );
    return this;
  }

  So that I can write more compactly:

  new LoadAllCustomers().andThen( new SetInputAfterJobDone( customersTableViewer ) ).schedule();

Finally, pull up some of the common stuff in the server access code, so that you can re-use it for all other sorts of server accesses (like saving customers, querying customer details, and so on):

  public abstract class ServerAccess<T> extends Job implements Runnable {

    private T serverAccessResult;

    public ServerAccess( String name ) {
      super( name );
    }

    @Override
    protected IStatus run( IProgressMonitor monitor ) {
      run();
      return Status.OK_STATUS;
    }

    public T getServerAccessResult() {
      return serverAccessResult;
    }

    public ServerAccess<T> andThen( IJobChangeListener followUpBehavior ) {
      addJobChangeListener( followUpBehavior );
      return this;
    }

    protected void setServerAccessResult( T computationResult ) {
        this.serverAccessResult = computationResult;
    }
  }

Subclasses then merely have to implement the run() method with their own server access logic, and set the result. That should usually be a one-liner, and as you've seen, using these server access objects also normally is. (Correspondingly, generalize SetInputAfterJobDone so that it can cope with ServerAccess objects instead of simple Jobs).

So, here's one exercise left for the interested reader:

  new TryItInYourNext( project ).andThen( new GoAndHaveSome( coffee ) ).schedule();

Tell me how it worked :-)

Keep the bar green to keep the code clean II