Among the various extra-curriculars that take up my downtime, baseball is up there.  Baseball was a passion for much of my youth.  As I got older and my attention turned towards my career and the world at large, the dawn of Michael Lewis’ “Moneyball” changed my view on baseball entirely.  Baseball became, for better or for worse – less interesting to watch for 180 days a year and more intriguing to capture in the snapshots of statistics and numbers that engulfed baseball in the early 2000′s.  Lewis was demonstrating not a new approach to how baseball was played, but how Oakland A’s manager Billy Beane was simply exploiting an efficiency that none of his contemporaries had (yet) uncovered.

So recently I found myself thinking about the role of the manager in baseball.  As other teams began looking at the Oakland model for constructing successful, cost-conscious rosters, many copied their formula.  Others began to look for other efficiencies.  But the manager role – the ultimate “chicken” contrasted to the players as pigs, has only had cursory evaluation for their affect on the outcome of the game.  These evaluations are subjective by and large, and have all concluded in one way or form that managers tend to have zero net effect on the success or failure of a team.  But baseball managers are perhaps one of the most agile practitioners today.  They must be responsive, they must have enough foresight to plan only enough in that they can mitigate risk, but not so much as to commit themselves to a shortsighted approach.  Their sprints are innings, their releases are games.  They create ample time to adjust and re-plan.  They are agile creatures – perhaps the original agile management style.

Every Game is a Release

Before every game, the manager has a plan to build.  The lineup is first and foremost.  It ultimately has very little affect on the outcome of the game, provided that the manager has planned for his best hitters to be part of the game.  Additionally and potentially irrelevant statistically – the manager will prioritize his lineup which will offer the most potential value for that game.  The catcher amy be different depending on his own starting pitcher.  Lefthanded hitters, statistically vulnerable to left-handed pitching may sit on the bench to start the game while statistically inferior players fill-in for the handicapped lefthander.  Some players may be removed from the roster entirely, optioned to a minor league affiliate to allow for superior options to be available in the case the need arises.

The analogy here, is release planning.  Just like the manager has an assortment of players, we have a backlog.  Each story offers value that may change from one sprint to the next.  Each story may shuffle in priority from one sprint to the next.  But the focus is always on ensuring that during a release we are planning to ensure that we are successful in every area of our sprints during the release while aiming to maximize the success of our teams over the entirety of the release.

Every Inning is a Sprint

Baseball is a strange game in that every inning, the defense controls the ball.  There are 9 innings in a game, and 3 outs per inning.  These are our resources.  As a manager, my job is to allow my team to execute while intervening only when resource mismanagement is certain.  I may pinch hit for a hitter in order to maximize run production while minimizing outs.  Conversely, I may change pitchers frequently to ensure my competition is least able to use their resources to their fullest.

Managing sprints is a task in managing your resources optimally.  Each sprint we start with our planning meeting where we identify what resources are available for the current sprint.  Then, based on various factors – we figure in what we can get done during the sprint.  How we get there may change, but the goal is always part of a theme.  In baseball, this is a touch of a stretch – but it’s hard to think that a manager going into the 9th down by 3 isn’t planning his decisions on maximizing on-base percentage that inning with the prospect of his best power hitter in the dugout.  Often times when a team hasn’t scored the tying or lead run but is threatening to score, the bullpen will warm up, a proactive response to something which may not be certain.  A lot of baseball management is about thinking forward just far enough to be prepared for what you know is next and not focusing too much on the possibilities in September while it’s still May.

Baseball Teams Are Self-Managing

This is more hypothetically true than true in reality.  It’s not unheard of for players to be managers as well.  But unlike professional American football where the level of specialization and differentiation between offensive and defensive roles is stark, or professional basketball where the players can succeed independently of the team, even if the team fails – baseball players are incredibly independent in their affect on any given contest.  While Michael Jordan and Kobe Bryant may have single handedly one NBA titles for their respective teams, Derek Jeter and Curt Schilling would have not won their rings were it not for the complimentary (but not cooperative) talents of the players that surround them.  They exhibit classic examples of game theory where the benefits of the strength of their performance are best realized by them only when realized by all.  That is a key trait of the self-organizing team.

What Baseball Can Tell Us About Software

While these two things are markedly different, baseball managers and players are clear examples of self-sufficient and practical agile applications.  Managers aim to minimize risk and maximize production while only planning for the factors that are at hand or probable rather than trying to account for any possible scenario.  This is not too indifferent from how a Scrum-Master must operate.  If baseball managers went into game 1 concerned about game 60, they would be accused of guesswork and incompetence.  Uncertainty would doom their plan, the pundits would not.  This is not at all indifferent from Agile software teams that learn from recent results and apply the lessons from the most recent experiences immediately.

Engineers write code.  QA tests that code.  In the past, this baton model has been derived from extensive functional specifications and use case documents that break down the core, extension, and exception cases so that both the tests and software can be clearly defined.  In a waterfall model, this was functional as it mirrored the sequence of SDLC.  But SCRUM and most Agile processes really shun this kind of thinking and instead target flat teams that collectively own the deliverable.  As in a recent experience of mine, maintaining that same handoff creates all sorts of problems.  Acceptance and rejection of delivered stories occurs late in the game, QA is always in a rush to execute their tests, and product teams and development see QA as a bottleneck.  It’s tempting for management to look at this and begin to turn SCRUM into a series of miniature waterfalls.  But this is wrong. Instead, as with all things in SCRUM, the search for a solution needs to answer a few questions to make the possible actions palatable:

1. Will the solution eliminate dependencies my team has?
2. Does this solution change the value that one set of skills adds to my team, or does it distribute it?
3. Will this solution be responsive to change?
4. Will this solution require extensive infrastructure to implement?

Heroes Kent Beck and Martin Fowler will tell you that the best solution is to strive for full, automated testing.  I couldn’t agree more.  Manual testing, as Fowler recently said, should be considered “a human rights violation.”  It’s error prone, difficult to repeat, and handles regression poorly.  That said, there are some battles simply not worth fighting.  In larger, more established teams – this type of shift is glacial, largely because of the risk incurred by such a change in paradigm.  Pragmatism dictates that the value of your QA team be repurposed to continue to add value, though potentially in a different way.

If we can concede the anecdotal thought that most engineers are substandard at QA, then we can see a glaring gap on a SCRUM team.  Even full automated testing with xUnits and CI is going to relinquish some level of code coverage, particularly in legacy software that wasn’t designed to be tested 10 years ago.  It should be mandated in teams in this type of pickle that all future code adopt the best practices of TDD – even to the extent that older, less TDD-friendly technologies and architectures be phased out.  But as we engage a group of engineers in a methodology in which they may never have been educated, hired, or trained for – on the sidelines we have a group of people who have been identifying failure and success scenarios their entire careers – even if they aren’t engineers.  A QA team can be indispensible in this situation.

In your new, SCRUM-based project – QA no longer are the gatekeepers.  We can’t have multiple gatekeepers in SCRUM.  Instead we focus on delivery and acceptance, creating one gatekeeper – a single keeper of the truth: the product owner.  That role is intended to be their exclusive domain.  They are the most intimate with the domain, the most intimate with the user story, and the most knowledgeable of where value lies.  To vest these judgmental by proxy in a serial part of the development cycle only leads to confusion.  Who is really assessing value?  Who is ultimately going to decide whether or not a story can be delivered?  Who should decide whether it should be accepted?  The demographics of a SCRUM project exist to solve this very question.  Eroding the clarity of role simply detracts value.

So what does this mean?  QA can’t be relegated to the role of testing system.  They cannot be an analogue for some sort of software framework.  Our goal should be to migrate to full automation, even if that takes time.  This means that the role of QA changes to an engineering role.  Not in the software sense, but in developing the cases which need to be tested.  These will arise from:

1. Conversational contributions to the story, noting and documenting constraints by guiding the product owner through different user experiences.  These can come out of user flows or non-functional prototypes.  But QA can add extensive value simply by contributing their ability to identify edge and corner cases that may not have been considered.  These may well not end up in our confirmation of the story, and it’s important for QA to recognize, just like with an engineer – that the value of a contribution in the conversation is to help shape the conversation, and not set requirements in the stead of the product owner.
2. Test cases derived from the constraints outlined in the conversation.  Take this user story:

   A user must be able to provide a credit card for payment of their order.  Each credit card must be validated with the issuer, and we will only accept credit cards from Visa and MasterCard.

   Try it with Discover card.
   Try it with American Express.
   Try it with an invalid billing address.

   This is the way that the product owner will ultimately accept the story.  The QA resource working on this story may write several test cases that test the edge and corner cases of the acceptance criteria, but they need to maintain an open conversation with the team to ensure they aren’t incidentally increasing scope by applying tests that go outside of the criteria.  So while the following test case is very likely to be something we’ll want to test:

   Try a billing address where the street name is provided in all capital letters but matches the address on file which is entirely in lower case.  This is expected to be a positive case, with the validation succeeding.

   This test would likely be considered an expansion of scope or otherwise invalid:

   Try charging the card and ensure that we put a $5 hold on the card until the transaction is complete.

So, in the cycle of the development of the story, the test cases serve as the outline for the engineer – as they should.  The engineer uses the tests to validate their code – or more effectively, runs their code against an automated test that implements the test case as defined by the QA specialist.  This helps improve quality and minimize risk by taking those decisions which we earlier agreed that engineers are poor at, out of their hands.  The test cases serve now as the same direction that the rest of the story does.  For those tests which remain manual for whatever reason, the effort to complete those tests can now be load balanced across the team, rather than vested in the hands of one point of failure in delivery.

Ultimately, this is what is so enjoyable about an Agile project.  Because it suggests solutions through it’s objectives, it’s possible to solve any execution concern by referring back to the principles that make Agile delivery successful.  Being able to respond to change, emphasize people and interactions, and prefer generalization over specialization is the guidance we can use to look at the role of the heroic QA team in a different light – a somewhat specialized engineering team that delivers artifacts that infuse delivery with the value that it may not be realizing in engineering, and not acting as a secondary source of truth for the team.

So understanding how a unit test is traceable back to a technical design on an agile team is really a key differentiation between an approach that more closely resembles cowboy coding and an approach which is relatively holistic.  No scrum master, project manager, or even architect would advocate full designs in an agile environment – instead we opt for lighter weight approaches to technical design that provide a blueprint to the code.  For many teams, this is a CRC card – a simply way of documenting the various dimensions of an object-oriented unit.

What is a CRC, and how does it relate back to tests?

Class

The core unit of code that we will refer to is the class, but this can be really any aggregation – a function library even.  This represents a loose collection of related functionality that leverages common resources, represents a concept of actions or identity, or extends existing functionality.

Responsibilities

The responsibilities of the class form it’s raison d’etre. Commonly, we can analogize responsibilities with methods, but they may be more coarse grained – and our refactoring may indeed refine the level of detail with which we understand the functional responsibilities of the class.

Collaborators

From an interestingness perspective, particularly in the context of unit testing and mocking (or stubbing), collaborators really take the cake.  When we talk about dependency injection as the core pattern of test-driven development with mocks, collaborators are indeed what we are injecting into our class to power it.

I always ask my green developers to think about these things before starting any sort of development.  Once their CRC is done and we agree on the approach, I ask them to build the programming interface, stub the implementation, and then start writing tests.  A common example I like to use when I get that befuddled look is the idea of a car.  A car has a lot of complex stuff going on that if I needed to test dependent functions of the car against, I wouldn’t want to have to concoct all of the function they provide.  For instance, the engine is a complex piece of machinery – but if I solely want to to test how my transmission reacts to engine behavior – it’s the transmission class I want to ultimately test, and it’s reactions to different engine behavior.

Let’s bring it up a level and talk about a Car as the core class.  We want our car to be able to do two things – to have two responsibilities: accelerate and stop.

public interface Car {
   int accelerate(int additionalVelocity);
 
   void setTireGrip(int gripFactor);
 
   int stop(int overDistance) throws CannotStopException;
}
 
public interface Motorable {
   int injectGasoline(int mlOfFuel);
}
 
public interface Stoppable {
   int applyBrakes() throws PadException, RotorException, WetRoadException;
}

So we’ve defined 3 interfaces. Let’s say that I’m really interested in a case where I test how the car reacts to different stopping conditions. Now, I’m demonstrating this in Java, so I’ll use syntax similar to Mockito – but the rules are the same: I’m interested in testing the requirements of the car observing the expected behavior of the brakes in those conditions. When the grip is set low, we expect the braking system to respond negatively and lock up.

We’ll also assume for now that I’ve stubbed out a skeletal implementation of the Car interface (meaning that everything throws a NotImplementedException for the time being).

@Test
public void testWetRoadConditionsWithLowFriction() {
 
   // arrange
   Stoppable s = mock(Stoppable.class);
   Motorable m = mock(Motorable.class);
 
   Car c = new CarImpl();
   c.setStopAbility(s);
   c.setMotor(m);
   c.setGripFactor(3);
 
   when(s.applyBrakes()).thenThrow(new WetRoadException());
 
   try {
      // act and assert - no real assertion, we're expecting an exception
      c.stop(100);
      fail("Should have seen a CannotStopException - there's not a lot of grip for road when road conditions are wet");
   } catch(CannotStopException e) {
      // do nothing, we're a success!
   }
}

As you can see, we’re not actually testing the brakes – we’re testing that under the given conditions that when the Stoppable interface throws an exception that prevents our car from stopping successfully that we are throw our own CannotStopException in response. The stoppable interface is a collaborator; we are not interested in this case in unit testing it, we are interested in asuming a known behavior from it, and testing how the stop() ability of the Car implementation reacts to the WetRoadException that we have stubbed.

The alternative in the past would have been to actually create the implementation of Stoppable. But having to unit test both items makes it difficult to isolate implementation issues.

So ultimately young developers, your unit tests need to test a single responsibility and not the responsibility of your collaborators. You may eventually test how the two implementations interact – that is an integration test. Ultimately though, there is more value in testing each individual unit and ensuring that the code is built to the spec of your lightweight design. Then you can isolate integration issues and write tests that specifically address finding integration problems.

This typically meant providing a real collaborator, like a PersistenceUnit in Java, so that I could test trivial operations like persisting an entity.  At the time, I challenged a manager on this:

What are we really testing?  Are we testing OUR requirement, or are we testing something we should reasonably expect Hibernate to do on the basis of our initial technology selection?

Code coverage I believe is pretty overrated.  No one cares whether or not you can prove your method to persist an entity works because the developers of Hibernate/TopLink/LINQ to SQL have already proved this through their unit tests. What you are building when you write a test that actually checks the database to see if your middleware used a pre-packaged object-relational manpping framework correctly is an integration test.  There’s certainly value to integration testing with data access layers, particularly when difficult to test components of your middleware (like stored procedures and triggers) are involved.  Saving that however, the focus of test-driven development – namely unit tests, should be on the business logic that is proprietary to your applicatiom, be it the front-end or the middle tier.

Now, suffice to say that to a good number of developers and architects this isn’t news, but if you really want to isolate the code which you wrote that is specific to your business rules and requirements – mocking frameworks are the way to go.  Particularly in an Agile project where we can readily identify certain injected dependencies (like an EntityManager, for example) as a collaborator of the class under test, your unit test is really interested in asking the question based on this interaction with this external component, how should I expect my logic to react?

In these scenarios of course, I mock or stub the EntityManager.  In the past, a fool like me would have tried to load a container within my unit test environment.  This is actually an integration test, in addition to the fact that it is costly and doesn’t add the value a truly isolated unit test which focuses on the code you wrote does.

In .Net, RhinoMocks is scalding hot with it’s supprt for mocking through lambda’s.  In JEE, I’ve fallen in love with Mockito.

So, stop worrying about how you can get JBoss running so that your CI environment can execute inside of it.  Instead, use mocks to give your code the resources it needs to vet out the conditions in your own code that you want to test.