...And How Are You Going to USE That Information?

I cited in my last post a case where Counting Wasted Time. I ended by giving my favorite response when someone is asking to collect data about the project:

...And how will you use that information?

Don't get me wrong. I really want a great answer to that question. I seek metrics that help understand the project. To paraphrase the NRA:

Metrics don't kill projects; people do.

But I digress. Some additional examples of useless information gathering will help you see the importance of the question, "And how will you use that information?"

Example 1: Revised Estimates

In a recent fixed price project, we dutifully re-estimated stories at the start of an iteration. We had release level estimates with which we did our original planning. The iteration estimates were supposed to set our "commitment" level for the iteration; they were supposed to set client expectations.

We over-delivered by 100%. We repeated the exercise, and over-delivered again. So on the third cycle, we asked the question, "How did we use the estimates we made?" The team, including the project manager, acknowledged that we hadn't looked at them since we collected the numbers. So we dropped them.

(BTW as I cited in yesterday's post about Counting, I have rarely found use in resizing just in case things have changed.)

Example 2: Defect Tracking Database

We haven't tracked bugs in the last three development projects I have participated in. How can this be?

One of the key Lean practices is feedback; feedback is most effective when rapidly given. This leads to a positive metric of story cycle time, a measure of the time from the start of analysis until story acceptance by the customer. So if a bug blocks a story, then the clock is running on that story until it is fixed and moves forward. Developers on a Lean-oriented team will attack bugs rather then work on development of new stories. The net results is few bugs live for more than a day. In tracking a recent project, over half the days were finished with zero open bugs.

Instead, we post the bug's existence on a red task card on our standard card wall. When it is fixed and verified, we take the card down. I would like to say that we throw the card away, but we still like to put them into a stack and admire them (for some reason). I do have to admit to a certain curiosity myself. But in the last couple of projects, I have yet to see anyone looking back at those cards!

There are great bug tracking tools out in the market. Many IT shops have standardized on one, and have process guidelines requiring their use. An Agile/Lean project shouldn't need any of them.

[In a bit of cruel fate for a vendor, I was recently asked to give a statement on a recently announced bug tracking tool that purports to solving all your tracking needs; indeed, it is the next generation of tracking tools, rendering all other obsolete. Needless to say, I commented that it was a solution in search of a problem as far as I was concerned. If you need that level of sophistication, your project is going to fail anyway.]

Bad Reasons to Collect Information

There are two reasons for collecting metrics that are, in fact, good reasons not to collect metrics:

Because I might need them: I consider this a euphemism for "I don't know." I lump into this same category, "We always do it this way" and "We have to". In the case of "I don't know", we drop the counting for an iteration or so, and see if we miss it. It is always hard to argue against a short-term experiment. In the case of "we always do it this way", start the discussion to drill down and figure out why we did it that way originally, and see if the reason still exists (especially if you have introduced Agile and/or Lean practices). This is just good development practice, but in particular is a good Lean practice. In the case of "we have to", it is time to engage the process architects and gauge if this is a battle worth fighting. I tend to relish that idea having done the process role in IBM many, many years ago, but we won't discuss my personality defects here. Regardless, the tactic is the same: Find out what it was useful for and see if the need still exists.

I need to understand how well everyone is doing. Translation: "I need to pin the blame on someone when things go wrong." Now you have hit a hot-button of mine. Blame is not a way to run a team. Blame creates fear, and fear kills motivation and productivity. And kills fun! Agile teams, especially collocated Agile teams, have many positive ways to address inhibitors to productivity. This includes many ways of addressing skill deficiencies which will always exist in software projects because of the wicked pace of change. So if someone attempts to collect information for the primary purpose of assigning blame, well, let's just say they have to get past me first.

There Are Good Metrics

Less I have led you astray (and you will use these posts to justify collecting no information), there are many good uses of collected information. I will cite my favorite Agile/Lean project metrics in a future post.

But for now, do you have examples of gathering useless information?

Counting Wastes Time

It's there. You see it. You have this overwhelming urge to count it. It's a moral obligation, a call-to-arms; your personal mission.

I am referring to counting artifacts and measuring artifacts. I am talking about project managers and team leads, and sometimes programmers, testers, and analysts.

And if you stop to think about it, it is a big time waster. A better core philosophy:

"Never count your money while you're sitting at the table. There'll be time enough for counting when the dealings done." -- Kenny Rogers from The Gambler

This thinking is based in Lean, a process borrowed from manufacturing. One of the practices Lean espouses is the identification and elimination of waste. Waste is broadly defined as anything that does not add value to the delivered product.

Agile software development produces loads of artifacts, like stories, tasks, test cases, and code. All of these artifacts have different useful lives. Code endures. Tasks are transient artifacts of development. There is a time and place for counting each. But if you count outside of those boundaries: waste!

Here is a real life example from an Agile project. It occurred between the client project administrator (sort of a cross between a administrator and a project manager) and myself in my role as tech lead and process coach. The discussion was around task cards. A task is a small unit of development work identified by the programmer for a story, which itself has an average of 10 or so tasks, with a huge variance.

• PA: "I need to track task cards."
• Me: "There are going to be a lot of them."
• PA: "I still want to track them."
• Me: "Could you just track stories? Development only takes a couple of days."
• PA: "I still want to track them."
• Me (seeing a pattern here): "Well, if you really want to..."
• PA: "Great. We need to number them."
• Me (I am beginning to catch on): "So that you can track them?"
• PA: "Yes."
• Me: "You are free to put any number on them you want. They're on the wall." (Note the technique of delegating to the person who seems to have all the extra time.)
• PA: "And I need the initials of the pair that worked on them."
• Me (Can't see a quick way around that.) "Okay. We mark the finished as we go. We'll put our initials on it then."
• PA: "I need an estimate for each task."
• Me: "Two-to-four hours."
• PA: "No, I need an estimate for each task."
• Me: "Two-to-four hours. We break the story down into 2-4 hour tasks. That is the estimate."
• [Repeat several more times.]
  
• PA: "Okay, I will mark a range of 2-4 hours. Then I will need the actuals."
• Me: "No." I really try not to use this word with clients, but sometimes there isn't a better word.
  
• PA: "But how can I track how long it actually took?"
• Me (trying to use powerful Vulcan logic): "If we gave you a number, it is only an estimate of the actuals. We have to estimate to accommodate meetings, distractions, bathroom breaks, and the like. Even we don't know how long it actually took."
• PA (after long processing pause): "But I need actuals."
• Me (feeling a little guilty suggesting it): "You can track how long it takes to do the story, and just divide it."
• PA: "Okay, I guess that will work."

I was being a bit unfair to our project administrator. The whole conversation would be rendered moot as soon as she saw the number of tasks that were created, started, and completed each day (about 20-25 on an average day). The counting only lasted a week.

Most consultants would (and do) succumb to the pressure to count. Some may even advocate it. Frankly, I exploited my gray hair and used my authority voice to carry the day in this case.

Imagine the overhead if this process had been established? Even a few minutes per task add up when there are 3000 tasks on a project. And the killer question for this:

...And how will you use that information?

How I Discovered Lean

Before I delve more into Lean, the best management practice to support Agile, I would like to step back and talk about how I got to Lean. The path I took influences my thinking about the Agile/Lean integration. If you want to skip this post, that is fine. It adds little to the overall discussion.

I have been writing code a long time. I was around when waterfall was introduced as a management process. Two things happened to me in the late 80's:

• I had a chance to study Just-In-Time (JIT) when getting my Masters degree, taught by a leading authority, and
• A colleague from IBM Research gave me a copy of Smalltalk.

I found both topics fascinating, but did not connect them at the time.

The Lean (JIT) Path

Lean is the new name for JIT, a more appropriate name as JIT moved from manufacturing into other sectors. In my graduate studies, I actually wrote a paper for applying JIT to software testing. The thesis was a focus on waste.

When doing testing under waterfall at the time (20 years ago), specialized testers take the code base and run their predefined manual scripts. As a bug was found, it was documented extensively, then the tests were continued. Often, another bug was found quickly. It was documented as well, and tests were again continued. And so on.

As a recipient of these bugs (ie, the programmer that injected the bug in the first place), the first bug was usually real. The second bug, however, was often the same bug just showing up again. As I analyzed the bugs, it would get more and more difficult to discern whether a bug was really a new bug or not. I just wanted to throw those later reports away. But the process dictated that I close the bug as "unable to reproduce" or "duplicate" to complete the bug cycle.

Armed with JIT and its idea of waste, I saw the situation in a different light. The tester was creating work for me by testing after finding the first couple of bugs. It was unlikely these bugs were new. Regardless, I would probably ask the tester to reconfirm the bug after fixes to the earlier bugs. This was all waste!

Rather, the tester should stop testing after finding a couple of bugs. (At the time, I suggested 3 might be the magic number. Now I lean more toward 1 is enough.) My professor loved the idea; the first time I tried it with real testers, they were horrified! The testers were measured on completely running the tests (whether there were bugs or not). They get credit for finding bugs. Did I want them to just sit around and do nothing? Surely, it was more productive for them to at least try the tests.

But it wasn't more productive for the team for them to try more tests. They were better off just sitting around (or better yet, test something else). After a month or so of agony, they began to understand the message, dumped their original metrics, and starting becoming more effective testers.

The OO Path

Although I had early exposure to Smalltalk, my current job didn't let me do anything except play around with it. I do remember loving the introductory books and the cool tools. But when it came time to actually solve a problem with it, I was absolutely clueless.

When I went out on my own as an independent, I finally began to understand the paradigm. For me, the trick was coding GUI interfaces. In most UI frameworks, the system calls the code when something needs to be done. The application is not in control -- the user is! Once I got this reversal, the message-passing metaphor of Smalltalk began to click. I eventually wrote commercial Smalltalk, found a mentor who was part of Ward Cunningham's original team, and began to really learn Object-Oriented Programming. This was still, however, a long 5-6 year process.

Then Came Agile...

I was just starting an OO consulting gig when I attended OOPSLA '98. At a birds-of-a-feather session, Kent Beck, Ron Jeffries, Ward Cunningham, Martin Fowler and others presented eXtreme Programming (XP). There was no book, just a one page handout of the 9 practices and a diagram of how they related to each other.

Having been raised in the Ward Cunningham School of Objects, it instantly clicked. My client agreed to these ideas, and we embarked on an XP project.

I realized quickly that we were in a production process. And Lean was a perfect fit for code production under Agile. We started focusing on waste and cycle times. We immediately started running cycles faster than the stock XP suggested, simply because we could.

I have exclusively done Agile projects since then (1998). I continue to apply more and more Lean management principles as I go.

A Different Path to Agile-Lean

Mary and Tom Poppendieck lecture and write about Lean being applied to Software Development. They describe it very well in their two books. Mary got to the Agile-Lean by a completely different route.

What is marvelous is that we (and many others) have gotten to the same point. Lean is the appropriate management model for Agile.

Lean - An Agile Management Process

In Three Critical Enablers for Agile, the second critical enabler is use of a management process consistent with the Agile engineering practices that continually produce software. For this, I use Lean, a manufacturing process borne in Toyota and responsible (IMO) for the current dual dominance of Japanese companies in both automotive production and quality.

Think Production, Not Building

I cannot over-emphasize how radical project management is when shifting to Lean. Of all the Agile roles, the role of project manager changes most in Agile. It dwarfs the impact Agile has on the development, testing, and analysis roles. But to gain the benefits Agile promises, changing the thinking around project management is critical.

Agile focuses on continually producing working software. Traditional phased software development (aka waterfall) focuses on moving the project steadily forward one phase at a time, doing its best to ensure everything is right before building upon that phase in the next.

Traditional project management is based on traditional engineering; civil engineering is my favorite analogy. Software is like building a bridge. You need to know what is needed (a bridge). You need architecture (suspension bridge). You need design (number and spacing of struts). You need to build it in a specific order (base before pillars; pillars before road surface). You need to test it (weight bearing tests, plus inspections). Waterfall software processes follow this model. The prevalent tool for traditional project manager is the Gantt chart, a schedule of activities and dependencies that can track progress.

Before disparaging waterfall too much, it is worth saying waterfall works. In my years at IBM, I participated in many waterfall projects that delivered quality software on time and on budget. Indeed, I have been a part of a thousand programmer project spanning three years and delivering. So don't count me among those who say waterfall is bad. I remember the true chaos before waterfall.

But as I believe Agile is a better process than waterfall (and I have tons of experience with each), I believe that Lean is a better management process for Agile than foisting traditional project management onto an Agile project.

Lean management practices focus on production; this factor alone makes it a good fit for the constant production of software characteristic of Agile. But Lean management differs significantly:

• No useful information can be captured in Gantt charts
• It is not important what someone will be working on next week
• Individual assignments are subsumed by team efforts

Basically for the traditionally trained project manager, it is bizarro-land. Round is square; down is up; when means why; and who means when.

But all is not lost. Lean brings powerful new tools to the project manager, tools that give insight into the project success throughout the project. At this point, I feel completely naked on a project without these tools. They are my new security blanket. These Lean processes and tools:

• Track production, giving you new insight into the real progress
• Show skill imbalances with clear direction on actions
• Remove much guesswork out of the question, "Are we going to ship on time?"
• Provide superior techniques for productive feedback to individuals
• Support dynamic adjustments to schedule, content, and resource

Sound too good to be true? Consider that we will be stealing most of these techniques from manufacturing management practices that are at least as old as the engineering practices we use today. We are not inventing new things here, but rather borrowing from a different source, manufacturing instead of civil engineering.

No Management Alternative

I have seen many (and heard of many more) Agile projects that recognize the uselessness of most of the traditional project management practices. There solution: abandon the practices altogether, but substitute nothing else. Everyone is a professional, after all, and they will be doing their best.

For a short time, this seems to work well. But for a longer project, someone will start to count the cost at some point, and ask the question about when it will be done, how much more will it cost, and do you really need all that staff that is working on it now.

All these are valid questions. A matching management process should be able to answer these questions without over-burdening the team. I believe the Lean management processes do just that.

Key Manufacturing Practices That Are Borrowed

I focus on three key things to borrow from Lean to aid in managing Agile software development:

• Feedback is the source of quality,
• Identification and elimination of waste, and
• Reduction in the number of specialists.

Each of these will be the subject of future posts.

Are you using non-traditional management practices in your Agile team?

Some Agile/Lean References

This is just a quick pointer to a couple of Web items.

• Applying Agile to Ruby is a presentation I did last year at JAOO in Denmark. This Webcast has been nicely edited by the InfoQ guys for viewing. It is not just about Ruby, but about real Agile development practices using complementary Lean principles.
• Boost productivity with lean software development is a survey article on the use of Lean in the UK written by freelance technical writer Cath Jennings. Not only are some of my comments there, but also comments from David Norton of Gartner, a gentleman who really "gets" Agile and Lean.

Keystrokes are NOT Gold

Programmers tend to treat their keystrokes as gold. Perhaps they are driven to counting by project managers. Perhaps their typing skills are atrocious. Regardless, there is a great reluctance to discard code that is wrong. It is wrong because it doesn't work.

A programmer hates being wrong; at least in that respect, a programmer is like every other human being. But in the case of code that doesn't work, there is an almost irrational drive to make it right by adding more code, even if:

• It means abandoning the design,
• Coding practices must be abandoned,
• Complementary changes must be injected into completely irrational areas of system, or
• Changes to key interfaces must be renegotiated.

Traditional phased process (aka waterfall) tacitly accept and encourage these practices. Testing is done by someone else, usually at some point in the future; this lulls the programmer into thinking the code probably works, and it is someone else's job to validate that assumption. Bugs (the real name of code that doesn't work) maybe fixed by someone else. The chances that I, as the original programmer, might have to make it work are reduced.

But then comes Agile, upsetting this comfortable, lazy acceptance of bad code. Two practices in particular inhibit letting the bad code slide: Test-first development and pair programming.

Test-first has the programmer write the test first, then enough code to make sure it works. Then write another test. The effect of this is that code that doesn't work is immediately recognized. Since all the tests are run before check in (or at least run by automatic continuous integration tools), code that injects a bug somewhere else is detected. But what if the programmer skimps on the tests? What if the boundary conditions are ignored so that personal/team/manager schedule expectations are met?

The second practice, pair programming, makes that more difficult. Now two programmers have to conspire to ignore poor test coverage. But that focuses too much on the negative. Rather with a partner sharing the problem, the second pair of eyes helps identify the marginal cases, and helps find alternative solutions. It becomes almost a mutual quest to solve.

The New Problem of Keystrokes

Agile creates an environment where bugs are rapidly detected. So programmers can be lulled into taking bad code and continuing to change it in the hope that it will (magically) start working. In the ThoughtWorks programmer training course, one objective is to get programmers to overcome their reluctance to throw code away and start over.

The first trick to teach programmers to work in small cycles. In the class, we give a problem to the students (always working in pairs). The students are to write one or more tests and get the tests working. And they have only 15 minutes! Invariably, students start with a test that will prove everything works. When the timer goes off at 15 minutes, they are actually stunned that all the time has passed. They are nowhere near getting it to work. They ask for another 15 minutes, certain that they will finish in that time. Invariably, almost all fail again. Sometimes I give them more time, but it is only to get them to realize that their quest is fruitless. Then you finally have pity on them and show them the secret: write smaller tests. Get something working in the 15 minutes. That "resets" the 15 minute clock.

By the second week of the course, pairs are competing on how many cycles they got done in 15 minutes. They discover that in these tiny steps, they are marching to completing the work with absolute confidence that the solution works when they get there. Indeed, they are stunned when they write that ultimate test of the overall behavior, and it takes just a few lines of code to get it to work. A stark contrast to the frustration they felt when biting off too much.

Now the second trick kicks in. If you can't get the code working quickly, throw it away and start over. In these small cycles, they haven't written that much. The old code worked; the clock is ticking; rather than trying to get the bad code to work, they back up and start over. Code repositories are a huge tool in this; modern tools allow you to revert all the code in a couple of mouse clicks. Now the programmers don't treat that bad code as gold.

The top reasons to discard the code are:

• It doesn't work. So why are you spending more time creating a bug?
• If the reason seems to be a muddled design, throw the code away and re-think the design. Kent Beck said, "Code is where design meets the harsh reality of dawn." If the design is bad, the code will scream it out. Listen to the code. So why are you investing more time implementing a bad design?
• The programmers don't understand the problem yet. Getting stuck can be a sign of a lack of understanding. Step back and discuss the problem some more. So why do you think you will understand the problem better by continuing to write code?

See the trend? Continuing to write code is a waste of time. Discard and start over. It is not gold, but rather .

If you are a programmer, try this out: If the code seems to be going nowhere, throw it out and start again. Let me know how it goes...

Three Critical Enablers for Agile

I have been more successful with Agile processes than any alternative I have tried in my 40 years of programming. My early work predates even phased processes like waterfall. Roy Singham, our founder, often teases me by claiming I invented waterfall. Not true; however, I was one of the programmers IBM experimented on with waterfall in the early 70's. I even did a stint in an IBM programming practices group. So when I say have been more successful with Agile, I have tried quite a few other ways.

I have found that there are three critical enablers for successful Agile projects:

• Writing code in a style that let's me change it easily
• Using a compatible management practice (not Gantt charts)
• Throwing heaps of processing power at it

Writing Code That Can Change

I have previously discussed this in The Secret Assumption of Agile. If you can ignore future requirements while working on the requirement for today, you gain enormous productivity. But you can only ignore the future if your code can tolerate the constant changes demanded by that ignorance of the future.

The premise of Agile is that you absorb requirements as they come in, and you accommodate the change the new requirements demand. Does that mean the code you write today may get thrown away? Possibly. Perhaps, even likely. I estimate that for every 100 lines of code I write, I probably wrote and discarded 200 other lines of code. Wouldn't it be faster if I wrote it right the first time? Sure, but not if I have to think about it. What if in order to get it right, I would have to think about it (design, model, review) for the time it would take me to write it 10 times in Agile? That was my experience in waterfall. We often measured how much of the development dollar was actually spent coding (versus reviewing, being in meeting, creating designs, etc.). The answer was ten cents (10%). If I invented the best programming IDE in the world and could instantly write code in it, I would only save 10% of the programmer cost in a waterfall process. So don't spend too much time worrying about how much code I threw away. Rather, measure how quickly the development was done.

For me to spend most of my time coding, I need to code in a style that will not slow me down when I make changes in the future.

Agile-Compatible Management Process

The second critical enabler for Agile is to realize that common project management techniques are not compatible to Agile. These management practices assume that

Software is like building a bridge.

In other words, it is a step-wise process requiring that each step be completed before moving to the next (or almost completed). Hence I need complete requirements before architecting; complete architecture and design before starting construction; and construction must be done in a precise order (physics requires that the pillars be laid and cured before building a road bed on top of them).

But this doesn't sound like Agile. We are overlapping the phases almost completely. And software is not bound to the physics of civil engineering. It is called soft- for a reason.

Rather, Agile is much more like a production process. Indeed with Agile, I think

Software is liking making cars.

What is the best process for making cars? The Toyota Manufacturing Process, renamed Just-in-Time (JIT) when moving outside Toyota, and subsequently renamed Lean when moving outside of manufacturing.

Look for many more posts on the use of a Lean management process for managing Agile.

For now, consider the use of a Lean management process a critical enable for Agile success.

Processing Power

Agile thrives on processing power. I like using the term horsepower. It directly effects the productivity. Consider the typical use of processing power in Agile:

• Using OO languages with interpreters and virtual machines
• Running complete suite of unit tests before checking in
• Use of powerful refactoring IDEs
• Continuous builds being run with suites of tests
• Key code metrics being collected constantly (test coverage, duplicate code checking, etc.)

In an Agile project, you wouldn't choose to not do any of these things. ("Let's use Notepad. It is runs faster than eclipse." "We could use C instead of Java. Java runs too slow on my machine.") When some aspect of the development process is taking too long, the first choice should be to throw more horsepower at it. It is much cheaper than people's time.

Agile in the 70's?

If Agile is so much better, why did we adopt waterfall processes in the 70's? Why didn't Agile emerge earlier? Perhaps we were stupid. I would rather think that we didn't have the enablers:

1. We didn't have the languages (Java, C#, Ruby) and techniques (OO, rules engines) to write code that could be changed. Our paradigms were actually the opposite: Design your data structures, and the code writes itself.
2. JIT (Lean) hadn't made it out of the automotive manufacturing sector.
3. We didn't have the horsepower. I have more memory in my watch than I had in my first computer. And don't think of comparing an entry iPod to the first PC.

So basically, we couldn't do Agile in the 70's. But we can now!

Is your organization still stuck in the 70's?

Ideal Developer Ratios for Pods

In the last post, I postulated that an ideal team structure exists for Agile, a pod. The pod is a collocated, self-contained team able to execute the project. Let's drill down into the structure of this "ideal" Agile team a bit more, focusing on the programming skills the pod should have. These observations specifically apply to a pod using OO programming techniques (see the secret assumption of Agile) to write code that is easily modified. It is a language independent assumption, whether using Java, C++, C#, Ruby, or any of their brethren.

I will describe the team composition using the skills taxonomy I introduced earlier, the Master-Journeyman-Apprentice model. Briefly, an apprentice is the typically trained programmer; the journeyman has acquired the vocabulary of objects, including principles and design patterns; the master readily crafts object solutions to business problems.

The first rule of thumb for the team is that apprentices when paired together are not productive. Indeed, such a pair can be negatively productive, developing code that must be re-opened and reworked before delivery. (Of course, you do pair within the pod; a solo apprentice is truly a destructive force in the code base.)

So the guideline against apprentices pairing together leads to the first rule of an agile team:

Masters + Journeymen >= Apprentices

Simply put, the number of Masters and Journeymen needs to outnumber the Apprentices so that the Apprentices have a more experienced person with which to pair. That keeps everyone on the team productive. It also provides the mentoring to move the Apprentice to a Journeyman (see Building a Better Programmer). With care, the Apprentices gradually become Journeymen, providing more pairing options or the ability to absorb additional Apprentices.Master programmers provide continuous architectural and design direction to a project. They recognize the need for new components to support emerging stories, and pioneer these new components themselves to ensure a sound base exists for the rest of the team. How many Master programmers do you need? Fortunately, not that many:

Masters >= 10% (Masters + Journeymen + Apprentices)

In my experience, you can get away with as few as 10% of your developers as Master programmers; it is safer if you can work that percentage up to 20%, particularly in the early project stages. That may be the same number of Master programmers, but just keeping the team smaller (with fewer Apprentices early as recommended above.)

Pitfalls of a Team Out-of-Balance

The most common destructive mistake for an Agile team is to have too many Apprentices. Not only do they not produce in pairs, the best you can hope for is that they didn't damage the code base today. As individuals, it is an almost certainty that damage has been done. But the damage goes further than the code base; an Apprentice who hasn't positively contributed to the project is going home not feeling good about themselves. That creates an unnecessary morale problem for the individual and the team.. I don't know of any programmer who goes into work saying, "Today I am going to create some bugs." That this happens is not a failure of the programmer, but in the team creating the situation. So bring on apprentices only at the rate they can be productive to the team; otherwise, be courageous and defer the staffing, ignoring what your spreadsheets are telling you.

Even if you can tolerate 50% Apprentices in theory, you should not start a project with that mix. The early few weeks are the time in which the initial application architecture and design become visible. Apprentices can rarely contribute to these efforts, and can feel useless. There is one exception to this guideline: An Apprentice who has domain knowledge is very valuable to this initial team; providing key insights in what will need to be done in the long run. So start your projects with few, if any, Apprentices, and stage them into the project as the rate you can productively absorb them.

If you are attempting a reasonable size project without Master programmers, you are running a great risk of failure. Master programmers are the key to building the application architecture and design incrementally. Agile works by having such talent embedded in the team. Without it, significant up-front architecture and design is required to provide the road map for the Journeymen.

Many Agile projects seem to start fine, then seem to hit a wall in a couple of months. It is almost like Agile is not working. Refactoring is common, and is being cited more and more often in stand-up meetings. This is actually a sign that the design is weak, a reflection of the lack of continuous guidance a Master programmer provides. So ensure you have sufficient Master programmers on the project, particularly in the formative stages.

The flip side of the Master programmer requirement is that they may not be required in small or enhancement projects. A small project for certain short-term benefits may not reach a crisis point before it is complete. Similarly, if enhancements are being made to an existing, healthy code base, Journeymen are fully capable to following the tracks laid down by previous Master programmers. So Master programmers are not required for small enhancement projects.

Ideal Skill Ratios in Pods

So are, we have talked about pods, ideal team Agile team structures, and the proper mix of developer skills in the pod. In this post, I would like to describe the needs for other skills in the pod.

An ideal pod would contain:

• 4-6 pairs of developers (8-12 total)
• 2-3 business analysts
• 2-3 testers
• 1 ambassador (aka project manager or iteration manager)
• 1 customer or customer representative (rotating on need)

So adding to our pod formulas:

Analysts + Testers + Managers + Customers = 80% (Developers)

So if I calculate how many programmers a project should need, I add another 80% to cover the other roles. So between this ratio and the productivity uplift I give to the programmer skills (5:2:1 for Masters, Journeymen, and Apprentices), I have the tools for doing my Agile project plans for a pod. Of course, this is just the starting point for the plans. I adjust skills constantly throughout the project.

For example in the first release of the project I am currently working on, we ran it successfully with 3 pairs of developers, two analysts, a single tester, and a single project manager. The next release had the same 3 pairs of developers, but only a single analyst and a single tester; metrics showed we should have had a second tester, however. Overall, however, the project is successfully running light on non-developer resources.

On the other hand, an early project on which I worked at ThoughtWorks ran its last release with one project manager, one combined customer/business analyst, 3 pairs of developers, and over a dozen testers! The customer was writing the stories herself, perfectly able to keep work flowing to the developers. The developers were a Master-heavy mix, and produced accordingly. This high productivity drove the need for up to 17 testers at one point; no investment in automated acceptance tests had been made.

Despite these examples to the contrary, I like the above ratios for planning. It is my starting point, and I adjust accordingly.

Pods -- Ideal Agile Team Structures

I described in a previous post that you probably have the wrong team working on your Agile project. It is wrong because you can't find the ideal team members initially, it is wrong because the needs of the project change as the project progresses, and it is wrong because the members of the team themselves will grow and change. The core recommendation from that post was to constantly tweak your team.

Typical, productive Agile teams seem to conform to a certain shape. This observation is borne from my experience in almost a decade of Agile projects, and is reinforced by colleagues at ThoughtWorks from their experiences. I am calling such ideal Agile team structures pods. I offer yet-another-term for several reasons:

• The term team doesn't convey enough,
• I haven't read of an alternative term,
• Several other good words are already in use,
• I like the collocation connotation of pod,
• I like the self-contained connotation of pod (i.e., all the needed skills are collocated)
• It provides an organization unit for broader people-oriented discussions
• There is a quirky sci-fi angle I can't resist

I will be exploring many aspects of pods in a series of posts. Let me quickly summarize where this is going:

• There is a particular mix of developer skills in a pod (a mix of Masters, Journeymen, and Apprentices;
• There is a particular mix of team skills (analysts, developers, testers, management);
• The team is self contained and collocated;
• The team is empowered, making decisions in its areas of competence;
• The team is self-managing (including self-apprasing);
• Team metrics provide insight into the effectiveness and composition of the team; and
• The team is semi-permanent, spanning project life cycles.

Much of this is common sense. Much of this is obvious to practioners of Agile. But much like the practices of XP, the total is greater than the sum of the parts. I still strive to create the perfect pod for projects on which I work, but I haven't yet achieved it.

Do you have specific characteristics for your ideal Agile team?