Breidenbach - Grain and ICBMs

By Kevin E. Breidenbach - 29 October 2008

As you drive around the countryside of the Midwestern United States, you'll see silos towering over farmland. Depending on the time of year they may even be filled with grain. If you know where to look you might even be able to find a subterranean silo which housed one or more Minuteman ICBMs from the days of the cold war. In either of these cases the silos are used for a good purpose – to keep things in that either feed people or vaporize them!

For some reason though, there is a group out there that like to fill silos full of people. Even stranger is the fact that these people all work together but they may be put in different silos. It's really difficult to talk across silos, so nobody really knows what the people in the other silos are doing.

So who makes up this weird group that like to segregate people? After studying this group in the wild, researchers have named them Technicilli Managerius, or Technical Managers to us common folk.

In all seriousness this has been a problem for a long time. As an example, consider a large application that has many different major components. When the system is being designed, the managers often see benefits in creating individual teams to work independently on each component, with each team having its own manager, development leader and several developers. The teams start building their components and everything is dandy, but apart from the odd conversation over a beer between different team members, communication between silos is minimal. And this is where the problems begin.

If one team is having problems, it is unlikely that other teams will know about it. Different teams may very well be struggling with the same problem and coming up with different – even incompatible – solutions. Or they can play a game where each team waits for another to signal “defeat” and admit to being behind schedule. This defaults the entire application team into delay, giving everybody more time to complete their work – or simply another chance to play the game.

This inter-team disconnect is particularly prevalent in waterfall projects, which are far from transparent. Of course, the problem isn’t unique to waterfall; it can happen to any project. Agile teams have metrics such as burn down and velocity that allow us to see where they are, and across a large program of work this gives very good insight into which teams are behind schedule. But visibility doesn’t address the underlying problems of teams working in silos: multiple teams will still be working independently of each other.

I believe that fluidity of people across teams would improve communications and benefits the project as a whole. How would this work? Each major component has its own iteration manager and possibly a lead developer (although I'm becoming convinced that there is really no need for lead developers and a flat structure really is better). A developer can, when needed, move among component teams. In this structure, the "team" exists at the application level and not at the component level.

It can be argued that this fluidity is difficult to manage, and that developer ramp-up on each component is time consuming, both of which will impact timelines. These are valid points, but they don’t make a compelling counter-argument to trying this approach. If you have competent developers on your team, if there is nothing drastically different in the design of each component, and if there aren’t very different languages for each component, then developers moving among components shouldn’t be too much of a problem. Of course, not every developer may want to move between components, and they shouldn't be forced to make a move. But most developers are inquisitive enough to want to see what is going on in different components.

Another way to combat silos is through consistent, highly focused communications within and across teams. For example, a good development team holds daily stand-up meetings. They’re a great way to concisely communicate what people are doing and the problems they’re facing. While it isn't possible to bring a very large application team together for a stand-up, there is nothing to stop iteration managers from having regular stand-up meetings to inform each other of what their teams are doing and what problems they are facing.

Leave silos to grain and thermonuclear weapons: they are much better suited to that working environment than people. Development teams just get tunnel vision and projects end up badly disabled and sometimes dead. Plus, silos look terribly uncomfortable!

---

About Kevin Breidenbach: Kevin has a BSc in computer science and over 15 years of development experience.  He has worked primarily in finance but has taken a few brief expeditions into .com and product development.  Having professionally worked with assembly languages, C++, Java and .Net he's now concentrating on dynamic languages such as Ruby and functional languages like Erlang and F#.  His agile experience began about 4 years ago.  Since that time, he has a serious allergic reaction to waterfall and CMM.

Pettit - Volatility and Risk of IT Projects

by Ross Pettit - 22 October 2008

When we plan IT projects, we forecast what will be done, when we expect it will be done, and how much it will cost. While we may project a range of dates and costs, rarely do we critically assess a project’s volatility.

Volatility is a measure of the distribution of returns. The greater the volatility, the greater the spread of returns, the lower the expected value of the investment.

In finance, volatility is expressed in terms of price. In IT projects, we can express volatility in terms of time, specifically the length of time that a project may take. Time is a good proxy for price: the later an IT asset is delivered, the later business benefit will be realised, the lower the yield of the investment.

Using Monte Carlo simulation we can create a probability distribution of when we should expect a project to be completed. DeMarco and Lister’s Riskology tool calculates a probability curve of future delivery dates. It allows us to assess the impact of a number of factors such as staff turnover, scope change, and productivity on how long it will take to deliver. In addition to projecting the probability of completion dates, the Riskology tool will also calculate the probability of project cancellation. The inclusion of fat tail losses makes this a reasonably thorough analysis.

Monte Carlo simulation gives us both an average number of days to complete a project and a variance. Using these, we can calculate the coefficient of variation to get an indicator of project variability.

Standard Deviation (days)

CV = ------------------------------------------------

Average (days) to complete the project

The coefficient of variation is a metric of project variance. The higher the CV, the greater the volatility, the greater the risk there is with the investment.

Historical project data allows us to put a project's CV in context. Suppose we have metrics on prior projects that show the projects IT does for Accounting have an average variance of 1 day late for every 10 days of plan, with a variance of 0.5 days. Suppose further that a new project expected to last 6 months is forecast to be 6.5 days late with a variance of 3 days. Somewhat akin to assessing the Beta (although it is not calculated entirely from historical data), this comparison allows the IT project investor to ask piercing questions. What makes us think this project will be different from others we have done? Why do we think the risk we face today is so much different than it has been in the past?

It is appealing to think that by supplying some indication of the volatility and Beta we’re giving investors in IT projects a complete picture, but there are limits to what this tells us. Our risk assessment is only as good as our model, and our model will be laden with assumptions about people, capabilities, requirements and opportunity. What if we fail to completely identify and properly anticipate both probability and impact of the risks that we are so carefully modeling? Come to think of it, isn’t this what got Wall Street into so much trouble recently?

Risk management is a primitive practice in IT. All too often, it is little more than an issue list with high/medium/low impact ratings. Typically, risks are neither quantified nor stated in terms of impact to returns. Worse still, because they’re simply lists in spreadsheets, they’re often ignored or denied. This means that the IT project manager is focused on internal risks, such as technology challenges or days lost due to illness. They don't pay much attention to external risk factors such as a regionally strong tech economy that threatens to lure away top talent, or a supplier teetering on the edge of receivership. By quantifying the impact of risks – both individually and collectively – the project manager comes face-to-face with the broader environment on a regular basis.

Focusing on volatility also makes it obvious that the “on time and on budget” goal is achieved not so much by what goes right as much as by what doesn’t go wrong. That, in turn, strongly suggests that being "on time and on budget" is a misdirected goal in the first place. It also suggests that traditional project management has less to do with project success than we might think.

Let’s consider why. Traditional IT manages to the expectation that everything can and should go according to plan. Changes in scope, changes in staff, and even mistakes in execution are treated as exceptions. Traditional IT management looks to reconcile all variations to plan and will attempt to do so even when the sum of the variations – 100% staff turnover, 100% requirements change – creates a completely new project. The traditional IT project manager is trying to maximise effort for available budget: “the project will be on time, what might go wrong to make it late, and what must we do to compensate.”

By comparison, Agile project management expects change. It focuses on reducing volatility by making deliveries early and often. The Agile project manager is trying to maximise yield given available time: “the project will be late, what can we do to bring it forward, and what can we do to reprioritise.” This is a subtle - but critical - shift in thinking that moves risks from the fringe to the center of the frame.

Risk analyses are supplements to, not replacements of, good management. Risk models are abstractions. No risk model will capture every last exposure. Assumptions in underlying numbers will be wrong or arrived at inexpertly. Worse still, they can be gamed by participants to tell a particular story. All this can lead to poor investment decisions and ultimately investment failure. Still, while working in blind adherence to risk models is hubris, working without them is reckless. Investment managers (or in IT, project managers) are paid for their professional judgment. Investors (IT project sponsors) are responsible to audit and challenge those entrusted with their capital. What managers and sponsors do pre-emptively to ensure project success is far more complete when risk analyses are brought to bear.

The higher the expected yield of an IT project, the more important it is to be aware of volatility. If volatility isn't quantified, the project lacks transparency, and its investors have no means by which to assess risk. This leads to a misplaced confidence in IT projects because long-term success is assumed. If the recent financial turmoil has taught us anything, it's that we can assume nothing about our investments.

Kehoe - The German General Staff Model and IT Organizations

By John Kehoe - 15 October 2008

A couple of stories caught my eye this month and dovetail with a model I’m investigating. First, a column, Meet the IT Guy, outlines the typical, hackneyed view of the IT archetype (still funny stuff). The second, an excerpt from Numerati in Business Week, examines IBM’s effort to model consultant abilities and cost to map the right people to the right job and model how a successful expert is created (neat for high level experts, but a bit scary for lower level consultants).

This leads me to the characteristics an IT organization needs to excel. Curiously enough, they are descendents of the German General Staff (GGS) from the period between 1814 and 1900).

Staffing Model

The German General Staff (GGS) was created at the end of the Napoleonic Wars as a reaction to the military officer corps being staffed by men who purchased their positions. The "commissioned" officers (as in, paid for their commission) were on a quest for personal glory. As a result, they fought by rote, did not create new strategies, got a lot of men killed and wasted resources. The GGS mission was to create a professional officer corps to change that mindset and with it, achieve better results.

Candidates for the GGS were rigorously vetted for competency and motivation before an invitation was extended. Once in, GGS officers were categorized in two dimensions: motivation and competency. This can be represented in a 2 x 2:

This is how the GGS staffed the right person for the right role.

Clearly, in any organization, a mismatch breaks the lot. Play the mental exercise of putting your people in these positions. It is easy to find an implementer in a manager position or a manager in the general’s seat. Recognizing the mismatch makes obvious why things bog down or spin out of control.

Organizational Values

The GGS instilled the following values in its members:

• Dedication: By investing in an individual, that individual is more committed to the organization
• Motivation: Devise good strategies that are efficient, flexible and victorious
• Dissemination: Get the ideas into the field
• Doctrine: Get a common process in play
• Innovation: Think outside the box to adjust to the circumstances
• Philosophy: How do we go about our business? What are the ways and means we chose to use or not use.

Now, why put forward the GGS as a role model? Because the Germans where the first to do it. Today, every major nation has some sort of joint military command and training structure. In order for a military to succeed in a campaign, it must leverage every resource to maximum productivity and align tactical activities with strategic goals. The most successful operations – the most successful businesses – have the concept ingrained from top to bottom.

This approach makes clear how important it is to place each person to his or her abilities. Suppose a junior level person is designing a technical architecture for performance management. This mismatch is a high risk for the organization and the individual. Don’t axe the person because they perform poorly, move him or her to a task commensurate with their ability. Make clear that this move is not a punishment, just a rebalancing of the skills portfolio. If, however, a person falls into the lazy/incompetent quadrant, well, you know what to do.

Another interesting characteristic of the GGS is that it rotated people off the line into staff positions, and then back to the line. Many IT organizations have one set of people who put code into production and a cadre of architects in staff (or non-line) positions. Or they have people managing projects in their own way, ignoring efforts of a central PMO to create consistent and professional PM practices. Either is an example of an organizational separation within IT, with “central office” people on one side and “executors” on the other. By rotating people in and out of staff positions, IT policies are more likely to be actionable and not academic. There is also more likely to be buy-in and not resistance to IT standards. Perhaps less obvious, it contains the expansion of IT overhead as there are few career "staffers:" everybody will be back to the line before long. Finally, it makes IT less of a cowboy practice, and more of a professionally executing capability.

Mapping the Concepts

How do we map the GGS to an organization to measure its potential?

• Dedication is obvious by the rate of employee exit and replacement. IT is a highly mobile profession; "healthy" IT organizations will have 15% turnover or less.
• Motivation is recognized by the creation of value, not the number of overtime hours worked. Do we deliver in a timely fashion? Are we receptive to other organizations? Do we know and appreciate the impact of our action and inaction?
• Dissemination is judged by joint cooperation. Do tasks get done in a timely fashion across a joint team? Do people know the resources and responsibilities around them?
• Doctrine can be summed up simply by asking: do rules define the exceptions or do exceptions define the rules? If our rules are exception based, we have a problem. We have multiple options and cannot rationally consider their impact.
• Innovation is the game changer that gets us ahead of the competition. What can we change that drives revenue or improves margins?
• Philosophy defines the actions we accept and reject. What motivations do we accept or reject? Are they telegraphed throughout the organization? For instance, we should have profitability as our guide. How to we achieve it, growth or cost cutting. Does advantage come from outsourcing or offshoring? How do we balance the other 5 values?

The GGS was not a rarified career opportunity devoid of delivery expectations and obligations, but provided a means by which to circulate expertise and provide experience. It offers IT a straightforward way to fashion planning and career development, as well as a means to incubate ideas and individuals. It starts with a clean sheet of paper, a cup of coffee and insight into your business, organization and services portfolio. Give it a try today.

---

About John Kehoe: John is a performance technologist plying his dark craft since the early nineties. John has a penchant for parenthetical editorializing, puns and mixed metaphors (sorry). You can reach John at exoticproblems@gmail.com.

Ververs - Generation Y: Enabling Autostart

By Carl Ververs - 8 October 2008

A pack of Millennials sit in a conference room. The phone rings. Says one to the next: “Dude, you gonna get that?”, Says the other: “Like, Yah! And get, like reprimanded, or something!”. Five minutes later their manager storms in, yelling: “Why aren’t you picking up the phone??!?! I was wondering where you were!” Says one in reply: “We didn’t know we were supposed to!”

This would be cute if it were just a joke. But this is the life of managers of Millennials.

When compared to previous generations, Generation-Y comes off as a bunch of lazy, indecisive airheads. But there is more to it than that. And after a bit of closer examination, you will find that the Gen-Y attitude has some real advantages.

Millennials didn’t jump on their Facebooks and collectively decide to give every prior generation a hard time. Their work attitudes have a clear origin: the overzealous, results-driven approach to child rearing that their parents have practiced. Like many things in their lives, Late Boomers and Gen-X couldn’t just enjoy their children, they were to be made projects, with a results baseline and earned-value metrics. Consider a recent remark overheard from a mother of a 5-year old: “Jason still can’t write a full sentence. I’m considering occupational therapy.”

The cause for excessive parent involvement may be rooted in their investment approach to raising their children. In a Chicago Tribune article from September 9th, 2008, a helicopter parent justifies her behavior by saying that she feels parents must stay involved in their children’s life in college to make the $200K investment worthwhile. This cynical perspective will not be lost on the “assets” and will no doubt make them feel reduced to Boomer accessories (that is, if the dual-career-hound, passed-off-to-sitters pattern hasn’t done so already). There is, not surprisingly, an entrenched defense for helicopter parenting, as a CPA article from September 2006 shows.

As involved and engaged as these parents have been with their offspring, they have missed something crucial: their children can actually think for themselves and have drawn the conclusion that, like all generations before them, their parents are wrong. They have revolted against the workaholic, asset-oholic, keeping-up-with-the-Gateses attitude and are simply rejecting the now-bankrupt notion that personal success is measured by how much stuff one has. "Turn on, tune in, drop out" has been replaced with "iTune in, Log on, Check out your friends’ Twitter updates."

Millennials demand a better work/life balance. There are several advantages to this. For starters, they are less likely to burn out. They have to be very efficient to complete the work assigned to them. They tend to get auxiliary work input from multiple communications channels. Rather than working long hours, they challenge the status quo, demanding (as they should) solutions to lingering problems that create the need for excessive work hours.

However, an inflexible attitude to working late can make them unreliable in crisis situations. Evening and weekend work is sometimes necessary. A Gen-Y person in a key role unwilling to put in the effort when it is needed most can mean that deadline-driven work languishes and projects are delayed. Striking a compromise is difficult. A developer on one of my teams had no higher priority than running. He was consistently late with his work, which was often also incomplete. We agreed to small, very concrete deliverables. While they were completed properly, the experience left me feeling like I was micromanaging and impeding his growth.

While Millennials want to have a life, they also want to have a meaningful job. They want their ideas to be heard and their opinions to count. The upshot of this is that bright minds will not go to waste. Staff can be motivated by big ideas and by seeing clearly how they contribute to revenue. But youthful enthusiasm often gets dismissed as daydreaming, discredentializing fresh insights. Millennials must be taught that most people don’t “get” new ideas at first. Promoting and realizing them requires discipline and tenacity.

A bigger problem is the sense of entitlement Gen-Y, and to a greater extent Gen-I, feels. They expect everything to be on-demand and immediate. If something requires application, study and practice, it’s dismissed, considered to be not worth the effort. This puts Gen-Y at a disadvantage with people from other cultures who have not followed the Western patterns of generations. In the West, the mantra is "work smarter, not harder." But some skills cannot be obtained without actual practice. And without practice, one has to work harder to obtain a similar result as a skilled worker. To make things worse, the stimulus for Millennials, and Generation-I to try harder is muted by being told that “everyone is special”. In the infamous words of Dash Parr: “If everybody’s special, nobody is.” See also the Cerebral Dad’s point of view on this.

Do not expect Gen-Y to share your habit of getting up at 5:30, checking the market, wolfing down a processed breakfast and getting to work by 7:30. Just because Jack So-and-so or Rupert Whatisname did things that way means nothing to these kids. “Legendary CEOs” aren't Gen-Y role models; they want to be like the Google guys.

So what can we do with all this? How can we turn Generation Y’s work attitude into something beneficial?

To cater to their demand for meaningful work, tie anything that needs to be done to revenue. Coach them to set high personal development goals and track these at review time. Have prospective young employees define “meaningful”. If that does not align with what your company’s business is, they may not be a good hire. Alternatively, people being hired for non-revenue positions may very well be motivated by your company's participation in community outreach programs.

The pack mentality of Gen-Y is well suited for collaborative teams, and makes them uniquely prepared for Agile work environments. Rather than putting them in single cubicles with a window to an Internet full of distractions, put them in larger rooms with shared table-space. The peer pressure will keep momentum up and distractions down. The collaborative setup will bring out the best in them. The small chunks of work will fit their attention span and the rapid results will cater to their zest for instant gratification.

The concept of people being in an office 9-5 is bunk. Urban sprawl and associated traffic congestion, rising fuel prices and subsequent public transportation congestion conspire against this ancient rite and make telecommuting more attractive. Bosses who want people in an office at all times do not understand how to measure the value that their staff brings. Either the work gets done at a desired quality level or it doesn’t.

Generation-Y will be much more open to alternative ways of working and will profess to have a higher set of morals than our contemporaries do. This creates an opportunity for us to groom our future leaders, driven by their own ambitions. So if we let go of our idée fixe of how people are supposed to work and focus on achieving results, we will get full cooperation from our new recruits and just may learn a new trick or two.

Who knows, you and your executives may like this new-found efficiency and take up a new, Boomer-appropriate hobby. Harley, anyone?

Spillner - The Eternal Struggle Between Good Enough and Perfect

By Kent Spillner - 1 October 2008

Software development projects should not be managed with the assumption that the final product would be perfect if only there weren't budgets or deadlines. Although theoretically true -- given infinite time and effort, the perfect solution would eventually be developed (along with an infinite number of imperfect ones) -- this style of thinking only sets the team and business up for failure. Instead, project teams should focus on delivering a working product today, and a better one tomorrow.

Perfect Ain't Possible

I applaud the good intentions of teams that plan to build the perfect solution, but that's not a realistic way to work. No amount of planning can account for every contingency (well, short of infinite planning, but then there wouldn't be time to develop anything!) Ditto development. Endlessly cranking out code is not a recipe for perfection. Unfortunately, it's hard to argue against such behavior. That's partly because, again, the assumption is theoretically true, and partly because it's in our nature. When people can visualize a goal, they're able to figure out the necessary steps towards achieving it. That's a very powerful capability. It's also very misleading. It requires tremendous genius to correctly figure out every step required to build something of modest complexity. And it requires even greater will power to backtrack several steps when you start heading down the wrong path.

The rapid evolution of modern business makes it impossible to correctly figure out every step, often because the goal itself is constantly changing. And the dynamic nature of software makes it especially painful to head down a dead end because developers can push code quite a ways before eventually hitting the wall. Now throw in budgets and deadlines. Not only do teams have to do everything right, they also have to do it on time the first time. Trying to build perfect components that make up a perfect whole is an uphill struggle, even without the business constraints.

Lather, Rinse, Repeat

Instead of pretending to build the perfect system, software development teams should build a good one, then make it better. Not in the next release or the next update, but in the very next build. Continuous improvement, not ultimate perfection, should be the goal. The difference in approach can be staggering. Where teams that strive for perfection end up floundering, teams that focus on continuous improvement end up delivering. The uphill struggle becomes a leisurely stroll. This is not meant to imply that continuous integration is a convenient excuse for laziness or slack. On the contrary, continuous improvement is hard work that requires discipline and dedication, just like striving for perfection does. But whereas achieving perfection is impossible, stepwise monotonic improvements are manageable.

Prologue

When I was an undergraduate, I worked as a programmer at a research lab on campus. It was a formative experience. Super smart people, great work environment, interesting problems. Then we lost our funding. That's when reality sunk in: we weren't as good as we thought. To get our ship back in order, we hired a full-time project manager. Under his guidance, we rewrote two whole research systems from scratch in about nine months. Our experience from the previous systems certainly helped our productivity -- we knew several mistakes not to make again -- but it was his approach to building software more than anything else that made us successful. I've experienced similar situations three more times since then (I'm sure I'll learn eventually). In each case, the deciding factor in the team's turn-around was a dedication to steady improvement. Instead of shooting for the moon, we worried about building better software.

Stop Hitting Yourself

That's what this column is all about. I've been a part of underperforming teams. I've pulled all-nighters to meet deadlines, fixed bugs right up to the start of demos, broken my share of builds, and had my projects cancelled. I've also been a part of alpha IT teams. I've helped put systems into production ahead of schedule and under budget, I've worked on projects that release new versions week after week without any serious bugs, and I've felt the excitement return to the office as teams respond to disaster by building overwhelming success. I want to share with you the lessons I've learned, and help you avoid the mistakes I've made. I want to offer advice borne from my own personal experiences, and help you build successful software.

It's All About the Build

I used to think that the best place to start changing software development teams was at the bottom. Get the developers to write better code and focus more closely on building what the customer wants. Now, I think it's best to go straight for the gold: the build. Builds are the only true measure of success on a software project; they're the only product that delivers value. Hence, my first column on the importance of frequent, regular releases into production.

Then what? What should you do after delivering new features and functionality in individual increments? Make the product better. You don't have to invest in making it radically better, just better. And make it better with every release. And what then? Your product is improving, let's improve the process, too! I want to help you iterate, and iterate well. I want to help you organize your work, and then I want to help you manage it better. I want to help you engage every member of your team, and I want to help you keep every member engaged. And all the while, I want you to continue pushing out new versions of your software, and I want you to make each better than the last.

---

About Kent Spillner: Kent loves writing code and delivering working software. He hates cubicles, meetings and Monday mornings.