Pettit - Restructuring IT: Making In-Flight Change

By Ross Pettit - 24 Febuary 2009

All businesses are undergoing unprecedented changes. Revenue forecasts aren’t materializing, capital structures are proving unsustainable, and operations are being scrutinized for inefficiencies. This, in turn, means that businesses are being completely restructured in how they are capitalized, organized, managed and governed. As businesses restructure, so will IT.

As we face restructure, we have to look critically at our IT lifestyle. We know that if we eat a poor diet and don’t exercise, we run a greater risk of health problems than if we eat a healthy diet and exercise regularly. The same applies to IT: if our work habits lack discipline, we’re going to have health problems that, in turn, put business operations at risk equivalent to heart disease or diabetes.

Agile and Lean offer IT a healthy lifestyle choice: disciplined execution of a set of best practices give greater focus, consistency and transparency to operations than we get from traditional approaches to IT. Experience in a variety of business domains tells us that we can expect significant impact by taking on Agile practices: we will reduce the probability of a catastrophic failure, defects will decline substantially, delivery times will accelerate, and the business value of what we deliver will increase.

But restructuring to be an Agile / Lean IT organization is not something that happens by management fiat. Agile IT requires that each person embrace fundamental principles that value the business problem at hand as opposed to the technical problems we concoct. This requires behavioral changes that run counter to decades of training, messaging and mentoring embedded in the IT profession. In fact, it goes to the heart of the oft-cited gap between business and IT: day-to-day IT activity is often fundamentally misguided, as people are busy solving the wrong problems. Indeed, it is not uncommon for people in IT to subserviate a very real business need to go in pursuit of speculative technological “future-proofing.”

Given the urgent need to restructure, this behavioural gap presents IT leaders with a significant challenge.

First, restructuring takes a lot of effort. If restructuring IT requires changes to fundamental work habits, we must not underestimate the amount of effort that will be needed to bring this change about. Bridging the gap between “how we work today” and “the results we must achieve given the reality we face” is not simply an exercise in tools and training. It requires a concerted effort to change the behaviours that underlie how work is done: how requirements are defined, solutions are developed, teams are organized and managed and IT is governed. This comes about through experience. Change happens within each team and department as people gain proficiency with new work habits while delivering, supporting, and maintaining solutions.

Second, IT can't shut-down while it restructures; it must be restructured while delivery work is in full flight. That means the restructuring effort itself will be subjected to change and adaptation. This makes restructuring a moving target, which both blurs the vision of the target state and wears down people’s patience and energy for the change.

Significant organizational effort spent in pursuit of a moving target will put the change leader in a constant state of conflict. On the one hand, he or she risks managing to a compromise, where long-term sustainability is sacrificed for short-term "results" (often, ironically, in the name of pragmatism). For example, a project team may elect not to introduce unit testing because the effort is believed to be too great and the business need too urgent. The team may write code faster initially, but it will prove to be a false efficiency as defects rise and time spent refactoring obliterate any gains. On the other hand, the change leader must not be dogmatic. An IT organization doesn’t exist so that it can be Lean, it exists so it can deliver business results. Too great of emphasis on process – insisting on 100% unit test coverage, for example, just for sake of having high unit test coverage – risks prioritizing process in favor of results.

Most of the challenges the change leader will face during in-flight restructuring come down to a simple, if not always obvious, test: are we reconciling the reorganization to the business demands or reconciling the reorganization to old work habits? The prior part of the test helps us be sure that dogma doesn’t trump results. The latter tells us that we must not compromise in the name of making everybody happy.

To make this decision consistently, even when the goalposts are moving, change leaders must have a clear understanding of both the business need and the goals of the restructuring. By keeping the business outcomes such as reduction of defects or accelerated time to delivery the clear priority, we bring unambiguous focus to the change effort. And that focus is critical. Every day, change will be challenged by all kinds of things: distracting and counter-productive technical objectives (e.g., “we need to solve every possible problem we may have in this and any future application that requires session management”), irrelevant, non-value-added IT practices (e.g., "we've always required effort-remaining project status reports"), and individual motivations that run counter to change (such as job preservation or people's "stationary inertia" at work). In-flight operations restructuring requires intense, unrelenting effort to swim against the tide of “how things have always been done here.” The change leader must have a clear vision for operations that is flexible to business demand but uncompromising to IT convenience if there is to be a real, durable restructuring.

The change leader must always be clear that the goal of restructuring is to have the organization executing in such a way that it sustainably yields better performance. To be sustainable, we don't just need solutions to report improved technical measures, we need to work in such a way that day-to-day project decisions do no harm, much like decisions we take in our personal lifestyles. For example, we can send code to the "technical clinic" for IT liposuction, where we invest time into remediating tight coupling, dependencies and complexity so that a team has a clean code base, free of technical debt. However, just as the stomach-stapled person may resume bad dietary habits and regain weight, so will the IT team with the remediated code base resume bad habits. This means that IT leaders must insist on a constant, independent validation that restructuring has taken root. One way to do this is to regularly scorecard team performance to scrutinize execution. Another is to make sure that organizational structures – incentives and rewards, recruiting and promotion, governance and oversight – reinforce the Agile value system. If these things are done, old habits are unlikely to return.

The pressure has never been greater on IT. Business is asking, “what are you doing for me this quarter” with increasing anticipation and scrutiny. The best guarantee that IT operations can adequately and professionally answer this question is to execute in a transparent, consistent and disciplined manner. In an uncertain business world, that’s the best form of “futureproofing” IT can pursue.

---

About Ross Pettit: Ross has over 15 years' experience as a developer, project manager, and program manager working on enterprise applications. A former COO, Managing Director, and CTO, he also brings extensive experience managing distributed development operations and global consulting companies. His industry background includes investment and retail banking, insurance, manufacturing, distribution, media, utilities, market research and government. He has most recently consulted to global financial services and media companies on transformation programs, with an emphasis on metrics and measurement. Ross is a frequent speaker and active blogger on topics of IT management, governance and innovation. He is also the editor of alphaITjournal.com.

Reiser - Getting the Most out of Offshore Development

By Greg Reiser 11 February 2009

Many businesses avoid using offshore resources to implement high-value, high complexity software projects and many who try are frequently dissatisfied. In a recent survey Forrester Consulting found that nearly half (46%) of businesses were unhappy with their offshore provider for development of mission-critical applications.

This begs the question, “Is offshore development incompatible with high-value high-complexity projects?”

My experience suggests otherwise. I have worked with many talented teams that have successfully delivered high-value high-complexity solutions using a distributed development model that takes advantage of talent located in multiple locations around the world.

In this article I will briefly describe how specific practices, some associated with agile development, will help you get maximum value out of your distributed development efforts.

(Please note that I prefer to use the term “distributed development” instead of “offshore development”. This describes projects where work occurs in multiple locations. It is my experience that there are no “offshore” projects. Rather, there are distributed projects where a good bit of the work is performed offshore.)

Involve the whole team in the planning – This is counter-intuitive to those organizations that rely on up-front analysis and planning and then engage a development team (possibly a vendor) for the implementation work. This presents two problems.

First, the development team will insist on doing its own analysis in order to fully understand the business requirements document, use cases, RFP, etc. Hence, some of your up-front effort will be redundant.

Second, and more importantly, it’s during the up-front analysis and planning that the business sponsors and subject-matter-experts establish a shared understanding of the project vision, business objectives and priorities. If the development team (or vendor) is not involved in this process they will be less effective in adapting when the inevitable surprises occur.

Sorry folks, documentation just doesn’t cut it. To paraphrase General Dwight D. Eisenhower, “The plan is nothing, but planning is everything.”

Monitor the product, not the process – Many agilists criticize earned value analysis (“EVA”). Although the EVA technique has shortcomings, the basic idea of assessing project progress by the value delivered is sound. It is much better to assess progress based on functionality delivered rather than interim deliverables such as the System Design Specification (“SDS”) or worse, budget or schedule consumed.

Whether the project team is distributed or not, in-flight metrics based on implemented functionality and user feedback are significantly more reliable barometers than traditional “percent complete” metrics. How many projects have you seen that are 90% complete for half their actual schedule?

Although the SDS may be a very important and valuable artifact (I’m not one of those people who demonize all documentation), its veracity is suspect until an implementation validates it.

Admit that you have a (communication) problem – It doesn’t matter if your project methodology is agile, waterfall or something else. Communication and efficiency suffer when team members are separated by miles. It gets worse when they are separated by time zones as well. In addition to the obvious investments in communications infrastructure and practices (high-quality speakerphones, high-bandwidth networks, overlapping workday schedules, etc.), consider some not so obvious practices.

One of my favorites is the “exchange program”. Have people from each work location spend time working (not just visiting) in the other work location. I’m talking about at least two weeks at a time, working side by side with peers in the other geography. This is both an effective knowledge-sharing and team-building technique.

Don’t give in to the argument that “money saved on travel can be more effectively applied to real work”. The improvements in efficiency far outweigh the additional travel costs. On one large complex program we experienced a 200% increase in throughput after we implemented an exchange program.

I don’t necessarily advocate expensive video-conferencing equipment and services. Although it is a blessing to have access to such, the cost-benefit is not as favorable as it is for less technically sexy investments.

The bottom line is that your project plan must address the communication challenges inherent in distribution. If the necessary investments exceed the benefits of doing the project in a distributed way then don’t do it that way.

Redundant roles – A typical pattern for distributed projects is to have the project manager, subject-matter-experts (“SME”) and business analysts at the customer site and the developers and testers at an offshore facility. This works quite well for small, short-term projects, but it is sub-optimal for large, complex programs.

Concentration of expertise and/or decision-making in one location slows the project down, as the development team often has to wait a full day (best case) when certain types of impediments arise.

Having business analysts and/or SMEs close to the developers and testers improves throughput as questions are answered much more quickly. Having project managers in each location improves throughput as many decisions can be made much more quickly. Having developers at the client site improves throughput as critical defects can be resolved immediately and integration challenges can be addressed in a timely manner.

From a lean perspective, redundant roles are actually a best practice because the apparent waste in redundant roles is significantly less than the waste generated by the wait times in the more typical staffing model.

Monitor technical integrity – “Technical debt” is a metaphor developed by Ward Cunningham that describes the long-term terms costs associated with “quick and dirty” development. The “interest” on this debt is realized in the form of increasing development costs due to inflexible and/or overly complex design, excess dependencies, duplication, defects, etc.

Technical debt management is important for any project. Distributed development makes technical debt management more difficult because some, if not most of the code is developed by the offshore part of the team. Hence there is a greater risk that by the time the symptoms of too much technical debt become obvious, the cost to correct will be high.

You might replace your offshore team due to poor performance and you might even recover some of your costs. But if this really is a mission-critical project, your business will still suffer. Although you can and should apply manual techniques for monitoring the technical integrity of the software under construction, I have found that the use of automated tools to monitor indicators such as automated test coverage, adherence to coding standards, design quality and design complexity can serve as early warning signals for technical debt growth.

Using these types of tools within an automated build process that sounds an alarm when metrics fall outside specified thresholds accomplishes two things. One, they trigger investigation and potential remediation when it is still possible to do so at low cost. Two, they automatically encourage design discipline. No one wants to be singled out for violating coding standards, for not following the agreed upon testing discipline, or writing unnecessarily complex code.

Once again, a modest investment can reduce risk and saves significant costs within a relatively short period of time.

These are but a few techniques, some obvious and some not so obvious, that help make it possible to execute high-value high-complexity projects and programs via a distributed model. Look for future articles for more practical learnings from the trenches.

---

About Greg Reiser: Greg is a software development professional with 20+ years of experience as a developer, project manager and a consultant. He has experience in a wide range of industries including banking, insurance, publishing, logistics, healthcare and telecommunications. Greg has helped numerous enterprises deliver mission-critical solutions using advanced software development practices. He is currently focused on helping organizations get the most out of global development capabilities.

Ververs - IT's Leadership Opportunity

By Carl Ververs - 4 February 2009

In his latest alphaIT article, Ross Pettit writes about how true leadership is urgently needed in IT, now that business restructuring is becoming a means of survival for many companies.

An August 2008 study by InformationWeek revealed that executives outside of IT value leadership more than any other characteristic in CIOs. Having experience running a business unit outside of IT is considered very valuable as well. These two factors combined leave no room for interpretation: IT needs to stop thinking that just “managing the shop” suffices and realize that providing true leadership is critical if it has any aspirations of retaining relevance.

I posit that in many cases IT has been lacking true leadership for a very long time. For some strange reason, people who do one thing very well rise to the top of management ranks: somebody who was the first technologist in a firm, somebody who ran a certain project well, somebody who has technical knowledge and skills that were pertinent at some point. What is neglected is the individual’s ability to lead, adapt to changing business contexts and inspire others to do the same. It is assumed that success in one context – as a manager or developer in a specific project team – automatically translates into success in another.

It is hard to resist promoting people who are like us and who work like us, but in so doing we create a self-sustaining pattern of like traits to be promoted, warts and all. In evolution, this is a guaranteed path to extinction. Weak leaders promote other weak leaders. We may be surrounded by competent technologist or technology managers but we’ve shut out true leaders who have the courage and determination to not just shake the apple cart, but make something other than apple pie. Arguably, the people we need the most – the innovators – are completely blocked out.

The Financial Times recently ran an article about the animators at Disney. After Walt died, the remaining veterans who had worked directly with him, the so-called “nine old men” were promoted to run the studio. They were not exactly top-shelf animators, and it showed. The outstanding talent Disney had attracted over the years – among others an obscure guy named Tim Burton – walked out within a few years because they were managed badly.

In his blog, John Soat touches on the topic of hiring IT executives and quite a few people comment how it is only fair to promote from within. Luckily, several commentators have a level head and state that firm should hire and promote people who can effectively bring change.

Here is how this cycle can be broken. IT managers very often have a contentious relationship with the business. The business should seek out people who are not in senior management roles but who are very vocal and active in the company’s technology. Invite these individuals to partner with the business on pursuing the action items Pettit lists in aforementioned article and watch what unfolds.

To illustrate how effective this can be, let’s examine the case of a Chicago options trading firm. The business side was very unhappy with the technology leadership but could not find a way to improve on that. They hired a CIO who talked a good game, but did not have the mettle to change anything, either by motivation, coercion or force. Then the traders partnered with a few “vigilante technologists” who had been actively chipping away at the established IT order by introducing new technologies and innovative solutions and funded an innovation group. In short order, this group started to produce real revenue-generating solutions, introduced new ways of working and quickly overshadowed IT management in providing true leadership. Those who tried to interfere with this group’s progress were swiftly isolated and rendered harmless. This set the precedent that business-as-usual was over and IT as a whole was going to be held to a new, higher standard.

Yes, change is scary, especially if that change violates our core beliefs and assumptions. But with the current economic situation, companies don’t have a choice but to take decisive action. As Pettit’s, and related articles state, now is the time for companies to shake their IT department's apple cart a good bit, and shake the rotten apples out while they’re at it.

It’s hot in the kitchen. Time for the faint of heart to stand back.

---

About Carl Ververs:  Carl has been a business transformer through technology since the start of his career two decades ago. Always at the vanguard of new thinking and creative application of systems, he built CRM systems, used SOA and applied Agile techniques well before they were named.

Carl's technical expertise lies mainly in high-performance computing for derivatives trading and business process management. His background spans a wide spectrum, including business application specialist, hierarchical storage system architect, customer management systems designer, trading operations manager, Agile project Management coach, SOA practice lead, PMO/QA director and deputy CIO. Carl is an avid musician and composer, computer graphics artist and geopolitical pundit. He lives in Chicago with his wife and son.