Orchestration - It Should be the Focus of Most IT Organizations

You've got the best Information Technology (IT) department in the world, your organization develops and cranks out new applications and capabilities to your customers both internal and external like clockwork – you are the Chief Information Officer (CIO) of the century. However, one day you wake-up and find the IT world has actually passed by you and your company. How did that happen? Just yesterday it appeared that all was well.

Well, several things happened. Your crack team:

1. Loves to run systems
2. Loves to buy hardware
3. Loves to interact with vendors
4. Loves to develop code
5. Staffs for each new requirement 
6. Staffs to maintain each new application

All this sounds good, so what is the problem?

This is a team that hugs its environment. They want to be able to see their babies, listen to the noise of the fans, and watch the blinkin’ lights. This may also be shared by executives who want to see where their money is going. High self-esteem for the team is generated from having vendors come in and parade their wares, and doing show-and-tells of the latest feature or cool system configuration trick to corporate leadership.

In addition, the organization likes to grow. Success for many is measured by the size of the organization. So, for each new business demand, the size of development team increases, the size of the IT infrastructure increases, and the operations staff to keep it all working increases.

These two elements combined to cause the IT organization to stagnate. The organization grew too large and the culture becomes focused inwards on the tools and systems developed. Your competitors become more nimble and out execute your organization. As CIO, you see this coming, and want to take action to fix this (just as your CEO is starting to get on your tush).

There are several options you can take, but you really want to understand the root cause of the problem. To get a bearing, you engage some excellent consulting firms, and they tell you all about the best practices that will enable a technical course correction. You put the plan into action. You get your IT team trained based on consultants’ recommendations and you set expectations and goals. You wait for the changes. However, the changes do not come. What was the missing ingredient?

The problem is that the change necessary was not directly technical, but related to the overall manner in which your organization is run. The change is not technical but directional. The change is from engineering to one of “Orchestration”.

Orchestration means that you need to change the culture of an organization that loves engineering IT solutions. The culture needs to change based on new thinking and new goals as the current culture which hugs things must now embrace software and system capabilities that exist outside of their direct control (paraphrasing Obi-Wan: “use the Cloud…Luke”). For customized applications, a new development approach, based on commercial tools environment needs to be created that enables targeted outsourcing of development. The culture must understand what must be done internally, and what can be competed and placed into the hands of other companies. Engineering and development is only one of the dimensions, the other is to ensure that operations is also uses an orchestration model, tailoring its support to the specific items of high-value to your organization, and not to areas where benefits of scale are best served by another provider.

The is the approach that Netflix has taken in choosing not to continue to own and expand their own data centers, but to outsource virtually all of their IT system growth using Amazon Web Services. The basic issue was that the cost of scaling their own environment could never keep up with the cost to performance capability provided by an outsourced Cloud Services provider. In addition, it enables Netflix to focus on its real capabilities – obtaining content and differentiating their services in a competitive marketplace.

Some actionable points for an “Orchestrator CIO”:

1. It’s the Job. Get the team together and tell them that they love their job and not machines and software
2. Benchmark. More than determining comparative costs of different internal IT approaches, you need to benchmark against costs of completely alternative implementation approaches
3. Move Budget. It’s difficult to affect change if the engineering organizations control their budget and continue to do what’s comfortable. Move the money and move the organization
4. Reward. It cannot be all punishment (moving budget, benchmarking that the current approach is not cost effective, etc.). There must be a method to reward innovation and risk taking
5. Never Rest. Practice Constructive Non-Complacency

The Technology Yellow Brick Roadmap

Enterprise Information Technology (IT) organizations seem to have a deep desire to create Technology and Architecture Roadmaps. The rational of creating these roadmaps is to help ready an organization for known and potential changes in technology as well as identify gaps in the ability to meet future requirements.

Unfortunately, too often the rush is to understand the elements of technology, such as the latest speed and number of cores for a processor in a blade center, and not how these technologies solve business problems. The difference is that the first are Engineering or Technology Capabilities, and the latter are actual Services.

Commercial IT service providers, such as the major telecommunications service providers, understand the linkage between technology and business implicitly and use several basic questions that lay the foundation of the purpose of a Technology Roadmap:

1. What are the IT services that my customers need?
2. What are the potential technical solutions, and what are the trades in cost, performance, delivery, and ultimately customer satisfaction?
3. What is the Concept of Operations (CONOPS) for how the IT service is going to be delivered to my customers?
4. Is my organization and processes mature enough to bring technology and the CONOPs together?

A commercial provider examines the first question and begins to decompose from the customer need into functional areas that combined together make the complete service. Contrast this against many IT organizations, which may be lead by engineers who are not directly in touch with their customers and make technology and service decisions that relate to their parochial understanding. This missing piece here is a close understanding of the customer, and not anecdotal information that may lead to decisions that are not based on real requirements or needs.

The second and third items are the critical transition from an engineering solution to a service solution. Engineering organizations are generally concerned with whether a service is going to work, the details of the component selection, and a deployment. They generally hold in lower regard the operations staff in an organization. The thought process of the staff sees the complexity of the developed solution as too complicated for anyone but a certified engineering genius to operate. Well balanced IT organizations know that the ability to ensure high-quality, repeatable service delivery is the goal – not just the excitement of the initial deployment of a technology or capability. The balance between engineering and operations must be found. That is, developing a forcing function that identifies how a new or enhanced service is delivered as well as the overall framework for all services to be delivered.

Giving it up – it is painful to stop what you are doing and rely on others. However, this is the key to the last item. The mature organization, like leadership itself, understands that the various divisions (e.g., engineering and operations) take leads at different times. The engineering staff will want to continue to “hug” the technical baby. The operations staff will tend to want to respond only to monitoring services and performing basic break fix, all too happy to leave the heavy lifting of configuring and provisioning services to the engineering staff.

So, how do you make sure that an IT organization is balanced (that is find the forcing function) and enables the development of technology, the creation of the service CONOPs, and the proper transition of these elements into a production environment? The simple solution is to find the junction between customer needs, engineering, and operations and to then assign the responsibility of this position in the middle, to a product organization that lives only thought the ability to ensure the mutual satisfaction of the other three elements. Mature service providers, such as large multi-service telecommunications providers, have well established product managers.

These managers are the fulcrum, the center of the development of an effective technology and architecture roadmap. The manager’s role is traditional leadership as in many cases the person may not be the position leader of all the people involved. Leadership is needed to provide a vision to the key individuals of the other organizations, and provide the environment for collaboration to develop effective and actionable roadmaps. The product leadership needs to direct, although not specify, all aspects of the roadmap activity:

1. Development of the Architecture Roadmap, with the milestones of technology, services, and operations ensuring the collaboration of the engineering and operations groups
2. Development of the business case, ensuring the appropriate inputs from the various organizations, including executives and finance
3. Drive the endorsement and executive buy-in of the roadmap
4. Development of a top-level actionable plan for the implementation of the roadmap
5. Supervision of the implementation plan, with the responsibility to identify the critical milestones needed across all organizations
6. Tracking of service execution, delivery, projections, and customer satisfaction with the responsibility to coordinate resources, if necessary, to address growth and service issues
7. Periodic updates of the roadmap for service improvement as well as service retirement

By following this approach, the services developed will have the right balance, ensuring that customer satisfaction is measured and tracked, that operations measures the performance of the services delivered. As important, is ensures that engineering is available for escalations necessary for service quality or delivery issues, and is able to spend time on implementing the next steps of the roadmap.

Thus, the Technology and Architecture Roadmap no longer lives in un-actionable isolation – a dream only invented to meet some corporate feel-good requirement. It provides, when combined with the proper organizational, services concept of operations, and product leadership, clear direction and participation of each functional area with the result being services that can be consistently and efficiently delivered with high customer satisfaction.