SOA and Governance -

I've been working on the definition and implementation of a SOA governance as of late - and when you get right down to it, there isn't much new under the sun. That shouldn't be surprising, given that service oriented architecture is applying the concepts of commerce to technology. Here's my current thoughts (:

SOA Governance:

• Define and execute the organizational policy and processes required to achieve business success of an SOA within SOA and IT processes.
• Establish chains of responsibility, measurement, policy, standards and control mechanisms to exercise decision rights as part of the service life-cycle.

How does that become useful to a project manager that wants to know what to do - and how to create a consistent approach? Well, much of the work has already been done because SOA Governance concepts derive directly from traditional IT/Corporate governance processes and map into the SOA-RM. Now, the application of these processes may have a bit different flavor that traditional IT systems - because system boundaries are much more fluid and the emphasis is on dynamic interaction - rather than rigid definitions. So here's a working draft of the questions that a SOA governance process will address for a project manager.

Dynamics of Services –
Visibility:

• How will others know that a service is offered?
• What services are offered by others?
• Who is authorized to offer services?

Interacting with Services:

• What is the supported 'lifecycle' for services?
• What metrics will be collected on the –ilities of a service?
• Where will they be made accessible?
• What levels of quality are required?
• What are the requirements imposed on a service consumer?

Real World Effect: [I’m still working on this one]

About Services –
Service Description:

• What elements must be contained in a service description?
• What elements are change-controlled?
• What is the process for CM?
• What events require a new version of a service description be published?
• What is the sustainment of older versions?

Policy:

• At what stage of development is an SLA authored?
• Where is it stored?
• How will performance against the SLA be measured?
• Who is responsible for measuring the quality of service?

Execution Context:

• What are the key drivers for the initiative (e.g. reuse as much infrastructure as possible a la REST or build a WS-* infrastructure)?

*You'll notice that there are two distinct types of viewpoints embedded within this questions: CIO and Project Manager. The responsibilities and activities will vary significantly between these two viewpoints. However, they coalesce around several basic questions that must be addressed in any transition to a service-oriented environment.

Art, Science and Adaptability....

Without discussing the morality of war, the concept of war bears many parallels to the concept of commerce. As the military seeks to learn lessons from private industry, remember that business tries to learn from great generals and strategists. Case in point, I'm reading 'The Art of War for Managers'. Therefore, it makes sense that there is much to learn on successful SOA from our military successes - and failures....I'll continue my train of thought on Lt. Gen Van Riper and Service Oriented Architecture (SOA).

War is about adapting. Any potential enemy as well as we, the United States, if we didn't adapt, learn, and evolve from our past experiences, we would be a species or a nation that would not survive. And any enemy that wants to survive against the United States can't fight like some of our recent enemies have, or they won't survive.

So what the heck does it mean to adapt through services? Well, one think that I think that it isn't is creating an entire system through traditional development lifecycles - including infrastructure, expectations, etc. Often, the the world is discussed in terms of 'phenomenon.' That is, when you put a whole bunch of things together that act independently, usually in their own best interests - and then let them interact - things don't always go as predicted....In fact, a 'phenomenon' often occurs. Sorta similar to how disruptive innovations that change some basic assumption previously held.

I'm trying to think of a good example and the best that comes to mind is human flight. As long as we tried to emulate the flight of a bird, no such luck. As soon as we adapted (within the rules of aerodynamics, etc), we started to make progress toward today's jet engines and beyond. This type of progress didn't happen by defining everything from the beginning and codifying all the assumptions.

Scientific rules became the interface into the world. As more was learned, those interfaces adapted...and the implementations, interactions and choreography based on those interactions adapted accordingly....but WITHIN their own time (hence acting independently).

Where am I going? Well, I hear alot talk and pitches about centralization and single point of whatever as part of your and I'm not really on the bandwagon that centralization is necessary. In fact, I will go further and state that centralization often leads to the desire for rigorous control over all aspects of the behavior that occurs through a system - which is a recipe for failure. Independent action leaks out all over the place. In technology, we call it a 'one-off' or 'point to point integration' or spaghetti code (well, not the only reason for spaghetti code, but I digress).

The bottom line is that we have to account for adaptability and one was to do that is to encourage and assume independent interaction (lots of reference here to other chapters in the books The wisdom of the crowd).

Whew...I never even got to my comments on a recent Zapthink Take....

The immutable nature of....

I returned to work today from a six week leave of absence. The first week I spent in the hospital, recovering from pneumonia (I hope we never lose the drug barrier). That left me with a bunch of 'reading days.' I really enjoyed The Tipping Point by Malcolm Gladwell, so I asked my brother to bring his copy of Gladwell's latest book, Blink. There, among chapters on thin slicing, Cook County General, and Paul Ekman,there was a chaper that addressed the concept of net centric war-fare.

I found the discussion of the Milennium Challenge 2002 and Lt. Gen. Paul Van Riper quite fascinating. His approach to the challenge was, in many ways contradictory to the principles of the Joint Vision 2020 - which many people seem to cite as the underpinning of a SOA within the US DoD. I dug a little more and found an interview that Lt. Gen. Van Riper gave to NOVA. In this interview, Van Riper discusses the danger of placing too much faith in technology at the expense of a deeper understanding of the nature of war....

'We hear many terms, whether it's "transformation," "military technical revolution," "revolution of military affairs," all indicating something revolutionary has happened that's going to change warfare. Nothing has happened that's going to change the fundamental elements of war. The nature of war is immutable, though the character and form will change. The difficulty is that those who put forth this argument believe that something fundamentally has changed, and you can change very quickly without thinking your way through it. They want to apply the technology without the brainpower.'

We hear much ado from industry about how SOA will transform the way business happens. Sure much of this is really marketing-speak. (I recognize that promotion plays an important part of commerce, so this isn't a bashing on marketing). However, there is a certain belief that technology - particularly the latest and greatest - will transform/create a competitive edge/etc. So I started thinking, what can we learn from these observations about the fundamentally unchanging nature of war?

Basically, for me, it is intuitively obvious that SOA technologies are about changing the character and form of business - but the nature of business itself doesn't change. We specialize our capabilities. We find ways to add-value. We bring together supplier and demanders. We prove or disprove economic theories. Much like the nature of war itself doesn't change, the nature of business doesn't change either....therefore, SOA is about the form of business when effected through IT. Stated more simply, SOA is how manifest the business through supporting information technology.

So what else can we learn? The interview continues with "The first thing you have to understand is how you plan to fight in the future or in a particular engagement, a particular war. And once you understand how you're going to fight, then you bring the technology to it. If you lead with the technology, I think you're bound to make mistakes."

Great parallels! The application of SOA to a problem starts with the business. Here as well, we can learn from the seemingly boundless supply of military history and strategy. With a pure technology approach, we're bound to make mistakes....There isn't much of a bigger endorsement that I could imagine for a holistic approach to SOA.

When do Web Services Become a SOA?

There has been a long discussion on the Federal CIO Council's SOA Community of Practice around SOA and Knowledge Management and how to actually put together a meaningful SOA demonstration. One of the (many) valuable pieces that is emerging from this discussion is:
How to differentiate between Web Services as yet another point to point integration tool/pattern and Web Services as a realization of SOA.

First, we're recognizing that SOA must ultimately address the challenges of information sharing. Ultimately, the technical dimensions of the problem are the easy ones. The harder ones (even) to understand are not technological at all. Rather, they are the very same challenges that must be faced every day, in every line of business. IT, in general, and SOA technologies, specific to this context, are no exception.

The challenges are:
COMMUNITY
*Dedication throughout a community to overcome challenges to information sharing.

GOVERNANCE
*Agreements upheld by all stakeholders to identify information and capabilities to share (exchange).

PRIVACY
*Sharing and dissemination protocols consistent with privacy laws and regulations impacting all participating agencies.

CULTURE
*Overcoming community and personnel concerns that prevent effective sharing.

TECHNOLOGY
*Leveraging existing technology to integrate knowledge, consistent with the established rules of governance.

LEGAL
*Navigate the various laws and regulations impacting dissemination of sensitive and/or case-specific information.

I would suggest that any services (thus Web Services also) expressed as part of SOA must exhibit several dimensions as part of a litmus test for determining when a collection of web services evolves from a collection to SOA.

The proposal currently on the table from Paul Prueitt (see the mailing list):

Web-services (expressed with a SOA) must have the following dimensions
1) re-use that is measured against community transparent utility functions
2) agility measured as the ability to respond in novel circumstances, and to novel requests
3) governance that is open to inspection from stakeholders
4) commonality within a community or community of communities
5) competency that is measured at several levels including competency

Too often we see existing systems or information being “service enabled” simply by generating WSDL from the existing code used to gain access to the information. One of the key challenges is to get folks to understand why that type of step != SOA. As a first step, perhaps a demonstration that web services which are in fact a realization of the technical aspects of SOA must exhibit these dimensions BECAUSE they address the challenges set out above:
COMMUNITY
*Dimension 4

GOVERNANCE
*Dimension 3

PRIVACY
*Dimension 5

CULTURE
*Dimension 4,5

TECHNOLOGY
*Dimension 1,2

LEGAL
*Dimension 3,5

Globalization?

Here’s what’s fascinating to me – within relatively short order, the discussion of SOA and a particular demonstration of the applicability of SOA concepts to this context is not primarily focused on the technology aspects of the challenge. The same “lessons learned” will eventually be recognized in technology evolution, much as they are in cultural and societal evolution. While listening to an NPR newscast on globalization, I started thinking about how the characteristics of economic globalization may in fact also emerge in a truly service oriented technology system. Where do incentives come into play? What about middlemen, traders, brokers?

I posit that many, if not all, of the evolutionary patterns of the trading and economic system can be abstrated into the context of SOA.

Commentary on SOA Standards

David Sprott (CBDi Forum) recently published commentary "On SOA Standards". I believe you need to be a member to access the full text of the editorial.

However,a couple of very relevant excerpts:
“...SOA is very different to most of the IT trends that come sweeping through our industry from time to time. First as an architectural concept SOA is not primarily technology driven, rather it is technology enabled. Second SOA adoption is inevitably situation specific because the needs of each enterprise are driven by the existing and future business and technology situation. Third the real impact of SOA will only be realized by most enterprises when a critical mass of content as reusable services is available either from internal sources or from enterprise application vendors such as SAP and Oracle.”

“The core principle of SOA is about standardization and reuse of common functionality in many different contexts, in a structure that is architected and engineered to be evolutionary and to have a managed level of adaptability.”

“In a very short time, probably less than one year...vendors will find a new focus for their marketing dollars. But in the real world enterprises, systems integrators and developers will be embracing SOA practices and techniques and building them into their kit bag because it makes sense.
• Architects and developers will use concepts of contractual separation and reuse because it makes sense.
• Enterprises and their suppliers will embrace SOA, progressively solving critical business problems in a better way than they might have done previously:
• Introducing core business services to enable consistent data and business policy,
• Using process services to separate the process and application layers, and
• Creating utility services to deliver very high levels of reuse across the enterprise.”

We're a long way from the days of the Captain Picard in the Starship Enterprise simply telling the computer "Access data banks" to get a perfectly relevant response. However, there is growing recognition that standardization, re-use, and business needs should drive technology rather than hype driving technology. As we continue to head in that direction, long after the the latest and greatest of SOA products are phased out of technology petting zoos and into technology archives, we'll get closer and closer to the ultimate objectives of SOA.

There's nothing new under the sun...but the rate at which we re-discover and discard things is changing.

I was reading InfoWorld from January 9 and its discussion on High Performance Computing. One of the articles was about "Shipping company Con-Way [that] began its SOA journey eight years ago, providing one illustration of how the new architecture works for the long haul." One of the quotes from the article is that "the development team has to be very close to business." It reminded me of a discussion that I had about two weeks ago with one of my peers about how SOA applies the lessons of commerce to the delivery of IT.

What struck me was how quickly our change cycle is shortening. The idea that eight years is now the long haul is rather startling when you think about it. How long did it take for the computer itself to become integrated into the way we do business? How long did electricity (or any public utility) take, for that matter? Moreover, how long did it take for these things to be standardized.

Check out the "War of the Currents". It was only in 2005 that ConEd finally discontinued support for DC for its customer base in Manhattan that required DC for existing infrastructure like elevators.

What does that say about lifecycle and backwards compatibility? Certainly that support phase is much longer than most sunsetting technology products today.