|
From Resource, August 2007
Copyright by LOMA
Service Oriented
Architecture and the Legacy Organization
Done right, SOA can help your company reduce costs and improve
time-to-market. Done wrong, and it
can become another expense. At the
ACORD LOMA Insurance Systems Forum, David Koenig shared his experiences with
implementing SOA and how he learned to get it right.
By Tammy J. McInturff
Service Oriented Architecture (SOA)
offers companies a trifecta of more modular applications, reduced development
costs, and faster time-to-market. Recent advances are making it easier to
implement SOA across a broad range of systems. However, it is not a silver
bullet, especially for organizations that have large investments in older
technologies. At the ACORD LOMA Insurance Systems Forum, David Koenig,
president, Brookline Technology, discussed examples of SOA initiatives that
failed and how those experiences improved later iterations at companies that
were struggling to marry SOA with their legacy platforms.
Koenig
and Michael Galareau, director of architecture, personal market, Liberty Mutual
Group, have worked together for six years at companies including Liberty Mutual
Group and Travelers. Over that time they have implemented SOA in various formats
at those companies and have learned a great deal from those implantations. “We
learned from our mistakes and that allowed us to move the organization
forward,” Koenig said, adding that each implementation achieved moderate
success, yet under-delivered in ways that taught them where they needed to
change their strategy and expectations of SOA.
Today
the mere mention of SOA gets the attention of most IT managers. Koenig agreed
that there is a lot of hype surrounding SOA, but said that once you get beneath
the hype, “SOA is also a lot of good stuff that you can use in your
organization. He said that, “recent advances in some of the vendor products
are making it easier and easier to implement SOA across a broad range of legacy
systems.”
However,
Koenig cautioned that SOA is not a silver bullet, especially for those
organizations that have spent hundreds of millions of dollars in older
technologies. “SOA initiatives often fail in legacy organizations, not because
of poor design or lack of technical capabilities, but because of failure to
govern how services are built once the design is complete,” he said.
“Introducing services incrementally and learning from these less-than-ideal
experiences are the first steps toward making SOA work with existing legacy
systems.”
The
typical insurance organization is a patchwork quilt of different systems, built
at different times, using different technologies, often with no real thought to
how the systems are going to get integrated. Koenig and Galareau’s experiences
with implementing SOA at various organizations helped them develop a strategy
for introducing enterprise class service oriented architecture technologies into
the patchwork quilt that constitutes a typical operating environment for a lot
of organizations.
Getting
Beyond the Hype
According
to Koenig, vendors will argue that the combination of SOA and Web services is
very close to being the silver bullet that companies have been looking for to
finally realize IT’s long-promised potential and justify the expenses that
they have been putting into IT. “The business partners’ point of view on SOA
is a little more jaded than the vendors,” he said. “The business’
point-of-view is that it always seems that this year’s next big thing is just
a way for IT to get out of finishing last year’s next big thing. So when you
want to introduce it into your organization you have to understand that it is a
marriage of SOA to legacy. It can’t be rip and replace. It has to live
simultaneously for many years. Nobody is going to throw away stable systems that
work. SOA just helps you evolve things.”
“To
a legacy IT organization, SOA means constructing applications from components,
services and workflows,” Koenig added. “It means designing for
interoperability and reusability. SOA has a strong focus on standards and
governance. In other words SOA is really just good basic N-tier development with
an increase focus on interoperability and standards.”
Legacy Architecture and SOA
For the legacy organization SOA
provides structure and a set of tools for delivering a new platform in parallel
with current production systems. “Done right, the benefits of SOA are
enormous,” said Koenig. “SOA can help you achieve better business alignment
with more targeted functionality. It can also give you faster time to market,
without cutting corners. It is more flexible and configurable, to support
changing business needs. It also allows for more reuse, which can give you more
return on investment, as well as a more scalable, manageable and more
monitorable infrastructure. These are all benefits that the vendors are
preaching. Done wrong and SOA is just another example of where IT didn’t live
up to its promises to the business. Done wrong, and it becomes another set of
legacy code that you need to maintain and another set of vendor products that
you need to have within your system. It ends up increasing your ongoing lifetime
costs and doesn’t really add that much more to your business partners.”
SOA
Framework
SOA is an architectural
framework that enables applications to be built as a set of granular services
and to interact with one another in a loosely-coupled, platform- and
organization-independent manner. SOA facilitates reuse by allowing applications
running on one platform to consume resources running on other platforms. The
framework that Koenig and Galareau use to introduce SOA to an organization has
five parts. The five parts of the SOA stack include presentation services,
business services, data services, infrastructure and integration services.
According to Koenig, with the acceleration of SOA adoption, vendors are offering
better-and-better products at each tier of the platform. He said the key to
making SOA work is to respect “the stack.” Koenig cautioned, however that
even with industry-standard products, successful implementation is no slam-dunk.
“Presentation
services are audience driven, completely business focused, light weight, views
that should be configurable based on the audience as the business needs
change,” said Koenig. “The heart of SOA is the business services and these
are multiple levels of granularity, from core components up to meta-services
that are closely aligned with workflows and business rules. The data services
part of the SOA framework are services that can allow you to take the new stuff
that you build and plug them into the back-end where the real integrations
points have to occur. Integration services are messaging feeds of
transformation. The key to making SOA work is the interoperability and the
integration services allow you to pull all the data and slice it together or
allow you to plug SOA into other applications. Also, infrastructure, security,
monitoring, and deployments are all baked into the typical SOA product.”
Koenig
said that even though some vendors are better than others, there is far more
value in going with a single integrated vendor. “The benefit of going with a
single vendor far outweighs the risks of integrating best-in-class products from
different vendors,” he said. “We always say ‘good enough is good
enough.’ If you are an IBM shop, go with IBM; if you are a BEA shop, go with
BEA. The whole point of SOA is that it is interoperable and interchangeable.
However, you really don’t want to have vendors pointing the finger at each
other when something fails to work and you are not sure why. Every year the
standards get better but there are still problems with various implementations
and various vendor products. We have found that when we have two vendors they
typically point to each other when there is a problem. When we have only one
vendor with the stack they are far more involved with getting the problem
solved. More important than anything else, you absolutely don’t want to build
your own SOA. It seems easy when you begin but the cost of maintaining it far
outweighs any licensing cost you would save upfront.”
Portal
Technology
Within the stack there are
three sets of technologies that any SOA implementation needs to get going. These
technologies include a portal, business process management and an enterprise
service bus. Portal technology forms the core of the SOA presentation tier,
according to Koenig. “It provides the development framework infrastructure for
managing user interfaces across a broad range of Web sites, business
applications, databases, and other information resources,” he said. “At the
bottom of the stack you have all of your business services and data. From an SOA
point of view your old legacy content management system is just a source of
data. With portal technology everything in the organization becomes content,
whether it is business rules, additional content, third party services,
integration with the ODS, or integration with the data warehouse. The portal is
a tool to the services for creating unified views, regardless of the channel
that you deliver to and regardless of the customer base that you deliver to from
these disparate backend systems.”
Business
Process Management
The second core tool in the SOA
toolkit is BPM (business process management). BPM is by definition workflow but
Koenig said it is really a lot more than just workflow. BPM operates at the
Application Server level to deliver robust business services that chain together
individual components, back-end services, databases, legacy applications and
more—without the need to code workflow logic directly in Java or any other
language. “So in the past if you wrote object oriented programming you still
had to write JAVA or some other language to chain it together,” Koenig said.
“BPM allows you to do it though a business modeling language very similar to
how you would do it with a more traditional workflow system. BPM has the same
idea as the traditional workflow system but it takes it down to a services level
so you are changing other processes. It just allows you to reuse your services
and chain them together.”
Enterprise
Service Bus
The third technology tool in
the SOA stack is Enterprise Service Bus (ESB). Koenig advised companies to use
ESB to get control over data management. ESB is a protocol transformation
gateway for enabling standard interactions between business and data services.
“ESB is not a tool,” said Koenig. “It is a library of functions just like
a portal. It allows you to do process choreography, message routing, event
monitoring, and data security. ESB allows for a less-complex and more manageable
services architecture. It also gives you communication adaptors. It works with
your BPM systems and you can do your business modeling, process chorography, and
event logging. And built into it also is the ability to have security and other
things.”
Three
Learning Experiences
“If you don’t learn from
your mistakes how can you expect to move forward? How can expect to succeed,”
Koenig asked. Koenig and Galarneau learned a lot from the mistakes that were
made over a six year period of implementing SOA.
Koenig
described three examples of SOA implementations that had less than ideal
results. The first example was a Sales Compensation System, which he called
“SOA as implemented by mainframe folks.” “We went in and bought whole hog
in SOA,” he said. “But unfortunately we approached it from a classic rip and
replace. With this approach you may be trying to do the SOA architecture but
ultimately you are still designing monolithically. It was a very traditional
model in that we had an integrated front-end. In this model we had our agent
on-boarding in business services with licensing and territory management. We had
a compensation system with commissions and recovery in business services. We had
our agent information database and compensation database in data service. We
integrated with our upstream systems, our sales systems and human resources
systems. We integrated with our downstream systems—the financial systems and
the agency management system. It was a very traditional architecture and a very
traditional application approach.”
Koenig
said they learned a great deal from this experience. “We had well defined
workflows that led to quick user acceptance but because it was a classic
fully-integrated, rigid interface we actually found that a huge amount a
backlash came because it was seen as false configurability. We had a good
modular design from a business services point of view but because it was still a
monolithic, closed architecture the components were embedded with individual
applications. That monolithic business structure then led to isolated data. We
had our agent information separate from our compensation information. We had to
snap on a data warehouse to be able to get some of the data. We designed for
integration with up-and down-stream applications, which was good. However, with
the monolithic data and no ESB, we ended up coming up with a whole bunch of
point-to-point spaghetti code. We defeated a lot of the purpose. In the end, we
didn’t get a lot of the things that we wanted, but we did learn a lot about
Java programming. We were vendor independent. We focused on modularity but we
locked ourselves into more legacy type standards and we weren’t able to get
the reconfigurability benefit out of it.”
Back
Office Administration System
Their second experience with
SOA was with a back office administration system. This time they had a better
understanding of service and component design. They designed their work around
business processes but tried to do too much at once. Koenig called this example,
“indigestion from too much SOA at once.” He said, “We had our new business
platform, inforce, billing, claims, management reports and our Internet all
using an integrated set of interfaces to call shared backend services, quoting,
collections, payouts to call our various backend master data sources customer,
policy, billing and claims info, to use shared infrastructure and integrate with
all the systems possible,” he said. “This was a great idea but it was way
too much to swallow in one bite.”
“By
introducing portal to the presentation layer we fundamentally changed how we
think about design interfaces,” Koenig said. “But we home-brewed our own
portal and therefore we didn’t get any benefit.
Another unexpected downside was that we were too ambitious. As we exposed
more and more services to the portal presentation layer the business got very
enmeshed. It was actually quite a good relationship but the further you go to
the user interface the later you are binding your services up so that here we
had built something that the business users could configure almost on the fly
during the day. Somebody in the back office or the call center could change
things around depending on the needs of that particular day. The trouble is,
when you are binding so close to run time, it is almost impossible to test. So
there were just scenarios and defects that popped up unexpectedly months and
months after we had launched this because we just weren’t able to understand
and test it. From a business point of view we had strong business alignment but
the services were too granular, and too loosely defined.”
“On
the data level, we learned a lot from this experience. We focused on metadata.
We spread our information across many databases and we focused very much on NT
based connectivity so that we were able to separate the applications and reuse
them. We didn’t standardize our message formats, something we definitely
should have done. We swung the pendulum a little too far the other way. So
instead of having the monolithic data model, we now had two fragmented data
models. So in our first experience using SOA we had to snap on the data
warehouse because the data was too monolithic and we couldn’t combine it. This
time we had snap on a multimillion dollar warehouse because the data was too
fragmented and we’d gone too far the other way. From an infrastructure point
of view we baked monitoring and business intelligence into the infrastructure so
it was just part of the application and we didn’t have to do a lot of post
processes and that was a big benefit.”
Real-Time
Pricing Engine
Koenig’s third experience
implementing SOA was with a real-time pricing engine. This time they limited
their functionality only to that which warranted SOA and focused on integration
and data flow. He called this example “right architecture, wrong
governance.” In this implementation they were ruthless in their management of
scope. “This time we didn’t even have a presentation interface,” he said.
“We were building a real-time pricing engine and effective plugging with any
application that wanted to use it. Therefore, there was no need for a
presentation interface. We did the appropriate level of granularity focusing on
business processes. We had a real time pricing engine and behind the scenes we
had a more integrated database. We baked the data warehouse into our design and
we had all our upstream and downstream fee capability.”
In
this implementation they had the right architecture but ultimately they failed
to govern it. “We didn’t waste any effort on presentation services since we
were essentially working on back-end services,” Koenig said. “However, no
thought was put into how we were going to integrate with other presentation
services. With this implementation we had strong business alignment of services,
the right level of granularity and mixed technology (C, Java, COBOL) with
minimal rewrite. But on the other hand we had no industry framework and it was
not designed for reuse. It was still application focused. We came up with a
single database for customer and policy data, which was good. The problem was
that there is not perfect nirvana with the database. You can only hope to keep
ahead of the tide. We ended up with an overbuilt data model used by many
applications and not abstracted from an application tier. The data model became
almost impossible to manage as each different application got onboard and wanted
to use the master database.”
“Insurance
is not that complicated of a data model. It doesn’t have to be,” he added.
“We overcomplicated it because we used the data models to try to glue together
legacy systems. The actual data model itself doesn’t have to be that
complicated and once you get a complicated data model a lot of other things
start fall apart.”
Lessons
Learned
Koenig said that each
implementation was moderately successful and is still running in production now.
However, he also said that each implementation under-delivered in ways that
taught them how they needed to change their expectations and strategy. “We
identified six particular design issues,” he said. “First, we didn’t
engage the business early enough in the design. I can’t emphasize enough how
important it is for the business to be involved from day one from a design point
of view, not necessarily from an infrastructure point of view. Second, we were
slow to embrace industry standards. We did too much ourselves. We seemed to
forget the fact that we were an insurance company building insurance systems. We
shouldn’t have been building SOA infrastructures. Third, we had trouble
defining the right level of granularity for a given platform. The forth design
issue we learned was that not everything had to be a service, some could have
been just components. The fifth issue was that we focused too much on the
service and not the underlying components. You can go too far and expose
everything as a service but then it comes at a cost of inefficient components or
if you focus too much on a service and you don’t focus on the component you
don’t get the efficiencies. And the final design issue we identified was that
we confused ‘repurposing’ with ‘reuse.’ Everybody calls it reuse but you
will find more often than not it is not reuse it is repurposing. Reuse means
when you put a service out there, multiple services can call it without having
to modify that service. Repurposing means you take the code to that service
clone it, build another one very quickly but now you’ve got two to maintain.
Be careful if you are going to repurpose.”
Koenig
said they also identified five issues from a management point of view. First, he
said they implemented SOA in the wrong order. “It should have been more
revolutionary but we did it as rip and replace,” he said. “Second, we
abandoned governance when time-to-market became urgent. Third, we failed to
embrace open source management practices. We think this is very important. SOA
is all about spreading development. SOA is all about letting teams work
completely independently from one another but if there is one thing open source
teaches us it is let everybody work on their code separately. The fourth
management issue we learned is that we allowed business functionality to trump
architecture. We should have known better than to expose everything as a
service. We should have known that there was no way we could test services if we
bind them at the runtime level. We
could have delivered all the functionality our business partners asked for
without breaking the SOA structure but we didn’t. Finally, and most
importantly, we didn’t realize that people want SOA to be more than what it
is. It is just basic N-tier architecture with ever-improving standards. They
want it to be a silver bullet and we got sucked into the hype. We oversold it
and it came back to bite us when it failed to deliver. It isn’t going to
change the world.”
Recommendations
for Implementing SOA
Koenig offered some
recommendations that can help make your experience better if you are planning to
implement SOA in a predominately legacy organization. First and most importantly
he said companies should design a meta-architecture to guide SOA implementation
over time and across teams. “You’ve got to design business architecture with
the expectation that you will be supporting legacy code in parallel with SOA for
a long time,” he added. “You don’t have to do rip and replace and you
don’t have to go into it with a point of view that lets you get SOA as quickly
as you can.
Build
an inner architecture that marries all the pieces of your organization and build
as you go where you need it to.”
Koenig
also recommended that companies create a set of well-defined enterprise
standards and stick with them. He said SOA requires a small, but inviolate, set
of standards for all teams and all applications. “Development teams can have
flexibility on other aspects, as long as they respect the core standards,” he
added.
The
third recommendation was to not be cheap when building your go-forward
architecture. “Buy the best product, the one that you think fits your
organization the most,” he said. “Build the go-forward infrastructure in
parallel with the current operating environment. Don’t overbuild the
infrastructure; aim for scalability. Also, plan for a multi-year transition and
implement incrementally. Think integrated SOA from the beginning and then start
migrating the business services, one business process, one set of services and
one technology at a time.”
It is
also important to choose the right team to implement the SOA strategy.
“The
right team is 100 percent the most important thing that you must do,” Koenig
said. “You need people with strong business knowledge and respect for
standards. SOA is strategic, interesting and impactful. People should see it as
a privilege to be on the SOA team.”
Koenig’s
final recommendation was to cherry pick an initial set of services to be built
using SOA technology. “Use portal technology to build an enterprise
‘gateway.’ Use ESB to build a set of efficient data services and use BPM to
begin migrating high-priority business services to SOA,” he said.
The
“Go Forward” Infrastructure
Koenig advised companies to
build their ‘go forward’ platform infrastructure in parallel with their
legacy systems. “This is like building the new Yankee stadium in the lot next
to the old Yankee stadium,” he said. “You don’t tear down the old stadium
because it may work just fine. You build a new one in parallel and you slowly
start migrating over. The first thing you do is build your stack. Invest the
money in it. Then you start building your services, whether it is BPM or new
services. Then, start integrating those services wherever possible. There is no
reason not to reuse something just because it is written in COBOL. There is no
reason not to reuse something just because it is mainframe or vertically
aligned. Build it over time. You don’t shut down any of the old interfaces
unless you are done with it. In other words, implement a robust,
horizontally-scalable platform in parallel, and then migrate business processes,
services, technologies and applications one-by-one. The key is to make the
infrastructure enterprise class. You are starting with a clean slate, so don’t
skimp on getting it right.”
Advice
Koenig offered some advice to
companies that are considering implementing SOA. “Engage the business early
and often and always maintain a business process point-of-view when designing
services,” he advised. “Don’t skimp on infrastructure; build a robust
platform to handle growth. Go for horizontal scalability, respect ‘the
stack’ and spend the money to build a platform that includes security,
monitoring and environment management. Your total cost of ownership will be far
cheaper than your big upfront cost if you do it right. Your integration cost and
your issue resolution cost will be higher if you try to integrate multiple, not
fully compatible pieces.”
He
also advised companies to plan for heterogeneity of applications, technologies
and processes. “SOA is the glue to bring together disparate legacy systems,”
he said. If your legacy stuff works ‘live and let live’ or ‘cut bait and
migrate’ if it is just too much money to continue to maintain your legacy.
Sometimes it is worth the rewrite. Make that call one application, one service
at a time.”
Think
small and incremental when implementing SOA. “SOA is not a big bang. Don’t
go for a big bang or you will get indigestion from too much SOA at once.”
“In
our opinion, success is business focus plus governance,” Koenig said. “Stay
the course, maintain your standards, and work through all the political and
organizational design issues. If you don’t, SOA will just become another
competing legacy technology in your organization and you will fail to get the
benefits. There is no perfect time to start SOA and there is no perfect design.
The best thing that you can do is jump in with both feet. It is a discipline
that you add to your organization and good enough is good enough.”
n
SIDEBAR
Distribution
Technology and Emerging Technology Conference Preview
and Call For Speakers
Recruit your insurance and
financial services clients to speak at LOMA’s Distribution Technology and
Emerging Technology Conferences,
January 28 —February 1, 2008 at the Hyatt Regency Tampa Downtown in
Tampa
,
Florida
. The conferences grow each year and this year we expect more than 250 key
decision makers and influencers to attend. The conferences offer an intimate
setting for developing significant long-term relationships with industry
veterans and experts.
At the
LOMA Distribution Technology Conference, explore new ways to increase your
company’s bottom line, attend the number one distribution event in the
industry and learn how to develop effective distribution technology strategies.
Discover how to make the Internet a key priority for channel investment. Listen
to how others create integrated service networks—the right way. Attendees will
also have the opportunity to speak peer-to-peer about solutions, trouble spots
and success stories in interactive sessions.
At the
LOMA Emerging Technology Conference, get a firm grip on which emerging
technologies are powerful—not just popular. Learn to create secure technology
platforms for e-business. Discover how to improve process efficiency and
increase customer service levels. Find out how insurers are outperforming others
by using the latest technology
Sponsorship
and exhibiting opportunities are also available on a first-come, first-served
basis. From receptions to networking
breaks to excursions, LOMA has opportunities that will put your company in front
of the customers you are targeting, provide excellent visibility and drive
qualified traffic to your booth.
The
deadline for speaker proposals is August 17, 2007. You can access the Call for
Speakers form at ftp://www.loma.org/DTET08_CallforSpeakers.pdf or http://www.loma.org/WinterTechConferences.asp.
For more information, contact
770-984-3733 or infoman@loma.org.
|
