Enterprise Service Bus

ESB – Acronym for Enterprise Service Bus



Middleware/integration backbone that ties applications, services, and resources together within a business area
Connects Web services, J2EE, .NET, B2B, Portals, and Legacy applications (i.e., databases, ERPs, CRM, programs, etc.)
Defines service mediation (i.e., intelligent message routing, transformation, and monitoring)
Allows publishing and discovering of services (WSDL/UDDI)
Enables the connection of software running in parallel on different platforms, written in different programming languages, and using different programming/data models
Provides service governance, security, and error handling.

ESB Architecture

 

 

Benefits of ESB

• Simplifies the integration process and the reuse of business components
• Adapts easily to new business requirements
• Easy to use, lower-cost, and standards-based integration
• Enables incremental application development and deployment, thus reducing risk and up-front investments
• Centrally managed

Typical Standards required to support

• WSDL, SOAP, UDDI, WS-Addressing, WS-ReliableMessage, WS-Security, WS-Policy
• XML, XML Schema, XSLT, XPath, and XQuery
• JMS, JCA, J2EE

Typical Capabilities of ESB

• Can input requests, services, and commands through client application or portal
• Can view/output responses, services, and commands through client or portal
• Can define business processes/services through Business Process Management (BPM)
• Can orchestrate business processes with Business Process Execution Language (BPEL)
• Can create/maintain/execute policies (i.e., governance) through service registry/repository
• Can have service and transport-level security
• Can develop/implement/test through test management tools
• Can integrate databases through Data Service Platform

Components around ESB

• ESB – Enterprise Service Bus
• Data Service Layer – Composite Data Service Platform
• Databases – Database systems to store data
• Test Manager – Testing tools
• Portal – any Web Portal

Features of ESB

Intelligent routing

• Message routing according to XQuery-based policies or call-outs to external Web services
• Support for both point-to-point and one-to-many routing scenarios to support request-response and publish-subscribe models
• Dynamic routing destinations that enable service destinations to be set dynamically in the proxy pipeline. External rules can be configured to feed the actual business service to be routed at run time

Message brokering

• Should support for multiple messaging models including synchronous, asynchronous, and publish-subscribe
• Should support for multiple message formats including SOAP, SOAP with attachments, XML, structured non-XML data, raw data, text, and email with attachments
• WS-I compliance for SOAP/HTTP messages

Message transformation

• Should validate incoming messages against schemas
• Dynamic service selection, based on message content or headers with the ability to transform messages based on the target service
• Message transformation based on XQuery or XSLT
• Multiple formats: XML and structured non-XML data

Service bus security

• Should support authentication, encryption and decryption, and digital signatures as defined in the Web service security (WS-Security) specification
• Should have provision to use SSL to support traditional transport-level security for HTTP and JMS transport protocols
• To support one-way and two-way certificate based authentication
• To support message-level security
• To support HTTP basic authentication

Service Level Agreement (SLA)

• Administrators should be able to configure SLAs on a variety of attributes including throughput times, processing volumes, success/failure ratios of messages processed, number of errors, security violations, and schema validation issues
• Administrators should be able to configure alerts for SLA rules violations and trigger automated actions, and send alerts to the console or email

Error handling

• Should allow you to configure your system to format and send error messages, and return messages for consumers of services who expect a synchronous response

Testing

• Should provide a built-in test console in the development environment
• Should allow you to test inline XQuery expressions used in the message flow

Logging and monitoring

• Should be able to gather statistics about message invocations, errors, performance characteristics, messages passed, SLA violations etc.

Disadvantages of using ESB to build a SOA

• Don’t use ESB-oriented architecture, if it does not have “business value”. SOA must be based on business requirements
• An ESB is a means to an end, not the end itself
• SOA approach consists of several components such as service producers, service consumers, service registries and repositories. These components must be defined and developed first, then integrated with an ESB
• Integrating ESB with other software may be difficult, particularly legacies
• Organization may have to develop new skills, techniques, and technologies

ESB Service Lifecycle

• Planning
• Define/Develop services
• Discover services
• Integrate services
• Orchestrate services
• Secure services
• Manage services
• Accessing services
• Analyze SOA/services and environment
• Monitor services

Planning

• Define the business value upfront (i.e. make a business case)
• Define the scope of SOA (i.e., business processes, number of services, organizations, external users, etc.)
• Establish boundaries and alignments with current and future business and IT initiatives
• Use a phased approach, starting with a single, low-risk application or a pilot program that spans business functions and eventually expand to an enterprise SOA
• Use architecture guidelines, requirements, and policies
• Define SOA standards
• Define roles and responsibilities
• Assess the maturity of the current and desired target state, followed by a roadmap for transformation toward SOA
• Define SOA tasks
• Plan task schedule and resources
• Start the governance process

Define / Develop Services

• Create service candidates by defining roles and collaboration as stated in business processes
• Identify candidates for potential business services from business model
• Potential business services should be reusable
• Design and build services that correspond to specific steps within a business process
• Identify services that can be combined into a composite service or application for carrying out a specific business function
• Use a Business Process Management (BPM) tool

Discover Services

• Define the service registry/directory
• Service directory acts as service broker between service provider and service consumer. It registers the services through its metadata or service contracts (i.e., it hold data about the services).
• Define service providers and consumers
• Service provider is a person, department, organization, or application that offers the service. It registers service contracts to the appropriate service –brokering nodes. Define Service Level Agreements/Contracts and metadata.
• Service consumer is a person, department, organization, or application that finds services with which its client would like to bind and interact. The service consumer may query service broker to find contracts and the metadata associated with services (i.e., service location, methods or remote services, message input/output formats, policies, and other parameters that a client uses to bind to a service)
• Service broker is an entity that facilitates registration, location, discovery, retrieval, synchronization, replication, and/or delivery of service

Integrate Services

• Integrate services with other services, applications or IT systems such as databases, data transformation services, reliable message delivery with routing, monitoring, and management (i.e., could be ESB and Service Management tools )
• Integration often requires transformation of data to map between different data schemas, as well as dynamic routing for connecting the appropriate services at run time. This can be done in a Data Service Platform (DSP).

Orchestrate Services

• Orchestration is the combining of services into seamless, reliable process flows
• It differs from integration, because orchestration is the sequencing of services to fulfill a business task or process
• Orchestration is also called “gluing” services together into flow logic
• Can be done in Business Process Execution Language (BPEL) and Business Process Management (BPM)

Secure Services

• Define service security policies
• Define user access in SOA environment (i.e., authorizing and authenticating users, as well as provisioning them and managing their identities) must be mapped out before potentially sensitive information is exposed as a service
• Identify services and data with security requirements
• Provide information security controls to security requirements
• May be implemented in the Service Registry and/or ESB
• ESB provides message and transport level security
• Web service security standards can be applied (i.e., WS-Security, SAML, SSL, XML-Signature, XML-Encryption, etc.)

Manage Services

• Define Service Level Agreements (SLA)s for services, and how to enforce them
• Create SOA operational policies for daily operations, maintenance, auditing and billing (if appropriate) of services
• Define metrics for service performance
• May be implemented and enforced in Service Registry and Service Management

Accessing Services

• Services are typically exposed to users through a portal or a composite Web application
• May also be exposed through wireless devices such as cell phones and handheld devices
• SOA environments support multi-channel access to services and enable organizations to adapt user interfaces without modifying underlying services

Analyze Services

• Periodically analyze services, events, and business processes involved in the business operations
• Define how operational managers and users can effectively monitor, analyze, and respond to time-sensitive issues
• Identify bottlenecks and other problems in the process and alert the appropriate personnel when a particular event warrants attention
• Analyze if all requirements are being met

Monitor Services

• Monitor the service execution and results
• Ensure that the services are reliable, available, and constantly monitored for exceptions or failures
• Services can be monitored through the ESB, and performance tested with a Service Test Manager

ESB Design – Best Practices

• ESB prototype should be done first
• ESB as an integration and SOA solution is still a new technology
• Getting trained and skilled people may be a challenge
• Integrating ESB with other software may be difficult
• Standards and best practices are still maturing
• Organization may have to develop new skills, techniques, and technologies
• Management must endorse and enforce governance from the beginning
• Security must be considered from the beginning (i.e., SOA Security, Web Service security, Defense-in-Depth, DoD compliance, etc.), and you can never start the C&A process too early
• Performance requirements (i.e., reliability, availability, scalability, transaction time, response time, etc.) should be defined and tested, performance may be an issue
• SOA may not be the answer for all legacy integrations
• Demonstrate the ESB capabilities to stakeholders
• Use vendor experts as much as possible to assist with development
• Provide training to users