Home > Enterprise Service Bus, Integration Technlogies > Evolution of Open-Source Enterprise Service Bus (ESB)

Evolution of Open-Source Enterprise Service Bus (ESB)


Evolution of open-source Enterprise Service Bus (ESB):


Developers involved in complex system integration will know how intriguing system integration can get to. Integration of disparate enterprise applications is always challenging because of the need to get them work together. It also has become an increasingly essential element of IT, where we oftenly come across terms such as BI (Business Integration), B2B (Business to Business) etc. A decade earlier, technology choices were limited. Integration of enterprise applications had to chose either an application server or a heavyweight enterprise application integration (EAI) solution that required a huge upfront investment in infrastructure, money, and human resources. There was a need for a lightweight solution which is easy to deploy and manage. Many enterprises at that time, built thier own abstractions on top of thier messaging servers. By doing so, it was an extreme undertaking and a huge burden on the developers.

Since IBM first released MQSeries, enterprises have been sold on the benefits of decoupling systems using point-to-point message queues. When TIBCO brought Rendezvous to the market, it expanded the horizons of messaging by introducing the publish-subscribe model. The Java Message Service (JMS)—born through Sun’s Java Community Process (JCP)—set out to unify the point-to-point and publish-subscribe messaging models. It wasn’t long before enterprises required more than just messaging, they also needed a way to orchestrate messages between systems and perform transformations.

To address this need, major vendors such as IBM, Oracle, and Microsoft built EAI brokers that added message brokering and centralized transformation engines on top of their existing messaging servers. The problem with the EAI approach was that it adopted a hub-and-spoke architecture where all data had to flow through the EAI broker. Although this worked for many applications, enterprises were soon pushing the boundaries of these systems and it became clear that something more flexible, scalable, and distributed was required. Enterprises needed connectivity, transaction management, security, and message routing, and they needed to host services that operated on data moving around their systems.

Working on integration projects used to mean working with enterprise application integration (EAI) products, each of which implemented its own stack of tools with proprietary technology. To switch from one EAI product to another meant learning the proprietary technology and toolset from that new product. With more focus on open standards that emerged in the integration market, the market changed from EAI to service oriented architecture (SOA) and enterprise service bus (ESB) products. Examples of these open standards are Java Message Service (JMS), SOAP, XML, and WS*. With open standards available, more and more open source projects began to implement these specifications.  There was a need for an open source solution built on open standards.

The next step of this evolution led to what is now known as an ESB: Data and exchanges are conveyed from system to system in a single logical bus, decoupling all the systems from each other. This leads to a much more maintainable system and can save a lot of time in the long term. Integration technologies are becoming commodity software, and the rise of open source integration frameworks is hence becoming increasingly important. Open source is now unavoidable, from JMS brokers to SOAP stacks to ESBs. Companies generally use ESBs to convey sensitive data, and they sometimes need advice when they’re developing the applications hosted in the ESBs or when they’re putting these applications in production. The industry needed the ESB. 

With the advance of service-oriented architecture (SOA) in the last several years, there has been confusion unfold about what SOA really is. One affirmation that has resonated well in the industry is that the ESB is the foundation of SOA. The term SOA covers a much broader topic than just technology. ESBs realize the benefits of what service orientation has to offer. The ESB bridges the gap between old and new. It acts as a mediator between application integration and service orientation, enabling enterprises to build new applications and processes from existing technology.

The notion of an ESB involves a common messaging bus which is used to communicate between different enterprise systems. Typically, there is a shared messaging format on the bus. The adapters between the bus and back-end applications translate data from the proprietary data formats to the shared message bus format. The power of this model is that applications can share information while being totally decoupled from one another. Additionally, the ESB provides a common platform for handling security, transformations, message routing, transactions, and monitoring.

Enterprise service bus is a loosely defined term, and lots of products are claiming to implement the ESB concept. The goal of an ESB is to provide a distributed and reliable mediation framework that different systems in an IT environment can use to communicate, thus removing the need for a given system to know how to talk to the others in a specific way. Integration, which is what ESBs are about, is complicated: Each time you add a system, it needs to talk to all the other systems, and ad-hoc integration between the systems has long been considered a bad solution.

The open source model is perfect for integration and henceforth for ESB. After all, ESBs are about surfacing, manipulating, and moving data between applications. Application integration is complex on many levels. The permutations of applications, protocols, message formats, environment restrictions cause system integrators an unholy amount of grief. No single vendor of an EAI can realistically claim to understand the intricacies of your application environment, so all the proprietary vendors have instead chosen to offer products that make their customers abide by their rules when building an ESB solution. This quickly becomes a problem when the vendor’s approach doesn’t suit your needs. Sometimes, you want to get into the code and customize for the problem at hand.

By providing access to the source code, open source ESBs can alleviate these problems—but the benefits don’t stop there. By nature, open source projects are developed to be lean and modular. Developers working on an open source project don’t have time to maintain a huge product, and they would want the code base to be accessible to their community to encourage contributions. Given that the use cases for ESBs and integration are so varied, it’s impossible to have one vendor test every scenario. It’s far better to give the code to a community of active users who provide testing and feedback about their experience with the ESB. That same user community has connectivity and message format requirements far beyond what can be provided by a single vendor, so it makes sense to enable the users to build support for the more exotic protocols. These extensions can then be committed back to the open source project for the benefit of the rest of the community.

Basing on these fundamentals, the Mule project was founded in 2003, and after that many open source ESBs have emerged. Each has its own way of doing things, and each focuses on different areas. As examples, Apache ServiceMix is built on Java Business Integration (JBI), whereas Apache Synapse is built around the Web Services (WS) standards, and Apache Tuscany uses the Service Component Architecture (SCA) standard. Mule takes a pragmatic approach by embracing these standards without forcing the use of any of them, giving the user a great deal of flexibility when building an ESB solution.

In the recent years, Mule and Apache ServiceMix have proven to be mature and have good communities backed by vendors that can provide the needed support. Mule and ServiceMix have a lot of differences, but they also have common ground that’s focused on integration functionality. Mule was one of the first projects that provided a large set of integration functionality. After a while, it was called an ESB. Mule focuses on ease of development with support for all kinds of payloads, and it uses an architecture and design model, which isn’t based on a specification. ServiceMix implements the JBI specification and therefore focuses on XML payload and the implementation of binding components and service engines. ServiceMix implements the Java Business Integration (JBI) specification. JBI defines a framework to connect components and make them talk together in a standard way. In the JBI world, components can host business logic (a BPEL engine or a rules engine) or handle a particular protocol (HTTP, JMS, and so on). The key benefit of JBI is that new components can be wired easily on the bus as they all abide by the JBI specification. For example, ServiceMix doesn’t provide a BPEL engine by itself, instead you can plug in any JBI-compliant component for BPEL (ex: Apache Ode). The JBI specification though doesn’t target end users, but rather is intended to be implemented by ESB vendors and other software vendors who want to integrate their products in a JBI-compliant environment, such as a BPEL engine, a business rules engine, a transformation engine, or another specific technology. This is the key benefit of JBI.

Choice is usually a good thing for consumers, but with so many choices, it can be difficult to navigate the landscape of open source ESBs. With the wealth of emerging robust and reputed open-source ESB soultions, it can rather be a difficult question of which ESB solution to opt. But it can be narrowed down by asking few questions. All of them aim at a common goal but support different means to implement them. So the question should be on which means you would like to implement an ESB solution (Java Business Integration standards, Webservices standards, Service Component Architecture etc).


Source: Open source ESB in Action. 


  1. Jodimilade
    April 8, 2010 at 6:00 pm

    There are a lot of great points here.

  2. April 8, 2010 at 5:57 pm

    I found that when I wanted more information, and the links provided me with what I was looking for.

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: