Tuesday, September 2, 2008

Dynamic components in Mule

Imagine that you are given to implement the following scenario:

  • A system is composed of a number of uniform entities, each entity publishes its status on a separate pub-sub topic and accepting instructions over separate control queue.
  • Your application receives a stream of instructions over a control channel, instructing it to monitor and control or stop controlling certain entity.

or you receive a stream of active orders and want to subscribe to the right marketdata to calculate a relative performance benchmark, or...

Usually I write about how to implement stuff with Mule, but this time I'm going to write why Mule might not be your best choice if your use-case is anything like above.

The most straight forward approach for implementing the system above is to create a simple POJO, encapsulating the management algorithm and have a service listen to the control channel and instantiate/dispose a new component for each managed entity.

The problem is that in Mule the components are somewhat heavyweight and it's not so nice when you start having hundreds (approaching thousands) of them. Another problem is that the code for manually creating and registering a componment along with the whole paraphernalia of routers, endpoints, filters and transformers is quite verbose.

The latter problem is easier to solve. There are a couple of JIRA issues that aim to simplify the programmatic configuration (MULE-2228, MULE-3495, MULE-3495). The Annotations project at Muleforge is also worth watching (right now there's no documentation, so you'd need to dig in Subversion).

The problem with the too many components is more fundamental. The main application domain for Mule is service integration. Though it also happens to be a fairly good application platform, it does fall short when it comes to managing a lot of components. One of the important things missing is the ability to create groups of components with shared thread pools. Another missing thing is better support for variable subscriptions (provide an API to easily add and remove subscriptions to the same component using some kind of template). Another notable improvement would be to elaborate the models into full blown scope-controllers, allowing to create composite applications with proper interfaces, routing policies, etc. All of these are addressable, but there is a long way to go.

There are also some improvements that can be done to the JMX agent - use JMX relation service to connect component to its endpoint and statistics, all components of certain type, user defined groups of components. Allow the component to exercise some control over the Mule-generated ObjectName so users can organize the different components in groups and ease the monitoring by EMS tools by allowing to query the parameters from a specific group.

Please share if you've had similar issues, how you dealt with them and whether you think {with the benefit of a hindsight) that the solution was worth the effort. Also I'd like to hear from users/developers of other integration frameworks or composite application containers (Newton?) how they handle these problems.


Ross Mason said...

Excuse the very late response on this post. I definitely agree with you on the configuration side of things with Mule, there needs to be a Java DSL for it.

Regarding the management of components, you need to implement your own model that uses a much simpler Service factory that the current SEDA factory (which is all sing all-dancing). This is the purpose of the Model in mule, to manage/change the processing behavour of services. HTH.

Dimitar said...

Thanks Ross, I'll look into this.

Is there any documentation describing the roles and contract of the Model and Service interfaces?

About Me: check my blogger profile for details.

About You: you've been tracked by Google Analytics and Google Feed Burner and Statcounter. If you feel this violates your privacy, feel free to disable your JavaScript for this domain.

Creative Commons License This work is licensed under a Creative Commons Attribution 3.0 Unported License.