Tuesday, September 2, 2008

Dynamic components in Mule

Imagine that you are given to implement the following scenario:

  • A system is composed of a number of uniform entities, each entity publishes its status on a separate pub-sub topic and accepting instructions over separate control queue.
  • Your application receives a stream of instructions over a control channel, instructing it to monitor and control or stop controlling certain entity.

or you receive a stream of active orders and want to subscribe to the right marketdata to calculate a relative performance benchmark, or...

Usually I write about how to implement stuff with Mule, but this time I'm going to write why Mule might not be your best choice if your use-case is anything like above.

The most straight forward approach for implementing the system above is to create a simple POJO, encapsulating the management algorithm and have a service listen to the control channel and instantiate/dispose a new component for each managed entity.

The problem is that in Mule the components are somewhat heavyweight and it's not so nice when you start having hundreds (approaching thousands) of them. Another problem is that the code for manually creating and registering a componment along with the whole paraphernalia of routers, endpoints, filters and transformers is quite verbose.

The latter problem is easier to solve. There are a couple of JIRA issues that aim to simplify the programmatic configuration (MULE-2228, MULE-3495, MULE-3495). The Annotations project at Muleforge is also worth watching (right now there's no documentation, so you'd need to dig in Subversion).

The problem with the too many components is more fundamental. The main application domain for Mule is service integration. Though it also happens to be a fairly good application platform, it does fall short when it comes to managing a lot of components. One of the important things missing is the ability to create groups of components with shared thread pools. Another missing thing is better support for variable subscriptions (provide an API to easily add and remove subscriptions to the same component using some kind of template). Another notable improvement would be to elaborate the models into full blown scope-controllers, allowing to create composite applications with proper interfaces, routing policies, etc. All of these are addressable, but there is a long way to go.

There are also some improvements that can be done to the JMX agent - use JMX relation service to connect component to its endpoint and statistics, all components of certain type, user defined groups of components. Allow the component to exercise some control over the Mule-generated ObjectName so users can organize the different components in groups and ease the monitoring by EMS tools by allowing to query the parameters from a specific group.

Please share if you've had similar issues, how you dealt with them and whether you think {with the benefit of a hindsight) that the solution was worth the effort. Also I'd like to hear from users/developers of other integration frameworks or composite application containers (Newton?) how they handle these problems.

Monday, September 1, 2008

Transformers vs. Components - Take 2

In my previous article, I claimed that one of the main criteria for which Mule artifact to choose is the multiplicity of the inbound and outbound messages. I've been thinking about it and I want to add two more points:

  • Transformers are one instance per endpoint reference. This means that even if you use pooled components, all messages go through single transformer instance. In such case, you need to make sure that the transformer is threadsafe (which is not true if it is holding a JDBC connection or Hibernate session). The components do not have to be threadsafe, though if you are injecting some shared state you still need to properly synchronize the access to it.
  • Mule2 services allow you to configure different ways of instantiating the component (singleton, pooled, looked up from Spring/JNDI/OSGi/etc.). In contrast, all transformers defined using the mule-core namespace are prototype-scoped.
  • I actualy agree that direct database access belongs in a component. I still think that accessing a cache is kindof a gray area though.

Saturday, August 16, 2008

InfoQ, Firefox Hacks and Media Snobbery

I happen to be one of those people that don't like TV - it's not that I don't watch it when I get a chance, but I've made a conscious decision not to buy one and to get my news and movies from other sources. My primary issue is that TV as a media is too engaging, passive and slow. Radio is also passive and slow, but less engaging, hence better.

For example: in 10 mins browsing news sites, I can learn more than 10 mins of browsing newspapers, which still fare better than 10 mins of radio, which is just the same as 10 mins of TV, except that when I watch TV I don't do anything else.

I'm a much faster reader than listener. When I read I can always stop and mull over a sentence for a couple of minutes, go back or skip forward with natural ease. Not so with video and audio. Even though most streaming media these days have some kind of non-linear controls (skip-bar, etc.) it's still impossible to do the equivalent of scanning the headlines or speed-reading in the case of podcast or video content.

One of the reasons I like the InfoQ interviews is that they have full transcripts. One of the resons I hate them is that the transcript is enclosed in small box, 250px high in the middle of the screen and one has to click the open and close buttons and scroll like crazy. What I often end up doing is to fire DOM Inspector, cut off all the DOM nodes except the ancestors of #interviewContent, switch to CSS properties, delete the height and max-height props and this way I end up with full-screen view of the transcript, which I can print or browse at my convenience.

Today as I was debugging some stuff, I remembered that Firefox supports user stylesheets. By putting the bellow statements in my ${FIREFOX_PROFILE}/chrome/userContent.css, I was able to expand the transcript to be readable in a browser without inner scrolling and tweak the printable view to display only the relevant content (no empty squares in place of the flash player or vendor links I can't click on paper.)

/*
 * This file can be used to apply a style to all web pages you view 
 * Rules without !important are overruled by author rules if the author sets any. 
 * Rules with !important overrule author rules. 
 * 
 * For more examples see http://www.mozilla.org/unix/customizing.html 
 */ 
 
div#interviewContent, 
div#interviewContent * { 
    height: auto ! important; 
    max-height: none ! important; 
    overflow: visible ! important; 
} 
 
div#interviewContent div div div { 
    display: block ! important; 
} 
 
@media print { 
    * { 
        border: 0px ! important; 
        float: none ! important; 
    } 
 
    #content, 
    #container, 
    #content-wrapper, 
    .box .bottom-corners, 
    .box .bottom-corners div, 
    .box .top-corners, 
    .box .top-corners div, 
    .box .box-content, 
    .box .box-content-2, 
    .box .box-bottom, 
    .box .box-content-3 { 
        margin: 0 ! important; 
        padding: 0 ! important ; 
        float : none ! important; 
        width: 100% ! important; 
        line-height: 130% ! important; 
    } 
 
    /* these don't work on paper */ 
    object, 
    .comments-sort, 
    .vendor-content-box { 
        display: none ! important; 
    } 
 
    /* visual garbage */ 
    .tags3, 
    .comment-reply, 
    .comment-header > p, 
    .comment-footer, 
    #interviewContent img[alt='show all'], 
    #interviewContent img[alt='hide all'], 
    #content-wrapper > .box > h2, 
    #footer { 
        display: none ! important; 
    } 
 
    /* Fix comment headings */ 
    .comment-header * { 
        display: inline ! important; 
    } 
 
    div.comment-header { 
        border-top: 1px solid gray ! important; 
    } 
 
} 
Now the only thing left is to convince the InfoQ guys to start providing transcripts for the presentations and downloadable slides :-)

Wednesday, July 23, 2008

Transformers vs Components

When I started using Mule, for some time I used to wonder what's the big difference between a transformer and a component. Why don't we just stick with the plain pipes and filters model and we complicate it with the 'component' concept?

In general, transformers have conceptually simpler interface - they are supposed to get a payload and return a payload, possibly of different type. You should prefer transformers for reusable transformation tasks. On the other hand, transformers can not:

  • Swallow messages - to achieve this use filter or router.
  • Return multiple messages - use router.
  • Pause temporarily

In addition, despite their more complex configuration components provide some neat features:

  • Message queuing - if your message goes through multiple stages of enrichment, each taking variable time, you might be better off implementing the stages as components and separating them with VM queues.
  • Statistics - automatically measures the processing time and volume through each in or outbound endpoint.
  • Lifecycle control - you can start, stop or pause component multiple times (while a component is not running the messages will queue up.) The transformers are only initialized and disposed once.
  • A componen allows you to tack on multiple filters, transformers, routers, etc. A transformer is one part of this chain.

To put it together, here is an overview of Mule's component roles in a simple message routing sequence (I'm trying to explain the roles, not the exact sequence.)

  • The message enters through an inbound endpoint, where the transport extracts the payload and the message headers and bundles them in a message. (The message and some other objects are bundled in EventContext, but it is still a mystery to me why do we need it.)
  • After the message is created it passes through a filter chain, where the filters are applied one after another and each of them votes with true or false a full consensus is required or otherwise the message is rejected. Depending on the config the rejected messages can be routed somewhere else. The filters are not supposed to modify the message (in practice they can, but it's a very bad idea.)
  • If a message passes all filters it goes through a transformer chain, where each transformer is applied to the output of the preceding one. A transformer is not able to gracefully reject a message, but it can throw an exception.
  • We might have an interceptor wrapped around the component, which can log stuff before or after invocation or measure latency, etc.
  • Based on the payload and on the component type, Mule resolves the method it should call and if necessarry transforms the payload to something that fits. The return value of the method is used as a payload to a single outgoing message.
  • The outgoing message is passed to an outbound router, that can split it in multiple messages or just drop it. The router then usually sends the message(s) to arbitrary number of outgoing endpoints. The outgoing router uses the filter chains of the outgoing endpoints to decide which ones to use. The outgoing endpoints can be Mule internal endpoints or periphery endpoints, conecting the application to external system.

In general, I would recommend that you start with transformer and refactor it to a component if you realize that you want to add any of the aspects mentioned above. You can even easily use transformers as POJO components.

Monday, July 21, 2008

Testing Mule applications

Antoine Borg has posted an intersting article at Ricston Blog about the feasibility of using mock objects for testing Mule applications. The bottom line is that in integration scenario is usually difficult to capture the application specification in mocks and often it's easier to write stub applications simulating the external systems (outside of your application process).

I've found that often I don't really test the whole application, but instead I test single components with a few attached transformers, filters and routers. I start by writing a simple Mule configuration - a single component using VM endpoints with queuing. Then I use MuleClient to feed in canned input data and assert the output from the outbound endpoint(s). As the application takes shape, I add transformers and routers as needed to approximate the real usage.

I could imagine that the next step would be to reuse my production configuration and extract the perimeter endpoint definitions in a new file (separating them from the model, connectors and internal endpoints) and pass the two configuration files to the config builder. This would allow me to create an alternative perimeter endpoints file using VM endpoints, so I can instantiate it in test case and use MuleClient for testing.

The benefits of the approach are that you are testing your component and routing logic and are not exposed to the peculiarities of the external system. Ideally we still want to have a full integration tests, including ones covering crash-failure, failover, connectivity loss and overload scenarios. We can achieve parts of this by stubbing the external system, but so far I usually find it difficult to reproduce the behaviour faithfully enough (especially when we have limited understanding about the external system).

No bulletpoints this time.

Monday, June 16, 2008

Build, Buy or Steal

This post is based on a text I wrote at my day job. It is edited to reflect my personal opinions and does not represent my employer’s view in any way.

The classic 'build vs. buy' dilemma is further complicated by the new wave of quality open source components.

Even if we assume that the open-source alternatives are generally inferior to their closed-source counterparts, their low price makes them an attractive choice for getting started, provided that there is an easy migration path to more powerful solutions. Additionally, many open-source tools are backed by commercial companies which can provide support, consulting and guidance on demand.

Open-source components are often (not always) higher quality than the internally developed libraries. This is because of the usually pretty-good community QA (many users; no time pressures) and the small-vendor mentality of the backing firms (they need to be much better than the industry standard in order to convince a client). Also, a solid OS project provides a decent documentation and community support, often has books written about it and, in general, makes it easier to recruit people already familiar with it.

We do want to build components in-house when we believe that we can provide and need more value than their OS and commercial counterparts can provide. We always need to keep in account that building good library is a continuous investment in new features, bug fixing and documentation. Providing a sub-optimal internal component is the worst of all worlds.

Another big question when to contribute our changes back to an open source project. Conventional wisdom says that you shouldn't give our work for free. Still, given the previous paragraph, I would claim that it's in our best commercial interest to contribute back any changes to the external module's core. In such cases, the initial investment is small, the maintenance is high (as we need to reapply changes with every release) and the business value of the code is low. Alternative is to fork the open-source component, which brings us to the problem in the previous paragraph.

We might also decide to contribute or open source internally developed extensions of Open Source library, gaining free QA and possibly bugfixes. We should not open source code that gives us [our product] significant advantage over alternative solutions on the market. We must not share non-generic code, capturing: business processes, algorithms, site-specific logic, etc.

And now is the time for the mandatory list at the end of the post. I'm going to enumerate a few build vs buy decision anti-patterns:

  • Not Invented Here (NIH) Syndrome - you know, when we write your own thing because we are too lazy to do our research or read the docs.
  • The Wrapping Party - every external library is wrapped in order to integrate into the proprietary architectural framework. Though it might look like a good decision, it often leads to difficulties in the debugging, inability to apply best practices and tools, and generally inefficient use of the library
  • Nobody Got Fired for Buying Expensive Stuff - when technical decisions are taken by managers without enough information (or understanding). Sometimes this is rationalized as that all the products look the same on paper, so at least this one comes from a reputable company we can sue. Problem is that often the very expensive do-it-all products require a staff of rocket scientists (or vendor consultants) in order to deliver anything after that.
  • The First One is Free - some vendors try to promote their products as open source, while capturing your data and interfaces in proprietary formats and protocols and then selling services around them. The SOA RAD tools and BPM tools are particularly bad offenders here. The problem is that this limits the ways for evolving your platform. The way to prevent this is to always be aware what part of the solution is platform specific: configuration (is it documented file format), POJO vs proprietary interface components, can you get the whole solution as a bunch of text files, what protocols are used for communication, what is the data storage, can we plug our custom infrastructure, where?

Tuesday, June 3, 2008

The Shape of Complexity

Some things are complex and there's no way around it. Still, even if we can't remove the complexity, often we can shape it into different forms. Some applications have shallow and wide complexity - many simple things with relatively few dependencies, but the overall system has mind-boggling emergent behavior (my favorite 25kloc perl scripts example). Other applications have narrow and deep complexity - small code base, but using so much infrastructure amd metaprogramming, that to understand what's going on, you need to be expert in the platform (think about Hello World written with EJB2 or any application using Ruby on Rails).

The horizontal complexity is pretty easy to deal with - find two similar things and create abstraction for them, rinse and repeat. It is important to stop and look every now and then for similar abstractions and for ugly usages. Eliminate the similar abstractions by extracting common functionality in helpers or superclasses; deal with the ugly usages by splitting the abstraction.

By definition, when we introduce abstractions, our complexity becomes 'narrower' and 'taller'. If we put too much stuff into the abstract classes or we nest them too much, we might transform the bunch of simple classes that we couldn't understand when taken together into a somewhat smaller bunch of more complex classes that we can't understand even in isolation. If go overboard in the other direction, e.g. adding too many facades and convenience methods, the API becomes too big without providing enough benefit to learn it (canonical example - I know a project that has a Strings class, containing constants for empty string, single digits, punctoation marks, single letters and other. Apart from being pointless, more verbose and difficult to apply consistently, this also couples all classes in that project to the util package).

So, how do we end up with code base that is neither too tall, neither too wide, but just the right shape?

  • Acknowledge that the right shape depends on the individual - some people can cope with more abstraction, while others can remember more facts. A metric for abstraction efficiency can be defined as delta-loc/nlevels-of-indirection (where n is a constant bigger than 2).
  • Keep in mind the choice of tools - for example IntelliJ IDEA excels at navigating well-factored code, while Vi people often prefer decoupled classes that can be changed with low risk of impacting other areas of code.
  • Consider the infrastructure maturity - using XA transactions with JMS is simple, debugging buggy JMS (caugh.. activemq.. caugh) is entirely different issue.
  • The experience of the team is important - for some people JMS is the most obvious way to send a piece of data once and only once to another system; others treat JMS as black magic and resort to FTP and cunning rename+move schemes, involving multiple directories and recovery scenarios.
  • And finaly, even if different pieces of infrastructure can provide similar functionality, sometimes they vary by the way they do it - in my previous article about component granularity, I mentioned the fat component vs fat Mule configuration scenarios. In this case, we can move the complexity between the Java code and the Mule code. Mule provides out of the box abstractions for threading, routing and transforming, but it is not great as a general-purpose process definition language. Java provides general purpose language, but does not provide high level primitives for threading and routhing. Another example: consider running OLAP against OLTP schema and dedicated warehouse schema - it works, but the difference in speed and CPU utilization can be orders of magnitude.

About Me: check my blogger profile for details.

About You: you've been tracked by Google Analytics and Google Feed Burner and Statcounter. If you feel this violates your privacy, feel free to disable your JavaScript for this domain.

Creative Commons License This work is licensed under a Creative Commons Attribution 3.0 Unported License.