OSGi at LinkedIn: Integrating Spring DM (Part 1)
LinkedIn has been extensively using the Spring Framework for wiring purposes and life cycle management (we seldom use other features like AOP, JDBC, Spring MVC, etc.). In other words we essentially use the IoC container. To give you an idea of how extensive we use it, as of this writing, we have over 1000 Spring files that make up the LinkedIn platform!
Before Spring 2.0, there was no easy way to extend the framework itself to add your own custom XML tags. One might wonder why you would want to do this? Well I guess the best answer lies in the fact that it was introduced with Spring 2.0 :-). More seriously, the idea of having your own custom tags can be compared to the ability of creating a JSP tag library for JSPs: you can create tags that are more specific to your domain, define required attributes so that it can actually be validated with an XML schema (instead of properties), etc.
Example without custom tags:
<bean id="dataSource" class="...JDBCDataSource"> <property name="dbURL" value="jdbc:..."/> <property name="timeout" value="1000"/> </bean>
With custom tags (achieving the same result):
<lin:jdbcDataSource id="dataSource" dbURL="jdbc:..." timeout="1000"/>
The documentation for writing your own custom tags explains in details how to create your own. After you've created custom tags, your JAR file will look something like the following:
META-INF/ spring.handlers # http://www.linkedin.com/lispring=com.linkedin.spring.LispringNamespaceHandler spring.schemas # http://www.linkedin.com/lispring/lispring.xsd=com/linkedin/spring/lispring.xsd com/ linkedin/ spring/ LispringNamespaceHandler.class # extends NamespaceHandlerSupport lispring.xsd # contains the schema for the custom tags
Writing the code that handles the new tag is pretty low level and requires you to understand how Spring actually works under the covers. If I had one wish, it would be the ability to write the extension without having to write code (basically defining the extension as a normal Spring XML file!).
Spring Dynamic Modules (or Spring DM) is the integration of Spring and OSGi. It is very interesting because it allows you to stay in POJO land and be able to look for references/deploy services in an OSGi container without ever writing any OSGi specific code. For example you don't need to write a
BundleActivator
, Spring DM does the plumbing for you. Spring DM is extending the Spring framework using the namespace extension described previously, to offer a new set of custom tags to wire the services together (
<osgi:service>
and
<osgi:reference>
).
Example:
<?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:lin="http://www.linkedin.com/lispring-osgi" xmlns:osgi="http://www.springframework.org/schema/osgi" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd http://www.springframework.org/schema/osgi http://www.springframework.org/schema/osgi/spring-osgi.xsd http://www.linkedin.com/lispring-osgi http://www.linkedin.com/lispring-osgi/lispring-osgi.xsd"> <osgi:service ref="jobsDataService" interface="com.linkedin.jobs.ds.api.JobsDataService"> <osgi:service-properties> <entry key="linkedin.remotable" value="true"/> </osgi:service-properties> </osgi:service> </beans>
So now here is an interesting question: how do you make your own custom tags available with Spring-DM?
In regular (non OSGi) Spring, whenever a namespace is defined in the (rather verbose) XML declaration, Spring uses the classpath to locate the entry point to the code that needs to be executed for the namespace you define (
META-INF/spring.handlers
). In my previous post, I mentioned that in OSGi the concept of classpath is nonsensical, so clearly this mechanism does not work anymore.
Spring DM actually uses a famous OSGi pattern: the extender pattern. Roughly speaking, it comes down to implementing a listener on the OSGi framework bundle events like "bundle resolved" or "bundle unresolved". Whenever Spring DM detects a resolved bundle which contains the correct file (
META-INF/spring.handlers
), it adds the namespace to the list of known namespaces (and remove it when the bundle is removed).
This is a very good demonstration of how dynamic OSGi is: it is very easy to add and remove features to a running OSGi container. As a side note, "remove" is a concept that is often overlooked as it can even be experienced in the JDK itself! For example, there is a way to add new protocol handlers for URLs with the method
URL.setURLStreamHandlerFactory(URLStreamHandlerFactory factory)
. Not only this method can be called only once per JVM (as stated in the javadoc), but where is the unset method? Thankfully, the OSGi spec defines a way to add and remove dynamically new stream handlers.
Below is a code example for the extender pattern:
BundleContext context = ...
context.addBundleListener(new BundleListener() {
public void bundleChanged(BundleEvent event)
{
if(event.getType() == BundleEvent.RESOLVED)
{
Enumeration urlEnum = event.getBundle().findEntries("META-INF", "spring.handlers", false);
if(urlEnum != null)
{
// namespace extension detected!
}
}
}
});
Note that in the listener, you have access to the Bundle itself (
event.getBundle()
), so the extender pattern can be based on whatever you want which is accessible through this interface. For example it could be based on proprietary manifest headers that are accessible through
.
The answer to the original question (about how do you make your own custom tags available with Spring-DM) is now very simple. If you have a JAR file already containing your custom namespace, it is now just a matter of turning the JAR file into an OSGi bundle. Spring-DM will automatically detect it!
I have been using Spring DM 1.1.0-m2 for my testing and it works perfectly. Nonetheless be aware of an issue: there is a race condition which is not properly handled in this release (it is unclear to me whether it will be addressed in a future release). The scenario is the following:
- Install 2 bundles: B1 which contains the namespace extension and B2 using the namespace extension.
- If B2 happens to be resolved before B1 then spring will fail when it encounters your custom tag (because it doesn't know about it).
It is a race condition because it depends on the order and timing in which bundles gets resolved. It can be pretty tricky to enforce an ordering and I believe it would be better if Spring DM would handle this case gracefully. If an unknown namespace is encountered, it should wait for some amount of time for the namespace to be (asynchronously) registered. If the namespace does not get registered within the amount of time, then it should fail. Spring DM is already doing something similar when wiring services which may not be present in the OSGi registry at wiring time.
Stay tuned for part 2: I will be talking about how to extend the Spring DM application context using fragments.
For previous posts on OSGi at LinkedIn, please see my introduction and Java Compilation in OSGi.