OSGi at LinkedIn
For about 2 months now, LinkedIn has been actively working on a new iteration of its architecture: one that scales both from a customer point of view, and from a developer point of view. Our desire for modularity, decoupling, and service discovery led LinkedIn to look at OSGi for its new platform/container. Currently we’re using Tomcat for our frontend and Jetty for our backend. OSGi has the right properties that we believe will help us achieve our goal.
Probably one of the most known pieces of software that uses OSGi is Eclipse, the most popular open source IDE for Java (yes, it’s a lot more than that, but this is what it’s famous for). OSGi is a specification and Equinox is an open source implementation of the specification on top of which Eclipse is built. There are other implementations of OSGi, like Apache Felix and Knopflerfish. I went to the EclipseCon 2008 conference a couple of months ago to follow the OSGi track. The good news is that OSGi as a server container is getting a lot of traction as more and more people are embracing the new technology and seeing the advantages of what it can bring (multiple versions of the same class in the same container, dynamic updates of classes, etc.).
Moving to OSGi is not a trivial task as the programming concepts differ quite a bit from the more standard Java development lifecycle we are used to. Furthermore, the tooling is currently in its infancy. At LinkedIn, we have a pretty big code base, so it makes the migration all the more challenging.
In this series of posts on OSGi at LinkedIn, we are going to share our experience with OSGi, our pain points as well as our progress and successes. In the process, we will share some of the internals of how LinkedIn is currently built. In the next couple of days, I’ll be writing about how I was able to make our own (proprietary) JSP compiler work within an OSGi container.
trackback
http://blog.linkedin.com/2008/06/10/osgi-at-linkedin/trackback/
David CAlvert June 10th, 2008
Yan,
Just wondering if there are any security issuess for end users using the new platform. Will it be more secure, the same or what? Please comment. Thank you
Dmitriy Kopylenko June 11th, 2008
Hi Yan. Have you seen SpringSource Application Platform yet?
I’d recommend that you give it a close look.
Regards,
Dmitriy.
Dmitriy Kopylenko June 11th, 2008
And here’s S2AP link: http://springsource.com/products/suite/applicationplatform
Yan Pujante June 11th, 2008
@David:
Hi David. The new platform is not in production yet, but it is purely the foundation of the system. The security infrastructure is not going to be changed when we introduce the new platform. So it should have the same level of security.
@Dmitriy:
Hi Dmitriy. Thanks for the link. We are actually evaluating several options right now (plain old Equinox, S2AP, Infiniflow…). We have not settled yet on the final choice, but as soon as we do, there will be a post about what we chose and the reasons why.
Ishak Kang June 11th, 2008
Extremely interesting to hear that LinkedIn is migrating to OSGi. Its quite interesting that the history of OSGi was powering the embedded set top box. Does this spell new platforms for LinkedIn in the future? LinkedIn on my TV? Would be cool.
Jeremy Kraybill June 11th, 2008
Hi Yan, I’m curious if one of your goals when you mention “developer point of view” includes opening up more possibilities to third-party developers through an API? Or is this purely for internal scalability?
Yan Pujante June 12th, 2008
@Ishak: That is an interesting idea. It is not on the roadmap for the near future though :)
@Jeremy: Currently it is more for internal scalability. Nonetheless, the main goal is to enable internal developpers to be more productive and not burdened by the technology, try out new ideas, faster… Which could eventually lead to opening up new apis to external developpers, etc… The sky is the limit when the underlying technology supports it :)
Kjetil June 13th, 2008
It would be interesting to hear more about the experience of moving a large codebase onto OSGi, especially when the goal of the exercise is to increase productivity and isolate developers from the technology. Make them able to write POJOs, right, and provide them with the infrastructure from outside? This is fully possible on OSGi, but it implies some groundwork, like good library support or even some sort of container architecture. It takes some work to bring this into the foundations of an existing code base. Keep us updated.
Kjetil June 13th, 2008
It would be interesting to hear more about the experience of moving a large codebase onto OSGi, especially when the goal of the exercise is to increase productivity and isolate developers from the technology. Make them able to write POJOs, right, and provide them with the infrastructure from outside? This is fully possible on OSGi, but it implies some groundwork, like good library support or even some sort of container architecture. It takes some work to bring this into the foundations of an existing code base. Keep us updated.
Jo Ritter September 22nd, 2008
Yan,
the recent publication of the iPhone mobile LinkedIn application underlines LinkedIn’s attempt to go mobile. As there’ll be a couple of million OSGi phones on the North American market next year, it might be very interesting to develop an OSGi Client as well. You commented above that you’re concerned about your developer’s productivity. Wouldn’t it be a great proof of concept if they used one and the same technology (OSGi) both on the server and the client side?
Jo Ritter
http://mobileosgi.blogspot.com
OSGi@Linkedin « The OSGi Look February 11th, 2009
[...] OSGi at Linkedin Java compilation in OSGi Integrating Spring DM – Part 1 Integrating Spring DM – Part 2 [...]
EmeRsinnA May 12th, 2009
Здравствуйте. Не могу разобраться с разделом для темы. Кто сможет помочь?
Спасибо.