OSGI, ExtJS, DWR, Spring MVC

Summary

This blog describes how to  make  modular applications using ExtJS, DWR, OSGI and Spring. It also covers new testing strategies available using OSGI. A prototype will be developed to verify this approach.

Introduction

ExtJS can be used to create rich internet applications using javascript. Web 2.0 libraries like ExtJS allow for the creation of client-server applications where the client code is loaded from a web-server, run in javascript inside of the browser and communicate with the server primarily to access data.

Spring MVC is a very productive server side framework for the development of application using Inversion of Control (IoC) and Model/View/Controller.

DWR is an open source library that allows the remote execution of server-side java code from  a javascript application running on a browser. It can also convert back and forth the javascript data structures in java data structure.

OSGi is a Dynamic Module System for Java. It allows the development of truly modular java applications. Among the many features of OSGI is the ability to add, remove and update software modules (called bundles) to an application while it is running. It also manages  the versioning of bundles and bundle dependencies.

Large applications are often composed of core modules and plug-ins. These plugins are application specific and tend to grow in number over time as the application start covering more grounds. For example, a management application (called CAM for Common Array Manager) need to support an ever increasing number of hardware devices over time. After a few years, CAM may support 20 different devices, but a typical client may only own a few different types of devices. A modular plug-in architecture where plug-ins are implemented as OSGi bundles can help with this problem. Another example is the Notification Provider. A basic management application may be able to send monitoring alarms using Email and SNMP traps but other notifiers could be created and installed that use other technologies to transmit alarms. How many notifiers are installed at the same time can be left to the administrator’s and/or tester’s discretion.

The testing organization need to do regression testing against  the functionality of all devices for every new device added because when a new device is added, often the functionality of core modules is changed, which in turn can break other devices.

Using OSGi framework , it is possible to install two versions of the same core module in one application. This can ensure  that old device plugins are still calling the old version of the core module and that the new device plugin is calling the new version. Over time, the old device plugins should be upgraded and released against the latest version of the core module, but this flexibility can help optimize the testing schedule. In the following example, the core module ‘CM1’ is installed twice in the container.

Prototype Use Case: Notification Providers as plug-ins

What we want as end result of this prototype is a management application that can adapt seemlessly to the addition and removal of Notification Providers. Each notifier plugin must include the ExtJS code to configure the notifier along with the server-side functionality to store/retrieve the configuration and process alarms. The UI side will look like this:

Running the following on the osgi console:

osgi> ss
....
52    ACTIVE      com.sun.cam.web.notification1_1.0.0

osgi> stop 52

should produce the following after a reload of the application on the browser (Only one tab):

We also want to test the ability of these notification bundles to use their own version of the core bundle. This is done by adjusting the version# in the bundle manifests.

The framework

We will use Eclipse with the OSGI extensions to develop this prototype. We first need to create a target platform that include all our dependencies. See http://springosgi.googlepages.com/ch01.html for a good tutorial on how to do this. The target platform files are as follow:

catalina.osgi-5.5.23-SNAPSHOT.jar                          com.springsource.slf4j.api-1.5.6.jar
catalina.start.osgi-1.0-SNAPSHOT.jar                       com.springsource.slf4j.log4j-1.5.6.jar
commons-el.osgi-1.0-SNAPSHOT.jar                           configuration
com.springsource.antlr-2.7.7.jar                           jasper.osgi-5.5.23-SNAPSHOT.jar
com.springsource.com.mysql.jdbc-5.1.6.jar                  jsp-api.osgi-2.0-SNAPSHOT.jar
com.springsource.javassist-3.3.0.ga.jar                    jstl.osgi-1.1.2-SNAPSHOT.jar
com.springsource.javax.activation-1.1.1.jar                log4j.osgi-1.2.15-SNAPSHOT.jar
com.springsource.javax.jms-1.1.0.jar                       org.eclipse.osgi-3.2.2.jar
com.springsource.javax.persistence-1.0.0.jar               org.springframework.aop-2.5.6.A.jar
com.springsource.javax.transaction-1.1.0.jar               org.springframework.beans-2.5.6.A.jar
com.springsource.javax.xml.bind-2.1.7.jar                  org.springframework.context-2.5.6.A.jar
com.springsource.javax.xml.stream-1.0.1.jar                org.springframework.context.support-2.5.6.A.jar
com.springsource.junit-3.8.2.jar                           org.springframework.core-2.5.6.A.jar
com.springsource.net.sf.cglib-2.1.3.jar                    org.springframework.jdbc-2.5.6.A.jar
com.springsource.org.aopalliance-1.0.0.jar                 org.springframework.orm-2.5.6.A.jar
com.springsource.org.apache.commons.beanutils-1.7.0.jar    org.springframework.test-2.5.6.A.jar
com.springsource.org.apache.commons.collections-3.2.0.jar  org.springframework.transaction-2.5.6.A.jar
com.springsource.org.apache.commons.dbcp-1.2.2.osgi.jar    org.springframework.web-2.5.6.A.jar
com.springsource.org.apache.commons.digester-1.8.0.jar     org.springframework.web.servlet-2.5.6.A.jar
com.springsource.org.apache.commons.fileupload-1.2.0.jar   servlet-api.osgi-2.5-SNAPSHOT.jar
com.springsource.org.apache.commons.lang-2.4.0.jar         spring-osgi-annotation-1.2.0-m2.jar
com.springsource.org.apache.commons.logging-1.1.1.jar      spring-osgi-core-1.2.0-m2.jar
com.springsource.org.apache.commons.pool-1.4.0.jar         spring-osgi-extender-1.2.0-m2.jar
com.springsource.org.dom4j-1.6.1.jar                       spring-osgi-io-1.2.0-m2.jar
com.springsource.org.h2-1.0.71.jar                         spring-osgi-mock-1.2.0-m2.jar
com.springsource.org.jgroups-2.5.1.jar                     spring-osgi-test-1.2.0-m2.jar
com.springsource.org.objectweb.asm-2.2.3.jar               spring-osgi-web-1.2.0-m2.jar
com.springsource.serp-1.13.1.jar                           spring-osgi-web-extender-1.2.0-m2.jar

Next we need to create 5 osgi bundles:

• cam.war: the osgi web service.
• com.sun.cam.libs : The core bundle with shared functionality.
• com.sun.cam.web.notification1: The first plugin, an Email Notification Provider.
• com.sun.cam.web.notification2: The second plugin. This one is Snmp.
• com.sun.cam.web.notificationBase:  The notification base plugin with shared notification facilities and the Notification Service Interface.

CAM.WAR

Cam.war is an osgi web service. It is deployed dynamically (http://localhost:8080/cam) and can access classes stored in other bundles. It is deployed in the tomcat osgi container and has the following directory structure:

To work as an osgi web service, the MANIFEST.MF must include all the Required Plug-ins and all the Imported Packages necessary for Spring and CAM functionality. Web.xml must include references to OsgiBundleXmlWebApplicationContext and map /dwr/* to the cam-servlet.xml configuration.

Cam-servlet.xml include the following urlMapping:

This configuration refers to 5 different controllers:

• camController: main application controller, used primarily to populate ExtJS grids and forms.
• helpController: Used to populate the help system.
• jsController: Used to serve javascript files. Javascript files are usually static files rendered by the web container (tomcat), but we have defined a new extension (.djs) for dynamic javascript files. These files are generated at run time by calling, for each installed bundles, a  function that returns a javascript object specific to that bundle. The code of all these javascript object is appended to the base javascript file.  More to come on this.
• dwrController: This is the standard  DWR controller. In our application, dwr is configured to send all client request to the ajaxController.
• ajaxController:  This controller handles all dwr requests prefixed with ‘camFacade’. It also can convert javascript object in java objects.

COM.SUN.CAM.LIBS

Cam.libs is a bundle of java packages, it does not include any services. The services are exported using the ‘Export-Packages’ of the MANIFEST.MF.

COM.SUN.CAM.FMS

This bundle expose all the legacy non-osgi java packages as one osgi service. The acronym  ‘FMS’ stands  for ‘Fault Management Service’. To do this, all legacy jar files (storade,storade_device,bol,fms_agent)  must be included in the class-path of the MANIFEST.MF file and a Spring-osgi configuration file must exist in the META-INF/spring directory of the bundle:

The ‘fmsService’ bean is a Spring bean that exposes the Fms service implementation. The ‘fmsServiceOsgi’ exposes this Spring service as an osgi service under the FmsService Interface. The only new java files in this bundle are FmsService.java and FmsServiceImpl.java. The first one defines the interface for all fms services. The second xml entry (‘osgi:service’)  is the actual implementation of this service against the legacy functionality in the jar files. Because the fms legacy code is itself a  service, it needs to be started and stopped when the osgi bundle is started/stopped. This required that a ‘init-method’ and ‘destroy-method’ be implemented against the Spring Bean. Here is the overall structure of this bundle:

CAM.SUN.WEB.NOTIFICATION

These 3 osgi bundles represent 2 notification providers plug-ins and the base bundle. Notification bundles need to implement the Notification interface stored in the Base Bundle. Each notification bundle include the necessary javascript to render the UI aspect of each plug-in.  They also include all the java code necessary to implement the bundle. This includes the UI controls for the configuration form presented earlier and the actual generation of notifications. The general notification plug-in architecture is as follow:

There are many ways to send javascript code dynamically to a browser using servlet or , in this case, Spring controllers. In this example, the jsController receives a request (/cam/cam/Notifier.djs?action=getNotificationBundles) from the browser and dynamically insert javascript code into the Notifier.djs file before returning the javascript file to the browser. JsController loop over the list of all available NotificationServiceProxys and call the getJavascriptCode() method. This method returns the definition of a javascript object that can be used to render an ExtJS form for each notification providers. The original code from Notifer.djs knows how to loop over each of these objects tp dynamically generate the tabs and call the various methods to paint the forms. The java code look like this:

There are two inheritance paths in the Notification Bundle architecture:

• javascript in notification bundles: these javascript object inherit from a common cam.Notification.base javascript object.
• java code in the bundles: The Notification implementation extends BaseNotifier and implement the Notifier interface.

The next section describes how to configure Spring to inject the notificationServiceProxys correectly.

OSGI-aware controllers

Writing an osgi-aware controller is very easy with String MVC. Here is an example from the AjaxController that returns a list of all the available notifiers in Json format.

First, the Spring configuration file used identify the ‘notificationServiceProxy’:

Then, the controller code. Here we can see that Spring calls setNotificationServiceProxy() to inject the latest list of notification services before each call to getNotifierNames().

This method is available directly to the client using the Dwr utilities. This is a very easy way to test the dynamic aspect of OSGI by stopping services and calling the function again:

Testing Strategies

Installing 2 versions of notificationBase in the same framework.

TODO