Giving Classpath Problems the Strap

(home)

I’ve been meaning to write about a few code-related things I did a while go for some time now. Today I finally decided to get to it. What follows is a story of life, love and class paths.

Whether it be a night at the ballet, or a local poetry recital, there are many activities whose ability to inflict agony on the average Java geek are no match for insidious devil spawn otherwise known as the classpath. Launching a Java application in any production environment often involves cobbling together an abundance of 3rd-party dependencies, some voodoo magic in the form of shell or batch file scripting and a healthy dose of cellotape. While the alternative would require the entering of a start-up command whose length would see Tolstoy run screaming like body builder facing a drug test; the classpath remains a thorn in the side of many developers.

At some point during the sleeplessness otherwise known as the development of fedWS2, I broke. Imbued with the kind of self-confidence that only comes at 4am and buzzing with a caffeine high so intense it set off car alarms for miles, I began on a journey that lasted at least 90 full minutes. Sparked into action by the thought “there must be an easier way,” it turned out that unlike most of my 4am callings, I was correct. The secrets I share with you below are neither revolutionary, nor brilliant in any way. They are merely the result of having to write one too many batch files*.

Searching for an Alternative

Previously, I had attempted to work the automatic generation of launcher scripts into my Ant based build process. After spending a considerable amount of time on this I abandoned it as the effort involved made hand authoring batch and shell files look trivial. Battered and bruised from defeat I decided to should look for a third-party option.

A quick Google search and was on the Commons Launcher page. Let me state right up front that I have never been a fan of the commons libraries. Having a heap of these little jar files running around requiring managing just feels untidy to me (too much like manual RPM management). Looking into Commons Launcher didn’t inspire any more confidence. I poked around the documentation and found the whole process to be heavy and cumbersome. Looking around the download only reinforced this feeling. A quick glance in the library directory found no less than 31 dependant jar files weighting in at more than 3mb. So ended my foray with Commons Launcher. My nose picked up that definite stink of dead weight generally only found around Microsoft products. It seemed that all was lost. I dropped the idea and with head hung low returned to the fiery pits from whence I came.

If You Want it Done Right

It is at this point that our story jumps forward to 4am on the morning in question. I had just spent some time writing my own class loader that allowed for dynamic extension of the search path** (something you could do with URLClassLoader, if the method to add a URL were not protected). Suddenly it hit me. It was one of those moments when the blindingly obvious snaps into place, like when you can’t figure out why your pillow is so hard before you realise you’re sleeping in the gutter in the same clothes you went out in last night . . . . or something similar.

If I can dynamically extend the classpath, why not just read in a single file that lists all my dependencies, add them to the path and be done with the menace of scripts! A quick sip (read: bottle) of Coke and I was upon the keyboard like a drunk Japanese businessman on a karaoke machine. The result was a 9k jar file that contained everything I needed to fulfil my dream.

The Bootstrapper

Internally, it is very simple. Batch files and shell scripts are still required, but now they need only be a single line in length. To begin with, I’ll show you what I mean:

java -cp bootstrapper.jar -Dfile=example.classpath com.lbf.commons.classloader.Bootstrapper $*

It’s that simple. The new main class (for every application) is Bootstrapper. When invoked, this class will look to the system property “file” for the location of the classpath information. It will create a new ExtendedClassLoader (the name of the loader I wrote) and configure it with all the classpath information. It will then extract the name of the main class (also from the classpath file) and load that class with the newly created loader. Once successful, it will invoke the usual main method via reflection (passing the command line arguments it was given) and the application is off to the races.

The format of the configuration file for the bootstrapper is also really simple:

[The requested file http://tim.littlebluefrog.com/code/12-jan-2006/example.classpath could not be found]

There are 5 different directives that can be used (with only the “main:” one required), each of which is explained below

  • main: The fully qualified name of the applications main class
  • prop: Registers a system property
  • path: Adds the location to the classpath exactly as it is given
  • scan: Assumes that the given location is a directory and scans it, adding any jar, war or zip files to the class path
  • deep: The same as “scan”, except that any subdirectories are also searched

That is all there is to it! Because the newly created class loader was used to load the main class, it will also be used to load any subsequent classes. There is however, one potential problem. The code for your application cannot make use of the system class loader. While this won’t be a problem for 99% of applications out there, if you are included, say, embedded Tomcat (which fedWS2 does) you will be in trouble as it uses a loader hierarchy that will skip right over the bootstrappers class loader. That situation aside, everything works fine.

Parting Words

There is no doubt that the class path is a genuine pain in any Java developer’s neck. I believe that the approach described above does substantially reduce the pain of this situation and has helped me out no end. The bootstrapper is actually part of a library I used internally for labs stuff (called the “commons” :P) but it is packaged as a separate jar for this purpose. It is released under the LGPL and if you want to use it, head over to the littlebluefrog labs download site (here). If you want the source for either the bootstrapper or the extended class loader, you will have to download the source version of the commons library when I put it up. I’m still doing work on other portions of the library so it’s not available yet. If you really want the source and can’t wait, just email me at tpokorny at gmail.com and I’ll fire it off to you :)

* In case you were wondering, one batch file is one too many, so you can imagine my pain when exposed to many, many more than this. Also, I was annoyed because the fedWS2 launcher script had become so big that Windows 2000 could no longer handle it. [insert obligatory windows bashing here].
** For those interested, fedWS2 generates and compiles code at runtime (from an XML descriptor document) and this code is then used within the framework. However, for this to happen it needs to be loaded in. The class loader described is used for this purpose.


6 Responses to “Giving Classpath Problems the Strap”

  1. Anonymous Coward Says:

    Here’s a heretical thought - what about adding an option to read runtime dependencies from Maven POMs?

  2. tim Says:

    Heretical it certainly is :) I don’t know much about Maven, but it certainly sounds reasonable. Crimson should do fine for such a task, so no dependency would be added for the bootstrapper itself. My only question would be with regard to deployment: would you normally ship a POM with an application?

    I’ll make a deal with you Anon (or anyone else): if someone emails me a sample POM I can work from (any sensitive bits removed naturally) I’ll give it a go. The email address is at the bottom of the post.

  3. Jesse Wilson Says:

    Great idea! I have 2 suggestions:
    - What if there was a default classpath file it always looked for such as “default.classpath”. Just like how Ant looks for build.xml by default. That saves one commandline argument.
    - Adding a default main class to bootstrapper.jar via the Manifest allows you to use java -jar bootstrapper.jar. This saves another argument.

    The net result is that your batch file now looks like this:
    java -jar bootstrapper.jar $*

  4. tim Says:

    Jesse -

    1. The default file thought had already crossed my mind and I’ve made a note of it in my bug tracker to remind me to do it.

    2. That’s an awesome idea, I’ll put it in tonight!

  5. Scott Sauyet Says:

    Have you looked into OneJar (http://one-jar.sourceforge.net/)? I haven’t useed it yet, but it looks like it might be a good tool for my own projects, and might help you too.

    OneJar builds a single JAR file that contains all the JARs on which your application depends. It doesn’t expand them, just includes them and searches them as though they’d been expanded. I don’t know if that would do for you.

  6. tim Says:

    G’day Scott, I had thought about something along those line before (but just doing it manually/ant’ing it) but in some cases I would have run into licensing issues by doing that (or else I would have to repackage the jar’s to include the appropriate license files etc…).

    Ultimately I chose the bootstrapper road because it helps me during development as well as production. I have a small shell script that take as an argument the classpath file to use and then runs the bootstrapper with it, so I can easily have lots of little tests or the like hanging around and just point the script to a different classpath file to run whatever one I like. It’s the little things like that which really pushed me over the edge to do this :P