Netuality

Java Caching System is one of the mainstream opensource and free Java caches*, along with OSCache, EHCache and JbossCache. Choosing JCS may be the subject of an article by itself, since this API has a vastly undeserved reputation of being a buggy, slow cache. Exactly this reputation has motivated the development of EHCache, which is in fact a fork of JCS. But, JCS has evolved a lot lately and is now a perfectly valid alternative for production; it still has a few occasional bugs, but nothing really bothersome. I've recently had this interesting experience of cleaning up and tuning a website powered by JCS. This dynamic Java-based site is exposed to a healthy traffic of 0.6-1.2 Mhits/day, with 12.000-25.000 unique visitors daily, and caching has greatly improved its performance. This article is a collection of tips and best practices not comprised (yet?) in the official JCS documentation.

Where to download JCS

This is usually the first question when one wants to use JCS. Since JCS is not a 'real' Jakarta project, but a module of the Turbine framework, there is no downloading link available on the main site. If you search on Google, this question has popped many times on different mail lists or blogs and it usually has two kinds of answers, both IMHO wrong:

• download the source of Turbine and you'll find JCS in the dependencies. No, you won't, because Turbine is build with Maven, which is supposed to automagically download all the needed dependencies and bring them to you on a silver plate. Meaning: tons of useless jars hidden somehwere in the murky depths of wherever Maven thinks is a nice install location. Uhh.
• build it from scratch. Another sadistic advice, given that JCS is also build with Maven. So you'll not only need to checkout the sources from CVS, but also install Maven. Then try to build JCS. And eventually give up. Like for instance in my case, I installed the monster^H^H^H^H^H^H wonderful build tool, then ran 'maven jar'. Instead of the expected result [you know, building the jar !] Maven performed a series of operations like running unit tests, washing teeth, cooking a turkey. Well, I suppose it was doing this, because I couldn't read the huge gobs of text running quickly on the screen. At the end, it miserably failed, with no logical explanations (too many explanations is the modern equivalent of unexplained). So I gave up. Again.

Fortunately, some kind souls at Jakarta (think of these developers as of a sort of secret congregation) provide clandestine latest 'demavenized' binary builds in obscure places; for JCS, the
location is here. I used the last 1.1 build without problems for a few weeks and I strongly recommend it.

Using the auxiliary disk cache

There's a common misconception that one doesn't need no stinkin' disk cache. Even on Hibernate site the example JCS configurations has the auxiliary disk cache commented out. Maybe this comes from the fact that JCS disk cache suffered from a memory leak (not true any more) or from the simplistic reasoning that disk access is inherently slower than memory access. Well it surely is, but at the same time it's probably much faster than some of the database queries, which could benefit from caching.

Also, it is interesting to note that incorrectly dimensioned 'memory' caches will make the Java process overflow from main memory to the swap disk. So you'll use the disk anyway, only in an un-optimized manner !

I wouldn't advise you to activate the auxiliary cache on disk without limiting its size, otherwise, the cache file would grow indefinitely. Controlling cache size is done by 2 parameters (MaxKeySize and OptimizeAtRemoveCount) example:

jcs.auxiliary.DC.attributes.MaxKeySize=2500
jcs.auxiliary.DC.attributes.OptimizeAtRemoveCount=2500

Only MaxKeySize is not enough, since it will only limit the number of keys pointing to values which are in disk cache. In fact, removing a value from the disk cache will only remove its key. But, the second (OptimizeAtRemoveCount) parameter will tell the cache to recreate a new file after a certain number of 'removes'. This new cache file will keep only the cached values corresponding to the remaining keys, thus cleaning all obsolete values, and of course will replace the old cache file. The size of disk cache and the remove count is of course subject of tuning in your own environment.

Tuning the memory shrinker

Although one of the JCS authors specify that the shrinker “is rarely necessary”, it might come handy especially in memory constrained environments or for really big caches. With one exception: be careful and specify the MaxSpoolPerRun parameter (undocumented yet, but discussed on the mailing list) otherwise the shrinking process might lead to spikes in CPU usage. I am using the shrinker like that:

jcs.default.cacheattributes.UseMemoryShrinker=true
jcs.default.cacheattributes.ShrinkerIntervalSeconds=3600
jcs.default.cacheattributes.MaxSpoolPerRun=300

YMMV.

Cache control via servlet

Again, undocumented, but people seem to know about it. The servlet class is org.apache.jcs.admin.servlet.JCSAdminServlet but do not expect it to work out of the box ! This servlet uses Velocity thus you'll need to :

• initialize Velocity before trying to access the servlet (or lazy, but you'll have to modify the servlet source)
• copy the templates into the Velocity template location. The templates (JCSAdminServletDefault.vm and JCSAdminServletRegionDetail.vm) are not (bug ? feature ?) in the jar, so you'll have to retrieve them from the CVS repository. For the moment, they are at this location.

These are my findings. I would have really appreciated to have these few pieces of info before starting the cache tuning. If anybody thinks this article is useful and/or needs to be completed, write a comment, send an email, wave hands. I'll try to come up with more details.

*For a complete list, see the corresponding section at Java-source.

A mini-review

The book already has stellar ratings on Amazon, JavaRanch and other select places, and after reading a few chapters, the only thing I can do is add this post on the praise list.

Why only a few chapters ? Well, you see, this is not exactly the type of book that you read from cover to cover, it's in fact a 720-pages solid collection of JUnit best practices, the most comprehensive you'll ever found in organized, written form. Until now, I have found precise answers to all my JUnit questions.

The book is organized in three big sections weighting some 200-250 pages each. The first one ('The building blocks') is hugely useful if you are a JUnit newbie or even an absolute beginner. It's a detailed introduction to everything you'll need to know in order to start using JUnit: basics, usage in mainstream IDEs, testing patterns, managing test suites, test data and troobleshooting advice. Even seasoned programmers will find useful pieces of advice. I especially liked the 5th chapter ('Working with test data') : an exhaustive description of all the possible ways of organizing your test data. To be honest, it's one of the few chapters from the first section, that I've really read.

The second part ('Testing J2EE') covers testing of XML, JDBC, EJB, web components (JSP, servlets, Velocity templates, out-of-container testing & such) and J2EE applications (security and again some webapp testing – pageflow, broken links, Struts navigation). I can't really pronounce upon this section since I've read only a few subchapters (DBUnit and some related JDBC testing, as well as a few pages from the web testing chapter). But every piece of advice I've got was rock solid.

I've read in its entirety the third part ('More JUnit techniques'). Under this bland title you'll find a group of not-so-common JUnit info such as usage of GSBase addon (funny that I wasn't aware that such a useful addon exists). There's also an intriguing 'Odds and ends' chapter containing some interesting recipes (here's a good one for the QA-freaks like me : 'verify that your test classes adhere to basic syntax rules of JUnit' – sure, why not ?).

Something that I've really missed is a chapter dedicated to mock objects recipes. Yes, there is a quick explanation in the first section – and a reference to Easy Mock or some other mocking API pops now and then in different chapters – there's even an essay about mocking at the end of the book. But the main mock objects dish isn't there. I would've also loved to see some automated GUI-testing recipes (Abbot, Marathon & related tools). But then again, it's a >700-pages book so I'm probably asking too much.

To conclude, 'JUnit Recipes' is the best JUnit book I've ever came into contact with and it supercedes on my list 'JUnit in Action'. Which, interesting enough, was published by the same Manning almost a year ago. Can't have too many good JUnit books, don't they ?

NOTE : unfortunately, this trick is again unavailable starting with Eclipsev3M9, the last version which made that possible was v3M8

• both teams have grown = more activity on the repository,
• vast majority of developers is integrating frequently, aka every little bit of functionality is committed as soon as it's stable and working. Of course, before committing, there's the mandatory update to check for consistency against the most recent codebase. This means that at least a few synchronizations will be performed by each member of the team, each day.
• Code generators. It's true that common sense dictates that nothing generated should be stored in CVS, because this may be source of frequent conflicts and loads the repository in an inefficient manner. However, when one routine generation (for each of the few modules) may take between 3 and 10 minutes and produces thousands of files, it becomes pretty obvious that it should not become part of the build process. What gives : in the days when the model has some important changes, a few different subsequent versions of 3-15MB jar files are committed on the repository. Update process starts to slow down and soon a part of the team is lagging, waiting to download the update. The irony is that usually everybody is downloading slowly and inefficiently the SAME big file. Of course, there are also other kinds of traffic via the VPN, such as Netmeeting (but you can't really have a conversation when everybody in the team is updating the codebase and slows the VPN to a crawl).

This boils down to having a 'read-only' local repository, perfect mirror of the main CVS server, which will be used only for updates. Both the server and its mirror are Linuxes. My choice for mirroring was good old rsync. While cvsup seems to be all the rage nowadays, I headed towards rsync because a) it comes pre-installed within any decent Linux distro and b) I am using Gentoo on my home desktop, so rsync is a tool that I learned to use [and abuse] almost daily, via portage.

We won't lose a lot of time explaining how to set up an rsync server, since it's very well explained in this rather old but useful tutorial. There's just a small twist : we were planning to synchronize frequently so we ran rsync daemon independently, not via xinetd. The config file on the server:

2;root@dev  /etc> more rsyncd.conf

motd file = /etc/rsyncd.motd

log file = /var/log/rsyncd.log

pid file = /var/run/rsyncd.pid

lock file = /var/run/rsync.lock



[cvs]

   path = /opt/cvsroot

   comment = CVS Rsync Server

   uid = nobody

   gid = nobody

   read only = yes

   list = yes

   hosts allow = 10.0.10.100/255.255.255.0

is a classical read-only, anonymous (we're on VPN, right ?) mirroring setup. Next step is making a rsync service in /etc/init.d or appending the line “/usr/bin/rsync –daemon” to /etc/rc.d/rc.local, to be sure that the daemon restarts after reboot [especially when you are NOT administrating the server].

On the client, there are some small tricks to make it work. First one, setup your client CVS repository in the same location as on the server, '/opt/cvsroot' in our case (because you are going to synchro CVSROOT as well, which contains absolute paths in some if its files).

The mirroring script is something in the spirit of:

#!/bin/bash



source /etc/profile

cd /opt/cvsroot



RUN=`ps x | grep rsync | grep -v grep | wc -l`

if [ "$RUN" -gt 0 ]; then

    #already running

    exit 1

fi



rsync -azv --stats 10.0.3.193::cvs/CVSROOT/history /home/cvs/history >/tmp/histo



sum1=`sum /home/cvs/history`

sum2=`sum /opt/cvsroot/CVSROOT/history`



if [ "$sum1" = "$sum2" ]; then

    #nothing to do

    date > /tmp/empty

    exit 0

fi



date > /tmp/started

cat /dev/null > /tmp/ended

rsync -azv --stats --delete --force 10.0.3.193::cvs /opt/cvsroot > /tmp/status

cd /opt

date > /tmp/ended

This (badly written) script is heavily inspired from a script found on a Debian user mailing list:

• exits in the case of a long running rsync process
• assures that the full rsync is triggered by changes in CVSROOT/history. This is a neat trick which minimizes the server activity if you are syncing frequently, as we will do.
• outputs stuff in some files in the /tmp directory. This has a double purpose. First is avoiding root mailbox pollution (because we're 'cron-ing' it, the output will be visible as a few hundreds of mails each day). Second is providing data for a web page on the client machine which tells at a glance what is the synchronization status. AFAIK rsync will not write incomplete files, but the usual sound advice is to make an update between successive synchronizations.

The small 'synchronization status' page (left as an exercise for the reader ) just prints the dates and the last few lines of the output files; this is a dumbed-down sample :

The only step left is to create a file in cron.d containing the line:

0-59/5 7-23 * * * cvs your_mirroring_script.sh

(meaning sychronization each 5 minutes from 07:00 to 23:00) and you may start enjoying blazingly fast CVS updates in Eclipse.

Anybody looking for OSS reporting solutions in Java usually has to make a choice between Apache FOP and Jasper Reports*. While having somewhat different feature sets and addressing distinct reporting solutions, the two APIs boil down to the same basic thing : generate a report from an XML file (or stream/string/whatever). FOP has a clear advantage of standardization (based on XSL-Formatting Objects) while Jasper plays more in the pragmatic field of obtaining those 80% results with a minimum of effort and uses a proprietary XML format.

But FOP is not a standalone reporting solution : it's just a way of transforming XSL-FO files into a report. In order to fill the report with the necessary data, the obvious choice is a templating engine such as Jakarta Velocity. Thus a FOP report creation is a two-step operation :

• create the XML report via Velocity
• feed the XML stream to FOP

Jasper alleviates this problem by including its own binding engine, the only restriction being that input data should support some constraints (such as putting your 'rows' inside a JRDataSource).

Both Jasper and FOP allow inclusion of graphic files inside, usual formats (GIF, JPEG) are supported, however FOP has a nice bonus of rendering SVG inside reports. Unfortunately, this comes with the price of using Batik SVG Toolkit, which is a bulky (close to 2MB) and rather slow API. While processing your dynamic charts as XML files (Velocity again) is a seducing idea, the abysmal performance of SVG rendering will make you give up in no time. Unfortunately, I speak from experience.

At first sight, FOP has a lot more options for output format, compared to Jasper Reports. Of course there's PDF and direct printing via AWT, but also Postscript, PCL, MIF as well as SVG. These choices are quite intriguing, since Postscript and PCL are printing formats (easily obtained by redirecting the specific printer queue into a file), MIF is a rather obscure Adobe format (for Framemaker) and SVG … well, a SVG report is too darn slow to be useable (yes, I was foolish enough to try this, too). Jasper makes again a pragmatic choice by allowing really useful output formats such as HTML, CSV and XSL (never underestimate the power of Excel); and of course: direct printing via AWT and PDF.
While FOP's latest version (0.20.5) was released almost a year ago (summer 2003), Jasper Reports is bubbling with activity – Teodor releases a minor version each one or two months (latest being 0.5.3 at 18.05.2004).

I've decided to use as a 'lab rat' one of the apps developed during my 'startup days': the client GUI is written in Swing and features a few mildly complex reports generated using Velocity+FOP. FOP version is 0.20.4 (the current version back in Q1-2003, when we had to quit dreaming about the 'next financing round' and development halted) but as I already told you FOP has evolved little since then. Though, it's perfectly reasonable to use this implementation as a witness for comparison with Jasper (on the opposite, Jasper has evolved a great deal since Q1-2003).

Back then, the report development cycle was quite simplistic. In fact, the XSL-FO templates were written by hand inside a text editor and the application code was run (via a Junit testcase and some necessary configuration and business data mocking) in order to generate a PDF report. In the case of errors, we had feedback by examining the error traces. Visual feedback was given by the PDF output. While simple to perform, this cycle was extremely tiresome after a while as there was an important overhead : start a new JVM, initialize FOP, fire Acrobat Reader (plus we were using some crappy – even by the standards of 2003 – 1GHz machines w 256/512MB RAM). A WYSIWYG editor would have been nice, so one of my coworkers has made some research and the only solution he found was XMLSpy (Stylevision not available back then) – but, at 800USD/seat this was 'a bit' pricey** for us (only the Enterprise flavor covers FO WYSIWYG editing !?). Another interesting idea was to use one of the conversion tools (from RTF to FOP) such as Jfor, XMLMind or rtf2fo (of these products, only Jfor is free, but feature-poor). What stopped us from doing it was that the generated FO was overly complex : we needed comprehensible cut_the_crap files because we were going to integrate inside Velocity templates. And when you have tens of tags and blocks inside blocks and not the slightest idea which one is a row, which one is a column and which one is a transparent dumbass artefact, it's a gruesome trial-and-error task to integrate even simple VTL loops. And you'd have to do this each time you change something in the report : yikes ! Conclusion : the report development cycle was primitive for FOP and there was no way we could change it.

Things are quite different for Jasper Report : there are a lot of available report designers, and some of them are free. While the complete list is on Jasper Report site, I'd like to note at least three of them :

• iReport is a Swing editor and very interesting because it's not only covering the basic Jasper functionality but also supplementary features such as barcode support (which is admittedly as easy as embedding a barcode font in Jasper with two lines of XML, but much easier to make it via a mouse click). iReport is free, which is excellent, but is a standalone app without IDE integration, and as any complex Swing app is quite slow and a memory hog.
• if you are a developer using Eclipse, you'd appreciate two graphical editors based on Eclipse GEF, available as Eclipse plugins : JasperAssistant and SunshineReports. None of them is free and, at least on paper, the functionality seem identical, but SunshineReports has only the older 1.1 version downloadable, which is free but does NOT work with recent builds of Eclipse 3. How the heck am I supposed to test it ? On the contrary, Assistant has a much more relaxed attitude allowing the download of a free trial for the latest version of their product. Maybe too relaxed, though, because – even if (theoretically) limited in number of usages – you can use the trial as much as you want to***. But if you are serious about doing Jasper in Eclipse you should probably buy Assistant, available for a rather decent 59USD price tag. I am currently using it and it's a good tool.

So much for the tools, let's get the job done. The bad part : if you're experienced with FO templates, don't expect to be immediately proficient with Jasper, even with a GUI editor. The structure of an FO document has powerful analogies with HTML : you have tables, rows, cells, stuff like that, inside special constructs called blocks. It's relatively easy to use a language such as VTL in order to create nested tables, alternating colors and other data layout tricks. You can even render a tree-organized data via a recursive VTL macro, and everything is smooth and easy to understand. Jasper is completely different and at first sight you'll be shocked by its apparent lack of functionality : only rectangles, lines, ellipses, images, boilerplate text and fields (variable text). Each one of this elements has an extensive set of properties about when the element should be displayed, stretch type, associated expression for value and so on. Basically, you'd have to write Java code instead of Velocity macros and call this code from the corresponding properties of various report elements. If at the beginning it feels a little awkward, after a while it comes quite natural and simple. As for nesting and other advanced layouts, there is a powerful concept of 'subreport'. And yes I've managed to render a tree using a recursive subreport, but given the poor performance the final choice was to flatten the data into a vector then feed it into a simple Jasper report. So pay attention to the depth of 'subreporting'.

Once the reports were completely migrated, I've benchmarked a simple one (without SVG, charts, barcodes or other 'exotic' characteristics). The test machine is a 2.4GHz P4 w 512MB Toshiba Satellite laptop. In the case of FOP, the compiled velocity template and the FOP Driver are cached between successive runs. In the case of Jasper, the report is precompiled and loaded only on first run, then refilled with new data before each generation. The lazy loading and caching of reporting engines is the cause of important time differences between the generation of the first report and the subsequent reports. Delta memory is measured after garbage collection. The values presented are median for 10 runs of the 'benchmark report'.

While I am totally pro-Jasper after this short experiment, it is important to note that commercial and well-maintained FO rendering engines such as RenderX XEP claim improved performance upon FOP. Depending on your requirements, environment and reporting legacy apps, an FO-based solution might be better, especially when report generation is only on server-side.

Of course, usual disclaimer apply: this benchmark is valid only for my specific report in my specific application so YMMV.

* While I am aware that other OSS solutions do exist for Java, I consider these two as 'mainstream'.

** Did I mentioned that we were a startup with financing problems ?

*** No, I'm not going to explain here how it can be done.

Here's another nice “real life” story I'd like to share, and this time is about Velocity. Nothing nasty, just a quick check of if and how well Velocity caching is performing, why and how to make your own Velocity ResourceLoader.

The guys I'm working with are migrating a rather huge ERP app from mainframe to Java multi-tier. This isn't exactly an quick and easy job, so basically my first step here is to find out a lot of things about the old system (via some boring but extremely necessary training). However, in order not to lose my so-called “Java skills”, I am also performing some tasks, mainly testing and optimization stuff, preparing the baby to face the harsh real world.

If you haven't seen an ERP tailored for production sites before, you'll be amazed at the massive number of barcode stickers which have to be printed. They are everywhere, from production to distribution, relaying boxes, smaller boxes, bigger boxes, packs, containers, everything your mind can think of.
These barcodes are produced on special printers which are usually connected to the production systems via serial port (IP connection is possible, but quite expensive so it's used only in very special cases, like really large warehouses).

import java.io.ByteArrayInputStream;

import java.io.InputStream;

import org.apache.commons.collections.ExtendedProperties;

import org.apache.velocity.exception.ResourceNotFoundException;

import org.apache.velocity.runtime.resource.Resource;

import org.apache.velocity.runtime.resource.loader.ResourceLoader;



public class MyResourceLoader extends ResourceLoader {
  public void init(ExtendedProperties arg0) {

    // TODO Auto-generated method stub
  }

  public InputStream getResourceStream(String templateName) throws ResourceNotFoundException {

    //TODO: exceptions here

    InputStream in=
      new ByteArrayInputStream(
        (TemplateContentsSingleton.getUniqueInstance().getTemplate(templateName)).getBytes());

    if (in == null) {
      String msg= "*** BuiltInTemplateResourceLoader Error: cannot find resource " + templateName;

      throw new ResourceNotFoundException(msg);

    }



    return in;

  }

[...]

}

This is pretty “spike-ish” and completely non-thread-safe code, kids don't try this at home without correctly processing all errors and managing modification flags. Basically, the loader is using a TemplateContentsSingleton class that wraps inside it a hashmap of templates, indexed by their key (which by the way is a String, and that's just about perfect).

public class TemplateContentsSingleton {



  /** unique instance */

  private static TemplateContentsSingleton sInstance= null;

  /** template containers */

  private Map tmplContainers= new HashMap();

  /**
   * Private constuctor
   */

  private TemplateContentsSingleton() {

    super();

  }

  /**

   * Get the unique instance of this class.

   */

  public static TemplateContentsSingleton getUniqueInstance() {

    if (sInstance == null) {

      sInstance= new TemplateContentsSingleton();

    }

    return sInstance;

  }

  public void setTemplate(String key, String templateContent) {

    this.tmplContainers.put(key, templateContent);

  }

  public String getTemplate(String templateName) {

    return (String) this.tmplContainers.get(templateName);

  }

}

For your extreme comfort, this is an ultra-classic singleton contaning a hashmap.
Don't forget to initialize Velocity with the corresponding properties :

Properties veloProps= new Properties();

veloProps.setProperty("resource.loader", "custom");

veloProps.setProperty("custom.resource.loader.description","Customized Velocity Template Resource Loader");

veloProps.setProperty("custom.resource.loader.class", MyResourceLoader.class.getName());

veloProps.setProperty("custom.resource.loader.path", "");

veloProps.setProperty("custom.resource.loader.cache", "false");

Finally we'll be able to test the effect of caching different objects. Obvious candidates for caching are the template and the context. We'll render one of the (pretty small) templates 1000 times, then compute the mean rendering time. We'll do the benchmarking with Velocity caching disabled and enabled (by setting custom.resource.loader.cache to “true”). OK, let's get to work :