ioannis cherouvim

If you experience the following warning in your hibernate+ehcache application:

2008-11-20 13:02:42,937 WARN  (CacheManager.java:322) - 
  Creating a new instance of CacheManager using the diskStorePath 
  "D:\apache-tomcat-5.5.26\temp" which is already used by an 
  existing CacheManager.
The source of the configuration was classpath.
The diskStore path for this CacheManager will be set to 
  D:\apache-tomcat-5.5.26\temp\ehcache_auto_created_1227178962937.
To avoid this warning consider using the CacheManager factory 
  methods to create a singleton CacheManager or specifying a 
  separate ehcache configuration (ehcache.xml) for each 
  CacheManager instance.

then you need to set the following in your hibernate.cfg.xml file:

<property name="hibernate.cache.provider_class">
  net.sf.ehcache.hibernate.SingletonEhCacheProvider
</property>

Ehcache Hibernate documentation

I’m not ashamed of those sins any more, so here you go :)

1. ORM

Stupidity

Building my own Object Relational Mapping framework.

Consequence

Project is a mess after 2 years of maintenance with hardcore hacks to bypass my own ORM and call custom SQL queries.

What should I have done

Use hibernate, iBATIS, Cayenne or something similar.

2. EAV

Stupidity

Using an Entity-Attribute-Value model database schema design.

Consequence

Non scalable solution and total impossibility to run any useful queries on the database level.

What should I have done

Use an ordinary normalized database schema design.

3. Database Access

Stupidity

Synchronize (serialize) database access using one shared connection.

Consequence

Zero scalability. Very slow response times when more than 10 users where using the application.

What should I have done

Don’t do that and use a connection pool such as c3p0 and use a “new” (reused) connection returned from the pool for every request/response cycle.

4. IDE

Stupidity

Avoided learning and using an Integrated development environment.

Consequence

Inability to build test and deploy the application quickly and generally do anything useful.

What should I have done

Get familiar with an IDE. NetBeans, eclipse etc.

5. Transactions

Stupidity

Not using them.

Consequence

Corrupt data in an application involving e-shop like functionality.

What should I have done

Use database transactions. When in MySQL use InnoDB.

6. Prepared Statements

Stupidity

Using Statements, string concatenation and naive character escaping to assemble my own “safe” queries.

Consequence

SQL Injections possible in my application. I managed to login using ‘or 1=1;delete from users;– and alter the database state in a very nasty way.

What should I have done

Use Prepared Statements which correctly assemble and escape the query properly depending on the JDBC driver used.

7. Business Logic

Stupidity

Doing it in the template (JSP).

Consequence

Messy non maintainable application.

What should I have done

Do it in an MVC style with servlets or with a Front Controller. Even better by using an existing open source MVC framework such as Struts, Spring MVC etc.

Of course, all the bad choices above have probably made me a better programmer.

What follows is usual modifications I apply to java code, when I code review. Most of the following changes have a stylistic nature, but some of them can impact performance. Bad code is in red and good code is in green. Feel free to comment.

Unnecessary if statement

if (foo>bar) {
  return true;
} else {
  return false;
}

return foo>bar;

Unnecessary nesting

if (foo) {
  if (bar) {
    // do something
  }
}

if (foo && bar) {
  // do something
}

A bit of redundancy here

if (foo!=null && foo.equals("bar")) {
  // do something
}

if ("bar".equals(foo)) {
  // do something
}

Looks ugly

String comment = request.getParameter("post");
if (comment==null) {
  comment = "n/a";
}

String comment = request.getParameter("post")!=null?
        request.getParameter("post"):
        "n/a";

This only holds if request.getParameter("post") is cheap.

Unnecessary variable allocation

Something something = new Something(foo);
return something;

return new Something(foo);

Too much variable scope

Something something = new Something();
if (foo) {
  something.prepare();
  request.setAttribute("something", something);
}

if (foo) {
  Something something = new Something();
  something.prepare();
  request.setAttribute("something", something);
}

Method exposes implementation

public ArrayList getSomething() {
  ...
}

public List getSomething() {
  ...
}

A bit slow

String foo = "something";
if (!foo.equals("")) {
  // do something
}

String foo = "something";
if (foo.length()>0) {
  // do something
}

Too many lines

List immutableList = new ArrayList();
immutableList.add(foo);
immutableList.add(bar);
immutableList.add(example);
Something something = new Something(immutableList);

Something something = new Something(
        Arrays.asList(new Object[] {foo, bar, example}));

When generating the database schema using the hbm2ddl tools, foreign keys are being created for every relation. You can enhance the schema by using the foreign-key attribute to specify your own name for a particular foreign key.

But, what happens when you don’t want a foreign key generated?

There is an undocumented behavior of the foreign-key attribute, and that is to specify foreign-key=”none”. hbm2ddl will not create an FK for a relation which has such an attribute. The hibernate documentation and the two bibles Hibernate in Action and Java Persistence with Hibernate state absolutely nothing about this feature. I found it after hardcore googling in the following hibernate changelog:

Changes in version 2.1.9 (xx.x.xxxx)
------------------------------------
* foreign-key="none" can be used to disable generation of a foreign key.

Later on, I found another blog mentioning this behavior: http://blog.xebia.com/2007/02/05/let-hibernate-connect-your-world/

The most logical question now is “why don’t you want an FK?”. That depends on the system you are building. Sometimes you might have a table in your application which will be CRUDed from another system, which you do not control. You may want to be able to keep references (ids) to entities which may be deleted. foreign-key=”none” together with not-found=”ignore” can solve these kind of problems.

You want to create a helper class to analyse an HttpServletRequest and provide you with helper methods which determine the type of request. You basically care about GET and POST request methods. More specifically you want to be able to tell whether a POST is Multipart (usually containing file uploads).

Without losing a second, you create the class and make it compile. Right now you only care about the API of the class, and how will it be used from the client side of view. The class is immutable.

import javax.servlet.http.HttpServletRequest;

public class HttpRequestAnalyser {

  private final boolean get;
  private final boolean post;
  private final boolean multipart;

  public HttpRequestAnalyser(HttpServletRequest request) {
    // do nothing of value (yet)
    this.get = false;
    this.post = false;
    this.multipart = false;
  }

  public boolean isGet() {
    return get;
  }

  public boolean isPost() {
    return post;
  }

  public boolean isMultipart() {
    return multipart;
  }

}

Press F9, it compiles (NetBeans).
No matter what kind of request we construct this class with, it will never give us a true answer on whether the method was GET, POST or POST-MULTIPART. That’s because the implementation is the absolute minimum we need in order to define our API and have something that compiles.

Press CTRL+SHIFT+U to create the following empty test.

import junit.framework.*;

public class HttpRequestAnalyserTest extends TestCase {
  
  public HttpRequestAnalyserTest(String testName) {
    super(testName);
  }

  public void setUp() {
  }

  public void testIsGet() {
  }

  public void testIsPost() {
  }

  public void testIsMultipart() {
  }
  
}

Press SHIFT+F6, the test passes because there are no assertions yet.

The point is to create good tests for our HttpRequestAnalyser class. The tests will fail, and then we’ll try to make them pass by doing real work in our class, which is the System Under Test.

In order to test our class we will need instances of HttpServletRequest. It’s not (easily) possible to create such instances, as this is an interface, and usually the servlet container generates concrete implementations. We don’t want to incorporate a servlet container in our tests. What we are going to do is mock an HttpServletRequest, as if it was coming over http from a client.

In order to mock such a request we could provide our own implementation, possibly in an inner static class, inside our test and we’d have to implement all methods.

Another way is to create a mock object which will implement HttpServletRequest and program it with canned answers for our class. There exist many frameworks out there for mocking and we’ll use mocquer.

We need to add the servlet-api.jar and the mocquer dependencies on our project’s test libraries.

In order to fake HTTP request data, we need to know how these look like. Firefox and many other http header monitor tools out there help us gather the following interesting parts of the requests:

GET method

GET /example.html HTTP/1.1
Host: www.domain.com

POST method

POST /example.html HTTP/1.1
Host: www.domain.com
Content-Type: application/x-www-form-urlencoded

POST method with file upload

POST /example.html HTTP/1.1
Host: www.domain.com
Content-Type: multipart/form-data; boundary=...

The test now becomes:

import javax.servlet.http.HttpServletRequest;
import junit.framework.*;
import org.jingle.mocquer.MockControl;

public class HttpRequestAnalyserTest extends TestCase {
  
  public HttpRequestAnalyserTest(String testName) {
    super(testName);
  }
  
  private MockControl requestControl;
  private HttpServletRequest request;
  
  public void setUp() {
    requestControl = MockControl.createControl(HttpServletRequest.class);
    request = (HttpServletRequest)requestControl.getMock();
  }
  
  public void testIsGet() {
    request.getMethod();
    requestControl.setReturnValue("GET");
    requestControl.replay();
    HttpRequestAnalyser analyser = new HttpRequestAnalyser(request);

    assertTrue(analyser.isGet());
    assertFalse(analyser.isPost());
    assertFalse(analyser.isMultipart());
  }
  
  public void testIsPost() {
    request.getMethod();
    requestControl.setReturnValue("POST");
    request.getHeader("Content-Type");
    requestControl.setReturnValue("application/x-www-form-urlencoded");
    requestControl.replay();
    HttpRequestAnalyser analyser = new HttpRequestAnalyser(request);

    assertFalse(analyser.isGet());
    assertTrue(analyser.isPost());
    assertFalse(analyser.isMultipart());
  }
  
  public void testIsMultipart() {
    request.getMethod();
    requestControl.setReturnValue("POST");
    request.getHeader("Content-Type");
    requestControl.setReturnValue("multipart/form-data; boundary=...");
    requestControl.replay();
    HttpRequestAnalyser analyser = new HttpRequestAnalyser(request);

    assertFalse(analyser.isGet());
    assertTrue(analyser.isPost());
    assertTrue(analyser.isMultipart());
  }
  
}

We’ve provided method call expectations, and canned answers on the mock. The tests fail.
We fill in the HttpRequestAnalyser constructor implementation which now becomes:

import javax.servlet.http.HttpServletRequest;

public class HttpRequestAnalyser {
  
  private final boolean get;
  private final boolean post;
  private final boolean multipart;
  
  public HttpRequestAnalyser(HttpServletRequest request) {
    this.get = request.getMethod().equals("GET");
    this.post = request.getMethod().equals("POST");
    this.multipart =
        request.getHeader("Content-Type").startsWith("multipart/");
  }
  
  public boolean isGet() {
    return get;
  }
  
  public boolean isPost() {
    return post;
  }
  
  public boolean isMultipart() {
    return multipart;
  }
  
}

Post and multipart tests pass. The get test fails, because we haven’t set an expectation for getHeader(“Content-Type”) to be called. That is because the GET Http method doesn’t have such a header.
The multipart check should only happen if the request has been recognised to be a post:

import javax.servlet.http.HttpServletRequest;

public class HttpRequestAnalyser {
  
  private final boolean get;
  private final boolean post;
  private final boolean multipart;
  
  public HttpRequestAnalyser(HttpServletRequest request) {
    this.get = request.getMethod().equals("GET");
    this.post = request.getMethod().equals("POST");
    this.multipart = post ?
      request.getHeader("Content-Type").startsWith("multipart/") :
      false;
  }
  
  public boolean isGet() {
    return get;
  }
  
  public boolean isPost() {
    return post;
  }
  
  public boolean isMultipart() {
    return multipart;
  }
  
}

Tests pass, and the class is ready to rock! Happy testing (before coding).

Javadoc is a tool that you have in your %JAVA_HOME%\bin (Windows) or $JAVA_HOME/bin (UNIX) directory. It is being used to extract API documentation from your Java source code, into HTML.

Most IDEs fully support Javadoc. They assist the developer in writing, displaying and generating the Javadoc documentation. Javadoc is part of the development lifecycle. Good and up-to-date Javadoc is a sign of a healthy project.

API Users

The worst thing an API user can do is to ignore Javadoc. Having an easily accessible, local copy of the Javadoc for the library you are working with (Spring, Hibernate, Lucene etc) is the only way to work efficiently. Vast amounts of time have been put into compiling quality documentation for these libraries. Not using it will only slow you down.

Development Teams

One of the worst Netbeans feature is that it allows you to collapse all comments and even set this as the default behavior when opening a source file. Team members, working in the same piece of code, can no longer see the documentation written by other developers.
I actually had a developer asking me about what one of my methods did. For a moment I was confused and asked him whether I had written poor documentation. He then told me that he didn’t know that there was any Javadoc for that method. I don’t (want to) know whether Javadoc hiding is a feature of Netbeans or is a plugin, but that was definitely one of the most ineffective behaviors I’ve seen in a development team.

API designers

Writing good Javadoc involves many things which are documented by SUN. The general idea is that you don’t want to expose the implementation of the method through the documentation. API users will not care how you do it, but only what (and maybe why) you do.
So the following comment is bad:

/** 
 * This method uses StringBuffer to merge the name with surname and return
 * the person's full name
 */
 public String getFullName() {

A better version is:

/** 
 * Returns the full name of the Person. Will return "Nick Cave" if
 * firstname is "Nick" and lastname is "Cave".
 *
 * @return      the full name of the Person
 */
 public String getFullName() {

Conclusion

We (Java developers) are very lucky that documentation is so tightly integrated with the language. Not using it though, takes all this goodness away.

FreeMarker is a very flexible templating engine for Java. Exception handling (while rendering the template) is a very important issue for a templating engine. As with JSP the default behaviour of FreeMarker is to completely cancel rendering and display an error page. When developing a webapp this might not be very helpful. Sometimes errors might need to be tolerated; at least for the development phase of the application development.

FreeMarker provides the TemplateExceptionHandler interface with some implementations but we’ll define our own in order to provide a more failsafe and usable behaviour.

public class MyTemplateExceptionHandler 
          implements TemplateExceptionHandler {

  public void handleTemplateException(TemplateException te, 
          Environment env, Writer out) {
    freemarkerlog.error("template error", te);
    try {
      out.write("<span style=\"cursor:help; color: red\" " +
                "title=\"" + ExceptionUtils.getMessage(te) + "\">" +
                "[e]" +
                "</span>\n");
    } catch (IOException ignored) { }
  }
  
}

Then, in the code where you configure FreeMarker you need:

config.setTemplateExceptionHandler(new MyTemplateExceptionHandler());

This is what you’ll see whenever there is an exception thrown while rendering the template:

A nice little [e] with a tooltip containing the exception message.

I’m fond of optimization. When I code or design my database schema I try to avoid waisting CPU cycles or storage space (at least without a good reason). So when my domain class has the following field:

private boolean active; // determines whether this Person is active

I will let hibernate and the MySQL5InnoDBDialect choose what is most appropriate:

<property name="active" not-null="true" />

In that case it will generate a BIT:

...
active bit not null,
...

The problem

So far so good…
…until you read the blog post called “Why you should not use BIT columns in MySQL” by Xaprb.
Another serious deficiency is the fact that a database dump will not export bit data as “0” or “1”. Depending on the tool used to dump and the MySQL server version you may find one of the following:

INSERT INTO `person` VALUES (1,"foo","\\0");
INSERT INTO `person` VALUES (2,"bar","☺");

or

INSERT INTO `person` VALUES (1,"foo"," ");
INSERT INTO `person` VALUES (2,"bar"," ");

The third field is of datatype BIT. Row number 1 is false and row number 2 is true. The problem with that is that some MySQL client tools cannot import such things. It gives you an ERROR 1064 (42000) at line 23: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ” at line 1

Solution #1

Hand edit the sql script and change all false bits to 0 and all true bits to 1, the script can be imported like a charm.

Solution #2

Extend MySQL5InnoDBDialect and make all BITs rendered as TinyInt(1). The code is very simple:

package com.foo.hibernate;

import java.sql.Types;
import org.hibernate.dialect.MySQL5InnoDBDialect;

public class MySQL5InnoDBDialectBitFixed extends MySQL5InnoDBDialect {

  public MySQL5InnoDBDialectBitFixed() {
    super();
    registerColumnType(Types.BIT, "tinyint(1)");
  }
  
}

Now when using the MySQL5InnoDBDialectBitFixed dialect, hbm2ddl will generate:

...
active tinyint(1) not null,
...

Until we get better 5.x MySQL versions, with better BIT support, this plan should do the job nicely.

Good luck

One of the (many) reasons I switched from JSP to FreeMarker is that I couldn’t achieve what I describe in this post. Tutorials or blog posts regarding this situation were never to be found, and in addition it was really hard to find anyone considering this issue a real problem.

The problem

Suppose we are building an issue tracking system. We have a rich Domain Model which includes entities such as User, Project, Account, Role etc. We’ve also got an abstract Issue object which is the root of the issue’s hierarchy. Concrete classes extending Issue include Bug, Feature, Request and Change. These 4 POJOs inherit common fields from Issue but add fields, methods and logic of their own.

Each of the issue’s subclass will need to have a slightly different HTML view. I tend to use the composite design pattern for my views, so I can break the HTML down to small reusable components. So it is obvious that we’re going to need 4 different views, one for each of them. Here are those issue rendering methods (presented in an imaginary pseudolanguage which combines EL, HTML and functions):

renderBug(bug) {
  <fieldset>
    <legend>Bug #${bug.id}</legend>
    <p>Author: ${bug.author}</p>
    <p>Date: ${bug.date}</p>
    <p>Description: ${bug.description}</p>
    <p>Steps to recreate bug: ${bug.stepsToRecreate}</p>
  </fieldset>
}

renderFeature(feature) {
  <fieldset>
    <legend>feature #${feature.id}</legend>
    <p>Author: ${feature.author}</p>
    <p>Date: ${feature.date}</p>
    <p>Description: ${feature.description}</p>
    <p>Related URL: ${feature.url}</p>
    <p>Screenshot upload: <img src="${feature.screenshot}" /></p>
  </fieldset>
}

...

Our DAO (probably called IssueDao) is going to fetch a Collection<Issue> (a bunch of issues) from the database for a particular use case. The runtime type of each of those entities cannot be Issue but it will be Bug, Feature, Request or Change. The problem is that we are presenting them altogether in the same screen, so in order to render them we have to write code such as this:

foreach(issues as issue) {
  if (issue instanceof Bug) renderBug(issue); continue;
  if (issue instanceof Feature) renderFeature(issue); continue;
  if (issue instanceof Request) renderRequest(issue); continue;
  if (issue instanceof Change) renderChange(issue);
}

If this doesn’t seem very bad to you, here is an actual view implementation of a slightly bigger hierarchy using JSP 2.0 Tag Files:

if (t instanceof ActivityInternal) {%><p:activityInternalView pojo="${t}" /><%;}
if (t instanceof ActivityExternal) {%><p:activityExternalView pojo="${t}" /><%;}
if (t instanceof ActivityMilestone) {%><p:activityMilestoneView pojo="${t}" /><%;}
if (t instanceof Review) {%><p:reviewView pojo="${t}" /><%;}
if (t instanceof PublicationReport) {%><p:publicationReportView pojo="${t}" /><%;}
if (t instanceof PublicationWebsite) {%><p:publicationWebsiteView pojo="${t}" /><%;}
if (t instanceof InfoConference) {%><p:infoConferenceView pojo="${t}" /><%;}
if (t instanceof InfoBase) {%><p:infoBaseView pojo="${t}" /><%;} 
if (t instanceof InfoChannel) {%><p:infoChannelView pojo="${t}" /><%;}
if (t instanceof Meeting) {%><p:meetingView pojo="${t}" /><%;}
if (t instanceof Interpretation) {%><p:interpretationView pojo="${t}" /><%;}
if (t instanceof BudgetItem) {%><p:budgetItemView pojo="${t}" /><%;}
if (t instanceof FocusGeneral) {%><p:focusGeneralView pojo="${t}" /><%;}
if (t instanceof FocusResearch) {%><p:focusResearchView pojo="${t}" /><%;}
if (t instanceof Risk) {%><p:riskView pojo="${t}" /><%;}
if (t instanceof QAChecklist) {%><p:QAChecklistView pojo="${t}" /><%;}
if (t instanceof TargetAudience) {%><p:targetAudienceView pojo="${t}" /><%;}
if (t instanceof LessonsLearned) {%><p:lessonsLearnedView pojo="${t}" /><%;}

If you still don’t think this is bad, you can stop reading ;)

What not to do

In a project I did in my early JSP days, what I did was to put all the view logic in the Java class! So it looked like this (this is actual Java):

public class Bug extends Issue {

  ...

  public String renderMe() {
    return "<fieldset><legend>" + this.getName() + "</legend>" + 
           "<p>Author: " + this.getAuthor() + "</p>" +
           "<p>Date: " + this.getDate() + "</p>" +
           "<p>Description: " + this.getDescription() + "</p>" +
           "</fieldset>";
  }
}

Although this type of code is a perfect candidate for The Daily WTF, the (only) advantage was that I could now render my pojos using (pseudocode):

foreach(issues as issue) {
  issue.renderMe();
}

The solution

It seems that all we want is the ability to construct and dynamically (reflectively in Java terms) call the appropriate render tag each time. In freemarker we define macros which look like this:

<#macro renderBug bug>
  <fieldset>
    <legend>Bug #${bug.id}</legend>
    <p>Author: ${bug.author}</p>
    <p>Date: ${bug.date}</p>
    <p>Description: ${bug.description}</p>
    <p>Steps to recreate bug: ${bug.stepsToRecreate}</p>
  </fieldset>
</#macro>

We need a way to call renderXXX where XXX is the short class name of the issue in question. And here is how you can do this in freemarker:

<#local macroname='render' + issue.class.name?split(".")?last />
<@.vars[macroname] issue />

For an issue of runtime type com.example.Foo, it concatenates the word “render” with “Foo” and calls the macro with that name. The magic happens with the help of the .vars special variable. It allows us to access variables by name. The full code now becomes:

<#macro renderIssue issue>
  <#local macroname='render' + issue.class.name?split(".")?last />
  <@.vars[macroname] issue />
</#macro>

<#list issues as issue>
  <@renderIssue issue />
</#list>

By the way, this capability is usually present in dynamic scripting languages. So for example there are many ways to do that in PHP.

using dynamic evaluation

$functionName = "renderBug";
$functionName($issue);

using eval

eval("renderBug($issue);");

using call_user_func (probably safest of all)

call_user_func("renderBug", $issue);

When an http request to your /rss page needs 400 milliseconds to complete, it seems obvious that your website could benefit from some caching. Ehcache is a well known cache provider, which most of us know from hibernate. Since we are already “bound” to ehcache, lets see how we can benefit from caching some dynamically generated pages:

web.xml

<filter>
  <filter-name>SimplePageCachingFilter</filter-name>
  <filter-class>net.sf.ehcache.constructs.web.filter.SimplePageCachingFilter</filter-class>
</filter>
<filter-mapping>
  <filter-name>SimplePageCachingFilter</filter-name>
  <url-pattern>/rss</url-pattern>
</filter-mapping>

We set up the SimplePageCachingFilter in the web.xml of the web application and map it to one or more url patterns or servlets. All requests to /rss will be intercepted by the SimplePageCachingFilter.

ehcache.xml

<ehcache>
  <diskStore path="java.io.tmpdir" />
  <cache name="SimplePageCachingFilter"
         maxElementsInMemory="0"
         eternal="false"
         timeToIdleSeconds="600"
         timeToLiveSeconds="600"
         overflowToDisk="true"/>        
</ehcache>

We then configure the cache region for pages. We don’t want any elements kept in memory. Everything should be written to disk at the java.io.tmpdir location. The cache expires every 10 minutes.

Now hitting http://example.com/rss (our default rss page) results in a cache miss. The content is being generated from scratch but before returning to the client, the filter stores it locally. The second time we’ll get a cache hit. The content now is being fetched from the cache and its very fast. 10 minutes later this cache element will be invalidated. Note that the default implementation uses the URI together with the query string to calculate the cache key, so /rss?type=news and /rss?type=forum will result in two different cache elements.