28.01.2012

How to use Velocity to generate code in GWT

The JSON/XML mapper Piriti is heavily based on deferred binding and code generation. As I started to implement code generators in Piriti I looked around how other projects deal with it and read thru the ofiicial documentation on the GWT site. The usual way to generate code is to extend com.google.gwt.core.ext.Generator and then call GeneratorContext.tryCreate(TreeLogger, String, String). The returned PrintWriter is then often wrapped into somekind of IndentedWriter like the one used by GWT itself. This class adds methods to indent and unindent code and supports printf() like behaviour. Finally the writer is used to generate all code. This in turn results in code which looks like that:
writer.write("%s %s = null;", parameterizedValueType, value);
writer.write("List<Element> %s = filterElements(element.selectNodes(\"%s\"));", elements, property.getPathOrName());
writer.write("if (!%s.isEmpty()) {", elements);
writer.indent();
writer.write("%s = new %s<%s>();", value, collectionImplementation, elementType);
if (property.isConverter()) {
  // even more writer.write() statements
}
writer.write("for (Element currentElement : %s) {", elements);
writer.indent();
writer.write("%s currentValue = null;", elementType);
writer.write("XmlReader<%1$s> currentReader = xmlRegistry.getReader(%1$s.class);", elementType);
writer.write("if (currentReader != null) {");
writer.indent();
writer.write("currentValue = currentReader.read(currentElement);");
writer.outdent();
writer.write("}");
writer.write("if (currentValue != null) {");
writer.indent();
writer.write("%s.add(currentValue);", value);
writer.outdent();
writer.write("}");
writer.outdent();
writer.write("}");
writer.outdent();
writer.write("}");
I can't help, but this code somehow reminds me of the old times, where we generated HTML code in servlets. This approach might work as long as the amount of generated code is small. In Piriti the code generation process is somewhat complex and distributed over several classes. Changing the generated code became very difficult and error-prone. Only the correct use of writer.indent() and writer.outdent() is not a trivial task. To some extent this problem can be solved by the use of an abstract base class, which contains common code. The generated class would extend from the abstract base class. But at the end of the day you have to generate some code in the concrete subclass.

Velocity to the rescue
Velocity is a Java-based template engine. It permits anyone to use a simple yet powerful template language to reference objects defined in Java code. Velocity supports for loops, if-then-else conditions and custom macros. Templates can include other templates. This way you can put common code in extra templates and reuse it in other templates. Velocity is mainly used in web projects for HTML generation. Another common use case is to generate email bodies. But there's no reason not to use Velocity for code generation in GWT.

Doing so the above code snippet becomes something like that:
$parameterizedValueType $value = null;
List<Element> $elements = filterElements(element.selectNodes("$property.pathOrName"));
if (!${elements}.isEmpty()) {
  $value = new $collectionImplementation<$elementType>();
  #if ($property.converter) #createConverter() #end
  for (Element currentElement : $elements) {
    $elementType currentValue = null;
    XmlReader<$elementType> currentReader = xmlRegistry.getReader(${elementType}.class);
    if (currentReader != null) {
      currentValue = currentReader.read(currentElement);
    }
    if (currentValue != null) {
      ${value}.add(currentValue);
    }
  }
}
If you compare the two code snippets you get the idea! The velocity based code is much more readable. As you can see the velocity template contains variable references like $elements. Before the template is rendered all necessary variables must be put into the so-called Velocity context which is more or less a big map. If the variable refers to a java object you can use its properties and even call methods.

To use Velocity for code generation you have to setup the Velocity engine, create the Velocity context and merge the template. In Piriti the engine is configured with the following properties:
velocimacro.library = name/pehl/piriti/rebind/propertyMacros.vm
runtime.log.logsystem.class = name.pehl.piriti.rebind.VelocityLogger
input.encoding = UTF-8
output.encoding = UTF-8
resource.manager.logwhenfound = true
resource.manager.defaultcache.size = 0
resource.loader = cp
cp.resource.loader.class = org.apache.velocity.runtime.resource.loader.ClasspathResourceLoader
cp.resource.loader.cache = false
Finally the code generation process is reduced to the following lines:
PrintWriter printWriter = generatorContext.tryCreate(treeLogger, somePackage, implName);
if (printWriter != null)
{
  VelocityContext context = new VelocityContext();
  // Put all neccesarry objects into the velocity context
  context.put("foo", ...);

  InputStream inputStream = this.getClass().getClassLoader().getResourceAsStream(“name/pehl/piriti/rebind/velocity.properties”);
  Properties properties = new Properties();
  properties.load(inputStream);
  VelocityEngine velocityEngine = new VelocityEngine(properties);
  velocityEngine.mergeTemplate("someTemplate.vm", "UTF-8", context, printWriter);
  generatorContext.commit(treeLogger, printWriter);
}
If you want to delve deeper into the code generation process in Piriti, check out the trunk and take a look into the code.

09.05.2011

Announcement: Piriti 0.7.0b1

I'm pleased to announce the first beta of the upcoming Piriti 0.7.0 release. Piriti 0.7.0 is a major release with breaking API changes, lots of new features and bugfixes.

Breaking API Changes:
  • Piriti requires GWT 2.2 and GIN 1.5
  • Dropped GXT support
  • Removed @Json, @JsonMappings, @Xml and @XmlMappings annotations
New Features:
  • Devided Piriti into different modules:
    • name.pehl.piriti.commons.Commons
    • name.pehl.piriti.converter.Converter
    • name.pehl.piriti.json.JSON
    • name.pehl.piriti.property.Property
    • name.pehl.piriti.xml.XML
  • Simplified mapping setup: All properties in a POJO hirarchy are now mapped by default
  • Added new annotations to overide default behaviour:
    • @Order
    • @Path
    • @Format
    • @Native
    • @Transient
    • @CreateWith
    • @MapUpTo
  • Added @Mappings and @Mapping annotation for external mappings (JSON and XML)
  • Added support for polymorhic assoziations
  • Converters can now be used for any type
  • Added XML serialisation (not yet implemented, but scheduled for the 0.7.0 release)
  • Added support for IDs and IDREFs in JSON (not yet implemented, but scheduled for the 0.7.0 release)
Bugfixes:
  • Fixed problems when mapping collection implementations
  • GWT.create() is now used instead of new operator
I also restructured the wiki and added a FAQ and a comparison to other JSON / XML mappers. Feel free to visit http://code.google.com/p/piriti and test the new release.

03.12.2010

Taoki and TiRe mentioned in GWT-GAE-Book

Taoki and TiRe are mentioned in the free online book on GWT and AppEngine
development by Marius Andreiana.

The book covers also a lot of GWTP and its core examples are great illustrations for many features of the framework! It also makes use of Gin, Guice, Twig-Persist, Mockito and many other useful tools.

Go check it out: http://code.google.com/p/gwt-gae-book/

Many thanks to Marius for such a valuable resource, and congratulations on publishing it!

27.10.2010

Announcement: Piriti 0.6

I'm pleased to announce Piriti 0.6. This is a major update which adds support for JSONPath expressions. Please take a look at the release notes for further info.

03.10.2010

TiRe - Time recording made easy

Currently I'm developing a personal time recording tool based on GWT and deployed at Google App Engine. Besides being useful I wanted to use the latest web techniques / frameworks in a real application. TiRe uses the following stack:
Work is still in progress. But you can take a first look at TiRe following the link below. One last note: TiRe uses some of the new HTML5 / CSS3 features and therefore requires a modern browser.

http://tire-d8.appspot.com/

13.09.2010

Paging in resources

A common requirement for resources which return large collections of records is paging. Paging can be implemented in many different ways. See the following pages as an entry point to the discussion:
In the following sections I will introduce three different solutions. All of them are implemented in Taoki - a small extension for Restlet.

Template Parameter

This implementation uses parts of the url to carry the paging information:
GET /books
GET /books/0/50
GET /books/0/50/author/asc
The advantage of this implementation is that the resource can be cached by proxies. The drawback is that a lot of template parameters are "wasted". For example /books/report/{quarter} won't be available as route, since it is occupied by /books/{offset}/{pageSize}

Query Parameter

This implementation uses query parameter.
GET /books
GET /books?offset=0&limit=50
GET /books?offset=0&limit=50&sortField=author&sortDir=asc
Most proxies won't cache resources which include parameters, but there are no template parameters wasted.

Custom Header

This implementation uses the custom header Item-Range.
GET /books
GET /books
Item-Range: items=0-49
GET /books
Item-Range: items=0-49;author:desc
It combines the advantages of the other two implementations. A minor drawback of this solution is that the paging information is no longer visible in the URL.

07.07.2010

Announcement: Piriti 0.4

I'm pleased to announce Piriti 0.4. This is a major update which adds support for namespaces in XML document and XPath expressions. Please take a look at the release notes for further info.