Tag Archives: App Engine

WordPress for Google App Engine?

I love WordPress. A big fan. But then I once-upon-a-time used to be a huge fan of PHP, and then I’ve grown up. I know people will kill me for that statement. 😉

But the fact of the matter is PHP still has serious issues. And most recently, I can’t use it to run WordPress at Google App Engine, something that’s amazingly great for the starters who want to setup quick-and-not-so-dirty app.

I thought… would it be good to have WordPress for GAE? I came across Vosao – Open Source CMS for GAE/J. And I loved it immediately. Has a long way to go, and I do not see any active development on the same… surprised, what happened to the team.

StackOverflow has a question – What CMS runs on Google App Engine, but I personally didn’t like any of them.

And then I thought again… how about writing my own CMS with persistence in a Non-Relational Store with plug-in model so that I can run it on any store, be it GAE’s Data Store, MongoDB or Amazon DynamoDB or whatever.

Let me see if I can hold on my thoughts and really get on with it… At the moment, no promises! :-)

Google App Engine and GWT – Multi-Project Application

While working with Google Web Toolkit (GWT), it is a very common scenario to split the main application into multiple sub-projects and then use IDE to do a combined build and compile. However, the problem arises in scenarios where the GWT-RPC services and their implementations are kept in separate projects.

Recently working on an application, I structured it as follows:

  • MainApp: The final GWT+GAE application that will be deployed
  • AppCommon: Common classes to be used across projects
  • AppServices: Service definitions. It comprises of the GWT-RPC service definitions and their Async counterparts.
  • MainApp project contains the implementations of the services defined in the AppServices project.

When I “GWT Compile Project” or “Deploy App Engine Project“, the “shared” or “server” code of the dependencies is never copied to the war/WEB-INF/classes folder. Only the compiled JavaScript code is compiled.

So, I thought of automating all the stuff using Ant files.

GWT app-creator tool creates the “build.xml” for Ant and for GAE, there are guidelines on the web, for example here.

At an outline, the following needs to be done during the project build:

  • Enlist all the dependencies in order.
    If there are five projects – P1, P2, P3, P4, P5 – my script cannot identify which one depends on the other. It’s the developer’s responsibility to write them in the order so that independent or least dependent projects are listed and hence compiled first.
  • Compile all the projects in order with appropriate class-path…
  • Make a jar comprising of all the compiled class files and push it to war/WEB-INF/lib folder.
  • Compile the GWT module, as usual
  • Publish the application to GAE, as usual

I have made used of the following Ant tasks:

  • foreach from ant-contrib. You can download the jar from Sourceforge. Project home page is at http://ant-contrib.sourceforge.net.
  • Standard property, mkdir, delete, javac, copy etc tasks
  • App Engine task appcfg.

First step is to define a property, I named it projects, enlisting all projects in order.

<property name="projects" value="AppCore,AppServices,AppOne,AppTwo" />

Next is to expand the default task javac to compile all the dependencies as well. To take care of dynamic classpath entries arising because of the multiple projects, I copy all the class files to the <main-app>/war/WEB-INF/classes folder and that’s my standard classpath.

<target name="javac" depends="libs" description="Compile java source">
  <foreach list='${projects}' param="project.dir" target='compileDeps' />
  <mkdir dir="war/WEB-INF/classes" />
  <javac srcdir="src" includes="**" encoding="utf-8" destdir="war/WEB-INF/classes"
    source="1.5" target="1.5" nowarn="true" debug="true" debuglevel="lines,vars,source">
    <classpath refid="project.class.path" />
  <copy todir="war/WEB-INF/classes">
    <fileset dir="src" excludes="**/*.java" />

What this target does is compile all the dependencies before compiling the main application project.

Defining the compileDeps target is more straightforward now, noting that we have a helping hand from foreach task. Here’s how it looks like.

<target name='compileDeps' description='Compiles dependencies'>
  <property name="pdir" location="../${project.dir}" />
  <property name="cdir" location="." />
  <mkdir dir="${pdir}/war/WEB-INF/classes" />
  <javac srcdir="${pdir}/src" includes="**" encoding="utf-8" destdir="${pdir}/war/WEB-INF/classes"
    source="1.5" target="1.5" nowarn="true" debug="true"
    <classpath refid="project.class.path" />
  <copy todir="${pdir}/war/WEB-INF/classes">
    <fileset dir="${pdir}/src" excludes="**/*.java" />
  <copy todir="${cdir}/war/WEB-INF/classes">
    <fileset dir="${pdir}/war/WEB-INF/classes" excludes="**/*.java" />

This task compiles the dependencies, and copies all resources (non-java files) from the corresponding src folder and all binaries from the corresponding war/WEB-INF/classes folder to the <main-app>/war/WEB-INF/classes folder.

I further expanded the gwtc target to make a jar file rather than individual class files in the WEB-INF folder. Here’s the code for that.

<target name="gwtc" depends="javac" description="GWT compile to JavaScript">
  <jar destfile="war/WEB-INF/lib/main-app.jar">
    <fileset dir='war/WEB-INF/classes' />
  <delete dir="war/WEB-INF/classes" failonerror="false" />
  <java failonerror="true" fork="true" classname="com.google.gwt.dev.Compiler">...</java>

And that’s it!

Publishing the application to GAE requires no special change. All files are right there! Just use the <appcfg> task.

Google App Engine – Dealing with large number of files

One of the most common problem that one has to deal while deploying a large scale application at Google App Engine is the file limit – in terms of number of files.

As per the issue reported at http://code.google.com/p/googleappengine/issues/detail?id=161, the limit has been increased from 1000 to 3000, but there’s catch. Any static file (non-JSP files outside the WEB-INF folder, unless explicitly excluded in appengine-web.xml) are counted twice. Why? They’re pushed into a special folder __static__ and all such files are counted twice.

I looked at deploying the SmartGWT showcase on GAE and the “appcfg upload” gave an error about having over 9000 files. I was shocked! Literally.

And then I came across a simple yet powerful idea – zip all the static files and use a servlet to extract and serve on the fly. The solution servlet was uploaded by sindhom at http://code.google.com/p/googleappengine/issues/detail?id=161#c68.

Take that as a base, I made an update – again simple yet probably very useful. Look at the “If-Modified-Since” header and respond with with a 304 or 200 depending upon if the entry in the zip file is updated or an old one. There’s one thing hardcoded in the my solution – the Cache-Control / Expires timeline is hardcoded to 365 days, and it can be easily moved into the configuration.

The solution is mentioned at http://code.google.com/p/googleappengine/issues/detail?id=161#c88. The SmartGWT showcase is available at http://1.latest.mastergaurav.appspot.com, along with the complete code of the servlet at http://1.latest.mastergaurav.appspot.com/UnzipperServlet.java.txt.