So, we have been working on this application… web based, data driven, to be used in “social context”. I think today there’s nothing sexier than using the word social.

We designed this application nicely, fantastic by many ways. We were sure of putting this up on a public cloud, just that we were not sure of which one: EC2 or GAE. One of the most riskiest questions to keep open… but really, we were not sure. The only thing we were sure of was Java + jQuery + HTML5 (as much as possible).

EC2 gives you IaaS but then you must manage all the servers, load balancing, failovers. Even if you happen to choose BeanStalk, a lot of work has to be done, specifically database. Thankfully, we now have ElastiCache (launched around August 2011, IIRC) otherwise managing a cluster of Memcache was yet another task for the admins.

GAE provides you PaaS, so you don’t need to worry about doing all the stuff in previous paragraph yourself. Except that you have strong restrictions… a very small subset of Java SE library – specifically, not threads or direct sockets, no JDBC and Datastore, though amazing, is not an easy meat to digest for those not used to. And above all, the server side execution must complete in 30 seconds (now on what are known are front-end servers).

So, we designed our own data-access abstraction so that we can switch EC2 <=> GAE without any hassles. It took a while, but it’s a great thing that we have.

After writing core entities and business logic layer, it was time to start pushing it all up to the user interface. The dashboard – the landing page after login – has to get updates from various “components” in the system. All components worked great, except that the dashboard took somewhere around 1200ms to load for the first time. Note that most of the public resources (images, JavaScript, CSS) were loaded from GAE CDNs.

Just not acceptable!

So, what do we do? Except for one of the menus, all of the data is user specific, and may not be a good idea to cache it all. Above all, we had a restriction that GAE Memcache servers cannot have more than 1MB entity.

What we were damn sure about is that there’s nothing much that we can do to reduce the time to compute. Efficient datastore retrieval, great entity structures, had denormalized and duplicated the data a lot for faster retrieval… nothing much you can do.

Ajax. Great! We’d already been using it. The first thing was obvious – can we just “loading…” components with Ajax calls on the dashboard and then fetch the data.

But then, how many hits to the server? One of the key things in network interactions is use of chunky messages rather than chatty messages. Additionally, HTTP/1.1 says the browsers should not open more than 2 simultaneous connections to the server. And what more… these components are completely independent of each other, how do we combine then?

So, we wrote a mini-layer: For Combine-Split-Execute-Combine-Split-Display Process.

1. At the client, we’ll combine two requests into one, send it to the server.
2. On the server, the data will be split and two business operations will be executed. On GAE, I can’t start Threads, no Futures… so, one limitation we hit there. But that’s fine.
3. Combine the results from the business operations and respond back to the client.
4. Client will split the results and fire them off to the corresponding requesters.
5. There were some more optimizations that we did on serialized data model (messaging model) besides putting off some operations in GAE Queue

Voila! The results of all this was that the landing page is not loaded in less than 18ms (sans network latency) and a much better user experience.

But it’s not the end of the day yet… we’re tuning the application further more.