Grails Monitor plugin 0.1 released

February 20th, 2008 Comments off

Just in time for the upcoming 2008 Groovy/Grails Experience, I released my Grails Monitor plugin in version 0.1 (Apache License).

It’s basically a preview version, but for anybody who was dying to get his or her hands on it, the Grails wiki has all the details on how to install and use the Monitor plugin. For an introduction, see my previous post.

Installation is still a little clumsy as you have to jump through a few hoops because some patches to Grails and the Quartz plugin are needed in order to get the monitor plugin installed and running.

Lots of features are still missing, but for a first version it’s not too bad. I’m open to suggestions, criticism and usability reports. Of course, patches are most welcome 🙂

Happy monitoring,

Wolfgang

Categories: grails, monitoring, plugin, release Tags:

Introducing the Grails Monitor Plugin

February 14th, 2008 13 comments

Grails is a fantastic web framework based on Groovy, which brings the convention-over-configuration paradigm pioneered by Ruby on Rails to the Java world. There is currently a lot of buzz around Grails and so far I had a lot of fun working with the framework.

One aspect of Grails is easy extensibility using plugins. There are already some plugins, which cover a lot of functionality. I have been working on a plugin of my own, a generic monitoring plugin, which will be released shortly under a Apache License.

The monitoring plugin provides pre-defined monitors for many aspects of a web application. Additionally it is really easy to define your own monitors to track usage of application-specific metrics.

The plugin is based around rr4j, which is a Java port of the well-known RRDTool and uses its data storage and graphing engine. It requires the Quartz Plugin (with my patches for GRAILSPLUGINS-190 and GRAILSPLUGINS-213) and Java 1.5 or greater.

So, what does it look like? Some Screenshots (click for bigger image):

The monitor view showing the System group:

The Web group with the Requests graphs:

The Web group with the UserAgent graphs:

So, how do I define a monitor of my own? Following the Grails philosophy of convention over configuration, the monitor plugin defines its own artefact type: Monitor.

Creating an application-specific monitor involves defining a class ending in ‘Monitor’ in the ./grails-app/monitor/ directory and adding some simple elements:


class SessionMonitor implements HttpSessionListener {
def activeSessions = 0L

static monitorName = 'Sessions'
static monitorGroup = 'Web'
static monitorDescription = """The ${monitorName} monitor contains
information regarding active sessions."""

static monitorDefs = {
'activeSessions'( type:'gauge', aggregation:'avg', min:0) {
description = "Number of active sessions"
}
}

static monitorGraphDefs = {
'activeSessions'() {
title = '''Active sessions'''
description = title

// graph sources
source(id:'activeSessions', metric:'activeSessions', aggregation:'average')

// what to draw
area(id:'activeSessions', legend:true)
}
}

void sessionCreated(HttpSessionEvent se) {
activeSessions++
}

void sessionDestroyed(HttpSessionEvent se) {
if (activeSessions > 0) {
activeSessions--
}
}
}

The monitor class specifies a name (monitorName), a group it belongs to (monitorGroup, e.g. ‘Web’, ‘System’, or ‘Application’) and optionally a description (monitorDescription). Two DSLs, one for metrics (monitorDefs), one for graphs (monitorGraphDefs), allow to specify the monitored values and what the graphs should look like. Values are collected periodically (currently every minute) using a Quartz-controlled job.

Details will follow once the plugin is released. It still needs some work for the UI and behind the scenes, but a first version will be available soon.

Roadmap

I have plenty of ideas for additional features and monitors.

Features:

JMX export
XML/JSON/CSV export
Support for non-numeric data (e.g. Java version, OS name, hostname, …)
Possibly storage backends other than RRD4J (e.g. a database)
Notification via EMail when certain thresholds are exceeded
Integration of other graphing solutions, like JFreeChart

Monitors:

Count per Domain object
Requests by controller/action
Database/GORM query stats (using Hibernate statistics)
Special monitors for some databases (MySQL “SHOW STATS”, …)
Security monitor for JSecurity/ACEGI/CAS plugins
Monitor for Quartz jobs (job runs, min/max/avg duration)

Basically, the idea is, that every plugin would provide its own set of monitors for the contributed functionality.

I you have any other idea, advice, or feature requests please feel free to post a comment below.

Categories: grails, monitoring, plugin Tags:

Web Application Infrastructure

February 10th, 2008 Comments off

Congratulations: after spending plenty of time on creating and polishing your perfect web application in your development environment, it is finally meant to be released for public consumption.

If it proves to be popular (or simply gets slahdotted), the application is in for a first real-world stress test. A single server will be overloaded pretty fast, but hosting the application on a server farm for traffic that might never come will get expensive very soon.

Amazons Elastic Compute Cloud (EC2) to the rescue. EC2 offers great flexibility for running web applications. Pre-created or custom application server images (so called AMI – Amazon Machine Images) based on Linux and Xen allow fine-grained control over available software. The web application might be running on a single server, but if the load is rising, the load may be dispersed by starting new server instances. If the load decreases, the number of servers may be reduced again.

The possibility to provide your own server images (AMIs) allows for server instances with different roles: one or more database servers might provide central storage services for your application servers while another server instance simply processes incoming and outgoing emails. Network security is ensured and can be specified with a fine-grained security policy.

Amazon bills by running instance per hour with three different hardware configurations available starting at 0.10 USD per hour. A single server of the lowest category amounts to roughly 70 USD per month, which is probably more expensive than other hosters. But the flexibility to balance load by starting or stopping server instances is priceless.

Web applications need one or more servers to run on, but what about storage? Each server provided by EC2 offers 160 GB (or more, depending on your hardware configuration) of space, but that storage space is only available as long as the server instance is running. Again Amazon provides a solution: its Simple Storage Service (S3). Space (almost) without limit at 0.15 USD per GB and month (+ data transfer, when accessed from outside Amazon’s network). About the only drawback is that the storage can not be accessed like a ordinary file system. This is due to the distributed architecture, which in turn provides high availability and safety for your precious data.

Communication between your servers may be performed using a third service offered by Amazon: its Simple Queue Service (SQS) allows message to be sent using one or more queues, which provides for an elegant load balancing solution for distributed services.

A recently established service, Amazon SimpleDB offers structured data storage. The service is still in limited beta, but seems to cater for your basic database needs without using a full-blown database server of your own.

All these services can be controlled and accessed through web service APIs with a wide variety of clients available in all major programming languages. For Java, these libraries offer all functionality exposed by Amazon’s services:

  • The excellent JetS3t library (Apache license) provides access to Amazon S3
  • Typica (Apache license) exposes the functionality of EC2, SQS and SimpleDB for Java developers

Toolkits and support for other languages can be found on the Amazon Resource Center pages.

All in all, Amazons EC2, SQS, and S3 provide the perfect environment for
web applications: great flexibilty for a reasonable price with plenty of room to grow according to your needs.

For an example of deploying distributed J2EE Applications using Amazon EC2 see this article, another example of using Amazon SQS, EC2, and S3 for distributed processing is described in an article by David Kavanagh, author of the excellent Java library Typica (see above).

Categories: amazon, ec2, infrastructure, jets3t, s3, simpledb, sqs, typica Tags:

Hello, world!

February 10th, 2008 Comments off

Trying to build the perfect web application involves a lot of effort. Technical aspects as well as design have to be considered.

This blog is basically meant to track my work in progress and helps me to collects links, thoughts and findings regarding web applications, including application frameworks, infrastructure, and network related information. Maybe it even proves to be useful to other people as well.

So, to myself, happy blogging 🙂

Categories: general Tags: