Archive for the ‘sqs’ Category

Web Application Infrastructure

February 10th, 2008 Comments off

Congratulations: after spending plenty of time on creating and polishing your perfect web application in your development environment, it is finally meant to be released for public consumption.

If it proves to be popular (or simply gets slahdotted), the application is in for a first real-world stress test. A single server will be overloaded pretty fast, but hosting the application on a server farm for traffic that might never come will get expensive very soon.

Amazons Elastic Compute Cloud (EC2) to the rescue. EC2 offers great flexibility for running web applications. Pre-created or custom application server images (so called AMI – Amazon Machine Images) based on Linux and Xen allow fine-grained control over available software. The web application might be running on a single server, but if the load is rising, the load may be dispersed by starting new server instances. If the load decreases, the number of servers may be reduced again.

The possibility to provide your own server images (AMIs) allows for server instances with different roles: one or more database servers might provide central storage services for your application servers while another server instance simply processes incoming and outgoing emails. Network security is ensured and can be specified with a fine-grained security policy.

Amazon bills by running instance per hour with three different hardware configurations available starting at 0.10 USD per hour. A single server of the lowest category amounts to roughly 70 USD per month, which is probably more expensive than other hosters. But the flexibility to balance load by starting or stopping server instances is priceless.

Web applications need one or more servers to run on, but what about storage? Each server provided by EC2 offers 160 GB (or more, depending on your hardware configuration) of space, but that storage space is only available as long as the server instance is running. Again Amazon provides a solution: its Simple Storage Service (S3). Space (almost) without limit at 0.15 USD per GB and month (+ data transfer, when accessed from outside Amazon’s network). About the only drawback is that the storage can not be accessed like a ordinary file system. This is due to the distributed architecture, which in turn provides high availability and safety for your precious data.

Communication between your servers may be performed using a third service offered by Amazon: its Simple Queue Service (SQS) allows message to be sent using one or more queues, which provides for an elegant load balancing solution for distributed services.

A recently established service, Amazon SimpleDB offers structured data storage. The service is still in limited beta, but seems to cater for your basic database needs without using a full-blown database server of your own.

All these services can be controlled and accessed through web service APIs with a wide variety of clients available in all major programming languages. For Java, these libraries offer all functionality exposed by Amazon’s services:

  • The excellent JetS3t library (Apache license) provides access to Amazon S3
  • Typica (Apache license) exposes the functionality of EC2, SQS and SimpleDB for Java developers

Toolkits and support for other languages can be found on the Amazon Resource Center pages.

All in all, Amazons EC2, SQS, and S3 provide the perfect environment for
web applications: great flexibilty for a reasonable price with plenty of room to grow according to your needs.

For an example of deploying distributed J2EE Applications using Amazon EC2 see this article, another example of using Amazon SQS, EC2, and S3 for distributed processing is described in an article by David Kavanagh, author of the excellent Java library Typica (see above).

Categories: amazon, ec2, infrastructure, jets3t, s3, simpledb, sqs, typica Tags:
Fork me on GitHub