Amazon S3 for Application Storage

Hosting content in the cloud can seem like a daunting task – but Amazon S3 makes life very simple for those needing cloud based storage.  At the heart of Amazon’s cloud based solutions, they provide three principal services.

  • Amazon EC2
  • Amazon SQS
  • Amazon S3

Amazon EC2 (East Coast 2) is a virtual server farm where you can run virtual servers (other Amazon virtual farms have other names such as AP for their Asia-Pacific data center).  Amazon SQS which stands for Simplified Query Service which is their database equivalent.  Lastly we have Amazon S3 which stands for Simplified Storage Service which is their content hosting and delivery service.  In this article I want to focus on the Amazon S3 service and how you can leverage it for your application, game or service.

Amazon S3 at its core is actually a very straight forward service.  Think of it like an FTP server where you can perform get, put and delete operations on files.  Each file within S3 is considered as an object and has a set of permissions and meta data that can be set to help control it.

The real golden goose of the service is that you can use it for many purposes.  Here is a quick list of how it can be used:

  • Content Delivery Network (CDN)
  • Data Backup Location
  • User Content Warehouse
  • Static File Server

Content Delivery Network (CDN)

If your application has a lot of media or large file size content then you can use S3 as a cost saver to serve those expensive static assets.  Amazon allows you to set public viewing permissions as well as gives you control to header information to best serve your content.  In essence your first step into the Content Delivery Network or CDN world.

Data Backup Location

If your application requires lengthy backups that need to be preserved then Amazon S3 can be a very useful solution.  Consider a scenario where you are required to create hourly backups of your database or data storage pool.  Let’s assume a size of 10 gb for that data set.  Due to the nature of the changes (or requirements) you have to take a full snapshot and not an incremental backup.  The space you would need for a full days worth of backups is 240gb.  For a month you would need roughly 7.2 tb.  It’s not hard to see how quickly the price of this backup strategy can add up.  Utilizing Amazon S3 you can quickly upload those backup files as compressed objects and forget literally forget about out them.  No more changing out tapes, plugging in external hard drives, etc…

User Content Warehouse

Many applications allow for users to submit images, videos, etc.  The challenge is that as your web application grows your web server farm grows to be hosted off of multiple servers.  Saving the same image across a pool of web nodes is not really a great way to scale a UGC (user-generated content) based feature. Amazon S3 is a great location to store this data as it provides an easy put operation, allows you to set permissions on the file and then allows for a CDN based delivery model.

Static File Server

When your application goes into maintenance mode you typically want to provide a good experience to the users and display some sort of message about the maintenance.  Well – what if the maintenance involves taking the web servers offline and therefore no pages can be recorded.  By lowering the TTL on you DNS settings ahead of your maintenance, you could actually direct users to a static web page served directly from S3.  This page can be straight HTML, images, etc… (no scripting such as PHP, etc…).  When the maintenance is done simple update the DNS records back to your server farm.

Pricing

Amazon performs its pricing structure on a broken down model.  Meaning that it charges based upon the put actions, the get actions and the monthly storage size.  While at first it might feel like you are getting penny pinched – this model actually allows for you to control your expenses as you really only pay for what you use.

If your usage is a set and forget operation such as the backup model above then your biggest charges will be on the long term storage.  If you utilize s3 as a content delivery network then you will find the majority of your cost going to the get operations. In essence you are really on a “pay for only what you use” model instead of a blanket this is your monthly price model.

In the future we can dive more into other services and how you can use them to launch you online game or web service.

Comments

comments

Posted in and tagged , , .