Using AWS to Store Large Media

Using AWS to store large media is  a sensible decision if you are paying for web space by the megabyte.

Let’s take a step back and compare various types of cloud storage and take a look at the type of media it may be sensible to host “offsite”.

What is Cloud Hosting?

This is the easy version. Cloud has been around for ten years at least and the business model keeps evolving. Cloud computing is the delivery of on-demand computing services — from applications to storage and processing power on a pay as you go tariff.

Before Cloud, if you wanted to host something offsite you had to rent physical space in a server farm and the spare disk space remained unused. As did the processor when no-one was using the web site. Then virtualisation took off.

Virtualisation enables a computer’s resources to be sliced and diced and offered as virtual resources. As a buyer, I can see how much disk space I have available, but that may be a subset of a much larger disk.

Virtualisation enabled the larger server vendors to maximise the utilisation of their physical resources in order to basically sell spare processor cycles and disk space to more than one person. For a company like Amazon or Google, this enabled them to maximiser their return on investment and for the customers, guaranteed levels of service and new technology. Win Win.

Soon, Cloud vendors started to differentiate themselves by offering specialist services. Web Hosting, Archiving, Streaming etc. We use two cloud vendors for our storage needs, Amazon (AWS) and Backblaze. They have very different characteristics. Backblaze is a lot cheaper, but is primarily concerned with back ups and archives. We use it to backup the digital assets that we cannot afford to lose. We are using AWS to store large media.

Web Site Hosting

Web site hosting is appearing as a specialist type of cloud service. In theory, AWS can be used to host web sites, however when there are providers around who can provide specialised support, optimised hosting, CDN’s and sophisticated cacheing services it makes little sense for the small business to host a web site there.

We often use Siteground for our WordPress hosting because they are WordPress specialists and offer very knowledgeable support. We use Krystal for Moodle for similar reasons. Web Site hosting is optimised for speed, and it makes a real difference. We were able to achieve better than 100% performance improvements by moving our web sites away from a more traditional type of server farm. The downside is that our hosting requirements and those we offer our clients are metered on storage and iNodes.

For storage this means that if we exceed a certain number of gigabytes, we have to move up a tier and pay for more space. iNodes are file system objects, so if I replace a picture (1 iNode) with a 360° virtual tour, I’m increasing my iNode use by potentially thousands. This is why we host our virtual tours on AWS. The individual components are small, but there are a lot of them.

Media Types and Storage Requirements

Our Photography and Video business, Helter Skelter Studios is very resource hungry. We display videos and virtual tours on the site. Both of these media types require large amounts of storage.

These are the media types we are interested in.

• Images
  Graphics and Photographs can be optimised at source – a good web designer will provide this service as part of the implementation. There are also wordpress plugins that can ‘shrink’ images. The best advice is to aim for a megabyte or less for the image you upload to the web site. Gifs are  tiny and can be used for small graphical images, jpegs are a compressed format that is very popular for photographs. png supports transparent backgrounds.
• Video
  Moving pictures, animation and vide require streaming. You can serve video from a non optimised server but you wouldn’t choose to. This is why we host all of our videos on Vimeo. The difference in performance between serving a video directly from the website and using a specialist service like vimeo is dramatic. Vimeo throttles the resolution denoting on bandwidth, thereby achieving maximum throughput at all times.
• Virtual Tours
  Virtual tours have been an issue for us and this is where using AWS to store large media makes perfect sense. We designed and host the Funky Frankwell web site and initially found performance to be diabolical. This in spite off the fact that the tour creation process divides the enormous hi-res panoramas into kb sized tiles so that it serves faster. This and the fact that the storage requirements for virtual tours are large (we have a couple that are approaching gigabyte size) persuaded us to migrate to AWS to host and serve the tours.

We opt to serve video from a specialist provider, host images on the same server as the website and our resource hungry virtual tours from a Cloud Provider. Amazon Web Services.

Using AWS to Store Large Media

Setting up a cloud account for the purposes of hosting media is pretty straightforward. The Getting Started Guide at Amazon is easy to understand – where they refer to “buckets” that is your “on demand” web space, where they refer to “objects” that can be a directory.

If you need a step by step guide, let us know and we’ll write one!

Further Reading

Why is SSL Important?

How to Choose the Best Web Hosting