Working with Object Storage (AWS S3 and Glacier) in RONIN
We've been busy here at RONIN, simplifying Amazon's S3, versioning and Glacier, so that everyone can benefit from what the cloud has to offer, not just tech savvy nerds like us.
One of the hardest parts of Cloud Computing is getting your data into the cloud and being able to utilise it effectively across multiple operating systems, while collaborating with teams that have different goals, then storing it in the most cost effective way possible. That's a big task for even the most advanced cloud computing users.
We've been busy here at RONIN, simplifying that process, so that everyone can benefit from what the cloud has to offer, not just tech savvy nerds like us.
Quick Start - for people with pocket protecters.
Windows / Mac users
Click here to see how to connect with Cyberduck
Linux users
Click here to connect with AWS Command Line Interface
Detailed Guide - for the rest of us.
What the heck is Object Storage?
Object Storage is, well, just a type of storage for your stuff. Similar to file storage, like what you would have on your computer right now, there are multiple advantages to using object storage instead of file storage in the cloud.
An example might be that Jose has some awesome data that he wants to backup multiple times incase his experiments go wrong. This is how it would look in traditional folder storage
Jose copies his awesome data into a new folder to back it up. Everytime he changes some of the data, he needs to back it all up again to keep it safe. And he changes it again. Now there is three very similar copies of Jose's data in storage. With any storage, you pay for the total amount that is stored, and this means the team pays for three exact copies of Jose's data.
Now imagine Jose backing up results every hour. The more he copies the data, the more he pays for storage.
Now here's where Object Storage differs. It's worth noting that every file put in the Object Store is automatically duplicated across multiple physical locations.
Instead of Jose's data being copied multiple times, only his changes are being backed up. This means that to Jose, he can restore all his data to any backed up time, but doesn't have to copy identical data multiple times. He only pays for storage of his data and each of his changes once!
You may also be interested to know that our backups / snapshots are stored as objects, providing the same durability, scalability and availability for your storage drives.
So... What does Amazon S3 mean?
On our simplification rampage of cloud computing, we believe Object Storage is the most descriptive way to describe what S3 does, so Object Storage and S3 mean the same thing.
And what about Archiving / Amazon Glacier?
Archiving is an optional life cycle of this data, to be moved to an even cheaper version of storage.
In the same way, Jose and Gareth's data can be duplicated multiple times, stored in a very cost effective way, and can be retrieved back to object storage in a matter of minutes. For when things get accidentally deleted / corrupted.
Glacier is Amazon's way of describing archiving, effectively 'putting your data on ice'. Yep, they're nerds who like puns too.
More on the benefits of Object Storage, S3 and Glacier can be found here
What are we waiting for? Lets begin!
Step 1 - Navigate to the Object Store page
To create your very own object store, Log in to RONIN, select a project, and click Object Store in the left Navigation pane
Check it out! The object store summary page!
Step 2 - Create your Object Store
On the first card, click NEW OBJECT STORE, name your new object store and then click CREATE STORAGE
Next, RETURN TO SUMMARY to see your newly created object store.
Behold our newly created Object Store!
Step 3 - Generate your secret access key
Now click GENERATE KEY
This will create your secure access key that will allow you to access your object store with your chosen program.
This is just like a real world key, and anyone with this key / file can access your object store. Be careful who you share it with and keep it in a safe place!
Your browser will automatically download a csv file with your credentials into your downloads folder for convenience.
You can also regenerate / delete a key whenever you like, meaning the previous key will no longer work. Click the options button on your object store card and select replace / delete key.
Once you've generated your key, you can now connect to your store!
Step 4 (Optional) - Turn on versioning
By turning on versioning, all files you modify within your Object Store will be preserved each time they're modified. This is very useful for times you make mistakes and need to restore something that was working before.
Because you're using Object Storage (as explained above) you are only storing the modifications of these files, and not the whole file over and over, but you can still restore a file to it's former glory.
To turn on versioning, click the Versioning button and then click ENABLE VERSIONING
Step 5 (Optional) - Turn on archiving (Glacier)
With another optional cost saving / data backup method, you can specify how often files are archived.
This won't affect your files in s3, instead it will create backup files on very cheap storage at an interval of your choosing. The trade off is those files take a little while to become available again (5 minutes at the time of writing this)
To turn on archiving, click the ARCHIVING button, enter the amount of days and then click ENABLE VERSIONING
Step 6 - Connect to your Store
Windows / Mac users
Click here to see how to connect with Cyberduck
Linux users
Click here to connect with AWS Command Line Interface
Once you've transferred some data to your object store you will be able to see the total size of the data that is in there, including the amount of data that has been archived (if archiving is enabled):
Note: Object store sizes are calculated and updated by AWS every 48 hours.