cluster

A collection of 21 posts

spack

Spack is Research Computing's Best Friend

Research and cloud computing are a perfect fit, but the reality of managing software on dynamic clusters in the cloud can be a challenge. Fortunately, nifty tools like Spack can help get you up and running with optimised versions of your required software in just minutes.

slurm

Parallelizing Workloads With Slurm (Brute Force Edition)

One of the reasons researchers turn to clusters and Slurm is to run many independent jobs. Recently, we posted about how to use Slurm job arrays to do this. Here we describe some additional intuitive ways of getting the job(s) done.

slurm

Parallelizing Workloads with Slurm Job Arrays

Do you have a script that needs to be run across many samples? Or maybe you need to parameter test a script with a variety of values? This blog post covers how to achieve this simply and efficiently with Slurm job arrays.

slurm

A simple Slurm guide for beginners

SLURM is a popular job scheduler that is used to allocate, manage and monitor jobs on your cluster. In this blog post we teach you the basics of submitting SLURM jobs on your very own auto scaling cluster in the cloud with RONIN.

instance

Selecting the Right Machine Type for the Right Job

RONIN and cloud computing represent a different way of thinking. Choosing a machine type means understanding how different selections for CPU, memory, storage, and other characteristics will affect the time to do your work. This blog post will provide some guidelines for choosing wisely.

aws

Using Spot Instances to Reduce Cluster Costs

By requesting unused AWS EC2 instances at a discounted price via the Spot market, you can significantly reduce your cluster costs. This article will explain the pros and cons of Spot instances and provide you with some tips for getting started.

hpc

A Beginner's Glossary for HPC in the Cloud

High Performance Computing (HPC) has its own terminology, which is complexified (a real word!) by cloud terminology. Our goal is to simply define these terms to make it easier for users who are not cloud or IT gurus to create and launch clusters using RONIN on AWS....

cluster

Configuring and Managing Your own Autoscaling Cluster

Configuring and managing your own autoscaling cluster in the cloud does take a bit of getting used to. Luckily we're here to teach you the ropes and help you get your cluster set up just the way you like it!

cluster

New Machine or New Autoscaling Cluster? Which Should I Choose?

When you begin to work with RONIN you are immediately faced with a choice — do you create a new machine? Or an autoscaling cluster? And what is the difference and why do you have to choose at all? Machines A machine is a single computer, even though it can have

webinar

Research Computing with RONIN on AWS Webinar Series

Researchers demonstrating benefits of RONIN and AWS in the real world. This 3 part webinar series hosted by AWS, shows real life examples of researchers using RONIN and AWS. A beginner’s guide to genomics in the cloud with RONIN and AWS Ready to harness the power of cloud computing

alerts

Pipeline Alerts with Slack Webhooks

Keeping an eye on any analyses that are running in the cloud is crucial, but watching a terminal window all day is no fun! What if there was a way you could also be alerted whenever a pipeline errors or completes? Well, with a nifty tool like Slack's Incoming Webhooks, you can!

netdata

Easy and Extensive Resource Monitoring with Netdata

When running analyses in the cloud, it's essential that you can monitor your resource usage. Understanding your compute usage is crucial in ensuring your analyses are running as efficiently as possible and making the most of your valuable cloud dollars.

cluster

Packaging Clusters in RONIN

In RONIN, a package is a cluster template that is ready to create preconfigured cluster, defined by you! Packages will have an Operating System, a Root Volume (for the operating system) and any user defined volumes, such as datasets, files, or pretty much anything you want. It will also have

allocation

Auto Scale a Cluster.

Easy to follow guide on how to scale a cluster on RONIN.

Hyper-threading

What is Hyper-Threading?

Watch this awesome short video using food analogies to get an understanding of what Hyper-threading is!

MPI

How to Enable Intel MPI?

Intel Message Passing Interface (MPI) is now available on the AWS ParallelCluster AMIs. To enable Intel MPI instead of Open MPI, the Intel MPI module must be loaded first. You can install the latest using module load intelmpi. The exact name of the module changes with every update. Step 1

ganglia

Ganglia on Auto Scale Clusters

All RONIN Clusters get a free open source tool called Ganglia to help monitor your hardware. This article shows you how

cluster

Create an Auto Scale Cluster

Auto Scale Clusters in RONIN are your very own High Performance Computers that automatically scale by CPU's as workloads get more intensive. They're really easy to fire up. No, really!

packages

Packaging Machines in RONIN

So, you've created an awesome machine. Instead of doing this set up each time you need a new machine with the same configuration, why not package this machine?!

schedule

Scheduling Machines in RONIN

Are you human? Us too! One of the real drawbacks to being a human is we have the capacity to forget things, like turning things off when we don't need them. Smart Schedule to the rescue!

terminate

Terminating Machines in RONIN

If you no longer require a machine, you can terminate it. Terminating a machine in RONIN is permanent and irreversable!!! There are no do-overs! Be careful!