Compute Canada and the future of HPC computing

Compute Canada is hosting a series of town hall meetings to discuss the future of high performance computing in Canada. Here are some thoughts:

In order to increase Canada’s HPC capability and make them more relevant for today’s scientific computing needs, it will have to embrace new computing models.

The next generation in computing is cloud computing. Compute Canada should embrace this model as part of its service offering such that researchers can use the cloud across Compute Canada infrastructure. Importantly, it must be possible for researchers to grow their cloud from local private clouds (we already have one setup in our lab), into the Compute Canada cloud and ultimately into commercial clouds (such as Amazon EC2), if necessary. Compute Canada also needs to invest in data storage, and create the means by which such storage may be accessed using data access standards (e.g. Amazon S3) and provisioned through networks (e.g. CANARIE <-> university <-> commodity networks). Compute Canada should endeavor to use open standards and ensure interoperability for any deployment.

A major issue with Canada’s HPC centers is that we cannot currently (AFAIK) host public services on them. As a bioinformatician, it would be invaluable for me to have Compute Canada host a server using my image, thereby ensuring scalable capacity and continuity for the software tool (as reported in publications). Importantly, some of our services require on-the-fly compute resources to accomplish their task, and it would be ideal if we could setup asynchronous services that use cloud facilities to compute, and then provide a link to users as to where they can find their results (stored on an S3 compatible store). Being able to do this would be a game changer for bioinformatics (and I suspect in other computing fields), and would create a new paradigm for open service provisioning across the world.

While full fledged cloud computing software is currently available as both open source and commercial solutions, this would present an opportunity to acquire such software 'en masse' to get started, while also creating new capacity in developing cloud computing solutions. I can imagine training my 4th year project/honours students to develop bioinformatic software using Compute Canada’s compute/storage resources – but not until we see cloud computing.