Open Source Engine Powering the Next Gen. Cloud Stack – Part 1

Recently there have been a flood of announcements in the Cloud Computing space that have heartened the Open Source community. This is particularly important to ensure large adoption of some of the guts of the cloud computing stack. This post will examine some of these announcements in the light of building an entire stack of open source cloud.

Motivations for going Open Source

When you have one company that is innovating rapidly as Amazon has been in the cloud computing space, it is often hard for others to overcome the first mover advantage.  Indeed, Amazon was first to the Cloud Computing party by offering its EC2 service way back in August 2006. Since then it has been innovating at a rapid pace and adding new services and features almost on a monthly basis.

The only way to compete with this kind of pace is for other companies to enlist the help of developers all over the world to help improve their cloud computing offerings. If the specs and code forming the cloud computing offering, are put on the Web for everybody to use and explore, it forms a powerful force for improving the core product. If your open source offering grabs more mind share than your competitors then you gain a tremendous competitive advantage.

Another key motivation is the building of the ecosystem. When a large enough user-base starts building around a product, the software community gets engaged around the product and starts building extensions and plug-ins that elevate the whole platform. Witness the explosive growth of the Eclipse platform. There are probably more than 1000 Eclipse plug-ins that are being actively maintained all thanks to its open source nature.

The final motivation has to do with Company Philosophy or Goodwill. Sometimes an organization is so well known for being a major player in the open source ecosystem that new products are, by default, expected to be Open Source (with a paid variant if need be). Famous examples of these are organizations such as Red Hat or Apache. The associated advantage is the goodwill that the company has earned amongst the developer community.

The Cloud Computing stack

The simplified cloud stack is depicted in the figure below:

Simplified Cloud Stack
Fig 1: The Simplified Cloud Stack

Let us look at each layer of the Stack and find out how Open Source can power the next gen. Cloud Stack.

Physical Hardware

This is the layer at which Racks and Servers are physically connected to Storage and Networking devices to form the guts of your cloud.  Facebook created a stir recently when it decided to open source all the specs to its data centers. Their recent Open Compute Project puts detailed blueprints for Servers and Datacenters out on the web for everyone to analyze and (hopefully) emulate. The Facebook team presented a very compelling case for more vendors to use the specs that they have released – namely Power Savings from Efficiency. Facebook engineers claim their servers are 38 percent more efficient than the off-the-shelf ones they were buying previously. And that has meant a 24 percent cost savings. The hope is obviously that other High growth start-ups that need to build their own data centers (Twitter, Square, etc being the prime candidates) can embrace these specifications and actually improve on them.

This is in direct contrast to Google that is notoriously secretive when it comes to the exact specs of their enormous datacenters. They maintain such a super-secret tight control on the exact specifications that the Google data centers have been compared to Area 51 sometimes.

This is no small feat on Facebook’s part. Facebook is engaged in a fierce battle with other web scale companies such as Google and having a super-efficient data center could be a huge competitive advantage for them. However, Facebook is betting that by essentially commoditizing the “secret sauce” of the efficient data center, they are unleashing an innovation ecosystem that will allow all players to benefit.

Infrastructure as a Service (IaaS)

This is the core software layer that enables IaaS services such a Provisioning-on-demand, Configuration and Management of the Cloud infrastructure. This software is also known as the Cloud Orchestration software, IaaS enabler, Cloud Director, Cloud Manager, etc.
The open source IaaS space used to be dominated by Eucalyptus software until recently. Eucalyptus grew from an academic project at UCSB into one of the most widely downloaded open source Private and Hybrid cloud platforms. Eucalyptus claim to fame was the user-facing API that it exposed was compatible with the Amazon API.  However last year it suffered many defections – the chief one being NASA who was one of the highest profile users of Eucalyptus.
NASA, reportedly unhappy with the scalability constraints of Eucalyptus, collaborated with Rackspace to found the OpenStack project. In a very short time, OpenStack has managed to attract a wide consortium of companies including Microsoft, Cisco, Dell, Citrix, Intel etc.

Rackspace is one of the founding members of OpenStack and have publicly declared that they are going to shifting all of their existing cloud hosting to OpenStack very shortly. Recently Internap has become the second provider to publicly announce that they are building an OpenStack based cloud. This is all the more remarkable as Internap is a competitor to Rackspace. OpenStack has also been gaining tremendous momentum lately with many major announcements – Citrx recently announced that they will be selling a Citrix branded OpenStack edition with a cloud ready Xen Server edition.

At its heart, the OpenStack framework allows anyone to build three different flavors of IaaS services:

  1. OpenStack Compute (Nova) – designed to provision and manage large networks of virtual machines, backed by orchestration and management support.
  2. OpenStack Storage (Swift) – designed for creating redundant, scalable object storage using clusters of standardized servers.
  3. OpenStack Image Service (Glance) – designed to provide discovery, registration, and delivery services for virtual disk images.

All of these services are backed by extensive API support that allows programmatic manipulation of almost all the control interfaces to these services. Cisco is another heavyweight that has shown interest in OpenStack. They have actively begun to work on it by submitting their Networking as a Service (NaaS) proposal to the OpenStack group. In another interesting tie-in, OpenCompute and OpenStack are now also collaborating to ensure that OpenStack can run efficiently on top of the OpenCompute reference architectures.
Obviously OpenStack is not the only player in the open source IaaS space and neither is it the first. Take a look at some of the other notable open source projects for IaaS in addition to OpenStack and Eucalyptus:

  • OpenNebula – One of the earliest entrants to the open source IaaS party and one of the most mature. OpenNebula bills itself as the “Industry Standard for on-premise IaaS cloud computing, offering a comprehensive solution for the management of virtualized data centers to enable private, public and hybrid (cloudbursting) clouds“. It has a number of clients using it including CERN and FermiLab. They are roughly equivalent to OpenStack Nova.
  • Red Hat CloudForms – Along with OpenShift (which we will look at in more details in Part 2) this forms Red Hat’s foray into the cloud computing space. As of posting this (June 2011) the CloudForms service was still in closed beta. The promise of CloudForms is enticing as it claims to offer “..Application Lifecycle Management (ALM) functionality that enables management of an application deployed over a constellation of physical, virtualized and cloud-based environments.
  • CloudStack – A community offering from that enables users to build, manage and deploy compute cloud environments. Note that a commercial version of this is also available that has more cutting edge features.
  • Puppet, Chef and the like are not full-fledged IaaS Solutions but do capture some of the more important functionalities in the IaaS enablement solution. With a little bit of work these utilities could form the backbone of a workable IaaS solution.

Check out this post for a comparison of some of the above mentioned open source solutions along with some Enterprise solutions in the IaaS space.
However, OpenStack has picked up a staggering velocity and has gone thru 3 major releases already in under a year.With the diverse nature of companies that are supporting OpenStack and large developer interest, it seems to be well on its way to becoming the dominant player in the open source IaaS space.

Stay tuned for part 2 of this post, where we will examine the many interesting open source players in the PaaS, SaaS and the Cloud Client spaces and how they are playing a part in driving the next generation of the open and interoperable cloud. We will also look at some other open source plays such as Open Virtualization Alliance that are taking a different track on the cloud computing issue.


One thought on “Open Source Engine Powering the Next Gen. Cloud Stack – Part 1

  1. Pingback: Apple iCloud – Hype Cycle or Tipping Point for Cloud Computing? |

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s