The Technology Behind the OpenStack Cloud Computing Project

Rackspace (a noted high-end hosting provider) and NASA recently announced a joint open source endeavor called OpenStack. The project aims to merge cloud computing technologies from both organizations into one common, open cloud computing platform.

The rationale behind the Rackspace and NASA teams sharing their code on a project is simple. Neither of them is really a software development shop -- they specialize in different areas -- and the current cloud computing software they have was grown in-house either to satisfy internal requirements (NASA) or to offer a solution in a competitive market (Rackspace).

In the case of Rackspace, they would like to see projects move away from proprietary, locked-in computing platforms (such as Amazon S3 or the Google App Engine) toward a common open standard. Hence, as long as your application runs on the OpenStack platform, theoretically you should be able to host it either in-house or with any specialized OpenStack hosting provider. It's a typical case of wanting to make the cloud computing pie bigger and more open. Rackspace is confident its ability to provide high-end hosting will give it an edge in such a larger market.

In NASA's case, they just want to focus on their responsibilities (processing massive amounts of data coming in from space exploration) and avoid having to maintain an in-house, proprietary processing infrastructure.

The Rackspace and NASA Parts of OpenStack

Both organizations created OpenStack by contributing key technologies that are building blocks for their vision of a common, open cloud computing platform. Each of its contributions -- object computing from NASA and object storage from Rackspace -- is quite mature and production-ready (NASA processes terabytes of satellite imagery data with it and Rackspace hosts probably equivalent terabytes of data as part of its custom cloud computing offering), so the effort is geared toward integrating them into one common solution and picking the best parts of each.

Although the code in both sub-projects is quite mature and deployed in production, the OpenStack platform is still very much in its infancy as a complete solution. Both are adequately described as being in a state of "Developer Preview" and are definitely not production-ready yet (unless you are very adventurous).

OpenStack Compute (Nova)

The OpenStack Compute part of this project (otherwise known as Nova) is the NASA donation. It is basically a cloud computing controller built to match the existing Amazon EC2 and S3 APIs. Its main responsibility is managing the large set of virtual machines that make up a cloud.

NASA had recently open sourced it as the Nebula computing platform and that is when Rackspace decided to partner up with them.

Nova offers the following set of features:

• Public API: allows you to send/receive commands via HTTP to the cloud controller
• Cloud controller:presumably the most important component, responsible for integration with LDAP, storage services based on the Amazon S3 API, internal node/worker management
• Nodes: workers that start instances on demand
• S3: an HTTP server that provides the S3 API
• User manager: providers LDAP integration
• Network controller: allocates/deallocates IP addresses and VLANs

The Nova developer preview code is available from Launchpad, but I suggest you start off with the documentation at nova.openstack.org.

OpenStack Object Storage (Swift)

Object Storage (otherwise known as Swift) is Rackspace's contribution to this project. It is basically the open sourced version of Rackspace Cloud Files . The Swift developer preview code is also available from Launchpad.