Your Homemade Infrastructure

The idea of Open Compute Project has been adopted years before it went public. High-tech companies like Google, Amazon, and Facebook build their own data centers from commodity components. But in 2011, Facebook decided to take this backstage process to the public in order to benefit from the suggestions that would help in enhancing their data centers and reduce the costs. At the same time, other companies could also purchase these newly fabricated ideas and in turn benefit as well.

The open-source custom-made motherboards and subsystems that were created by Facebook (and other high-tech companies) were originally designed specifically for large, hyper scale world of data centers. Due to a "Do It Yourself" hardware environment (mentioned as DIY), these designs could be easily implemented in small, medium, and large enterprises and as a result could be sold at a much cheaper price.

The goal of this project is to build one of the most efficient computing infrastructures at the lowest possible cost by custom designing and building the software, servers, and data centers from the ground up and then sharing these technologies as they evolve. Today’s purchased data centers use a large number of inexpensive general-purpose servers.

For example, when trying to purchase a Cisco server, we are limited to optimizing the specs due to the restrictions set by Cisco, like the number of hard drives or CPU. In addition, every server that is sold has its own power supply which causes a lot of data centers to convert power three or four times (from 200 to the UPS then back out of the UPS into a power distribution unit and into the server’s power supply) which causes a loss of 2 to 5 percent of the power every time it is stepped down ^[5]. And not to mention the power loss and the excessive power cooling for all these servers. The OCP community has worked with other open source software projects to successfully develop energy-efficient servers with a 100% air-side economizer and evaporative cooling system to support the servers and simple screw-less server chassis with an integrated DC/AC power distribution scheme explained later-on ^[1]. These servers are 38% more efficient and 24% less expensive.

OCP doesn’t go open-source all the way down to the CPU. There is no open-source silicon that is capable enough to do that, and for economic and resource reasons, manufacturers are not willing to produce custom-made components. The OCP hardware designs are open at a higher level. Designers use standards-based components to bring together the motherboards, the chassis, the rack-mountings, the racks, and the other components that make up a server. The compute sled has the barest bones of what you would expect on a server motherboard: two processors, 16 DIMM slots for memory and a few PCI slots. There is no power supply on a compute sled, all the power is pulled from the rack for the sake of efficiency. There are 12 volt power connections in the back of the device which are directly sent to the board. This removes the complexity of having power conversion and supply in the system. Their efficiency contributes to converting the power the least amount of times required. This will help reducing the cost, cooling, space and enhance the environment.

Figure 1: Compute sled left bare of accessories and useless space ^[5]

The efficiency continues with the design of the cooling fans in each compute sled. The chassis has big fan blades that turn slowly, taking three or four watts to keep the chassis cool. Note that a traditional 1U server can take up to 80 to 100 watts for cooling ^[5].

The approach taken is a deconstruction for the data center and disaggregation of the components that usually make up a functional server. This is done with the minimum of complexity and the maximum of efficiency possible.

There are few "servers" in Facebook's data center architecture. Instead, there are racks filled with "sleds" of specific functionalities:

General-purpose compute sleds: motherboards filled in with CPUs, memory, and PCI cards for certain tasks

Storage sleds: high-density disk arrays

Memory sleds: systems with great quantities of RAM and reduced-power processors designed for handling large in-memory databases and indexing

Figure 2: Sleds that can be yanked out, sharing a common power supply and cooling system ^[5]

Figure 3: Storage drawer filled with drives, specifically designed for high capacity ^[5]

A list of products and the datasheets of each are publicly listed on their website.

But how can we be certain that the designs listed are reliable, compatible, or even feasible?

If we check their website, we can see posted projects with technical specifications and CAD drawings in zipped files. Since they are open source projects, anyone can suggest alterations in the designs and many times OCP provides many of the solutions for free. Consequently, an enterprise can adopt them, modify them, and forward them out to contract manufacturers who are ready to build them. Up till now OCP has led to about a dozen game-changing new pieces of hardware ^{[3] [4]}.

As an example, the Wedge is one of the latest switches launched by Juniper that was initially an OCP design and is now pioneering in the software-defined networking technology and competing heads-on with Cisco[6].

What happens with Cisco, HP, IBM and other big players?

The OCP will and already has an impact beyond its original audience and is quickly being adopted by hardware makers, driving down the price of new systems. They like to call them “vanity free servers” where they are not beautified by extra plastic and useless space or extra components and logos and colors and therefore OCP has been disrupting the whole model those companies have been built upon.

Before attending a customer meeting, vendors conduct a thorough research in the competitor’s offering in order to convince the customer that what they are proposing is much better in terms of specs and features. However, and in the end, they are all converging to the same offerings with some differences between them. We have seen vendors reach a stage where they have run out of ideas to convince the customer and eventually as a last resort claim that it is the history of that company or sell its name. And several times we are disappointed with the results of these products or would wish they were different in some features.

Facebook and Rackspace are the first main adopters of OCP hardware so far. But that could soon change in a snowball effect as businesses are discovering all the benefits offered by OCP. Intel, AMD, and others are starting to invest their intellectual property and substituting their components to those of OCP. They are currently targeting servers but network and storage is next on their schedule ^[5].

The openness of hardware doesn't mean the death but on the contrary it is considered as a rebirth where we see a greater pace of innovation because we're not always reinventing the wheel ^[5].

OCP is highly beneficial in offering cheaper solutions and in proposing optimal designs by investing in brains from around the world. A drawback that has to be considered might be the time it takes to create a purposeful server (or rack sled) and then waiting for it to be tested and approved and then sold to the customer. However, experts can offer their comments and suggestions beforehand as a start in order to prepare themselves for any potential future project.

Check the video below for an introduction to their project: