SUN’s strategy was to define and deploy a standardized and modular data center computing unit. A single design effectively scales-up to handle a broad range of profiles from small lab to mega data center.
Dean Nelson, Sr. Director of Global Lab & Datacenter Design Services provided me a tour and tutorial recently. Dean and his team have generously shared their best practices, which are available for free download from the SUN site. A few may also be found in the resource section at the end of this post.
The technical concepts are highlighted by a gray box and may be skipped without diminishing one's understanding of the business approach and benefits.
SUN started on its data center journey about five years ago. The primary goals were to
- Reduce operating expenses: Use less real estate and less energy
- Enhance flexibility: Eliminate the data center “speed bump” that keeps the business from rapidly deploying equipment to satisfy changing business requirements
- Future-proof: Ensure DC infrastructure can grow to handle two times the load.
As with all elegant designs, the new design is deceptively simple. SUN created a modular data center design by deploying standardized “pods”. As Dean Nelson, explained “A pod is an energy-efficient building block that can be duplicated easily to create a data center of any size in any location.”
SUN made a sizable investment of $250 million over several years, or approximately $300 per square foot average across new and existing space which included server upgrades, environmental and power infrastructure equipment and improved server utilization.
- Year 1: Understanding the portfolio (over 1M square feet with a variety of computing needs), take baseline measurements and invent a new way to build data centers
- Year 2: Complete smaller projects around the world. Perfect the design and create the deployment strategy.
- Year 3 -5: Execute on the major consolidations. California and Colorado being the largest efforts.
Less real estate
As part of the consolidation, servers were upgraded to gain more compute power in a much smaller footprint. From 2,000 the average compute power has tripled.
In addition to DC plant improvements, SUN looked at the server utilization since it drives the number of servers required which in turn drives the size of the DC. Better utilization results in less real estate and less energy. A recent McKinsey study confirms that server utilization is still abysmal at most data centers and improvements can yield 20 – 30% return.
- In the U.K. data center, real estate was cut by more than half
- 60% reduction of real estate was achieved in the California consolidation
- Overall, real estate was reduced by 40%. From 1.4 million square feet to 860,000 square feet.
Next generation servers use less energy for comparable amount of work, but they provide a great deal more compute power. As compute power increases, more heat is generated in a smaller area. Consequently the heat gain is more sizable and the cooling more challenging. This is analogous to a space heater that is more effective in a small room than a larger one.
Without dramatic changes in cooling strategy, the energy consumed for cooling the identical compute power would have been greater. Rather than cool the entire data center as one unit, the cooling break-through was to decentralize cooling and cool each POD individually. Dean Nelson explained that “the POD improves cold air delivery and allows the DC to operate at higher temperatures”.
- In California, SUN lowered data center utility costs by a remarkable 50% and reduced Sun's global carbon emissions by over 1% in just three months. As a result of this effort, Sun received $1.2 million in energy rebates plus a $250,000 "Innovation Award" from its local utility, Silicon Valley Power.
- In the U.K. data center, Sun cut energy consumption by 70%.
One other benefit of the modular data center design is faster and less expensive installation to handle on-going changes to servers and racks. Dean Nelson reviewed how “It is now much less expensive to “snap-in” extra compute power since the power infrastructure was pre-configured to handle a range of power requirements.”
- It took just 6 hours to install 3,000 servers in San Jose facility
- Since SUN maintains the variety of equipment for testing of current customer install base and new products, there is constant reconfiguration of equipment. This activity once required weeks and with modular DC, it is just hours to reconfigure.
Traditional data centers usually build out the power and cooling for the maximum forecasted load. Dean Nelson shared that SUN took a different approach where “the infrastructure is also modular allowing it to grow to satisfy new requirements. This saves both capital and operating expenses.”
- The implementation includes the ability to easily double average compute power within the existing footprint.
- Planned ahead for potential new power and cooling approaches
- SUN has already added 30 kw server racks without engineering work; they were simply snapped into place.
Now that SUN has worked out the kinks, these strategies and techniques can be used for any size data center with any computing profile. Not only new data centers, but an existing data center can benefit.
SUN’s Bangalore data center is such a scenario. As Dean Nelson reviewed, “Bangalore presented the challenge of working within the existing office building’s power, space and cooling . Also no servers were replaced during the project. Despite these constraints, we were still able to reduce our space consumption by slightly more than half while reducing our power use by 17 percent.”
Even more benefits could be achieved if combined with a tech refresh. A business is then able to extend the life of its current data center and operate it at a lower cost.