Software Seminar

Flat Datacenter Storage

Ed Nightingale

Architect
Online Services Division (OSD) at Microsoft
 
Thursday, November 15, 2012
12:00pm - 1:30pm
2150 DOW

 

About the Event

Flat Datacenter Storage (FDS) is a high-performance, fault-tolerant, large-scale, locality-oblivious blob store. Using a novel combination of full bisection bandwidth networks, data and metadata striping, and flow control, FDS multiplexes an application's large-scale I/O across the available throughput and latency budget of every disk in a cluster. FDS therefore makes many optimizations around data locality unnecessary. Disks also communicate with each other at their full bandwidth, making recovery from disk failures extremely fast. FDS is designed for datacenter scale, fully distributing metadata operations that might otherwise become a bottleneck.

FDS applications achieve single-process read and write performance of more than 2 GB/s. We measure recovery of 92 GB data lost to disk failure in 6.2 s and recovery from a total machine failure with 655 GB of data in 33.7 s. Application performance is also high: we describe our FDS-based sort application which set the 2012 world record for disk-to-disk sorting.

Biography

Ed Nightingale is an architect working in the Online Services Division (OSD) at Microsoft. Ed runs the Cosmos storage system, which is a batch processing store for Bing that runs on tens of thousands of machines and processes petabytes of data every day. Before joining Bing, Ed was a senior researcher at Microsoft Research. Ed has always enjoyed building operating systems and distributed systems. More details on Ed's work can be found at http://research.microsoft.com/~edn

Additional Information

Contact: Stephen Reger

Phone: 734-764-9401

Email: sereger@eecs.umich.edu

Event Sponsor: SSL

Open to: Public