MIT's Sandook System Revolutionizes Data Center Efficiency and SSD Performance

MIT researchers have developed Sandook, a groundbreaking system designed to significantly enhance data center efficiency and storage device performance. This innovative solution directly addresses underutilized storage capacity and the persistent problem of performance variability in networked solid-state drives (SSDs).

Sandook is a software-based solution that simultaneously manages three primary sources of performance variability in networked SSDs, adapting in real-time without requiring specialized hardware.

The system operates using a sophisticated two-tier architecture. This includes a central controller responsible for overall task distribution and local controllers on each machine that facilitate rapid data rerouting during performance issues.

Addressing Performance Variability

Sandook specifically targets and mitigates the following types of performance variability in SSDs:

• Hardware Differences: This includes variations in SSD age, wear, and capacity from devices procured at different times or from various vendors.
• Read-Write Interference: Performance slowdowns occur when read and write operations happen concurrently on the same SSD, as writing new data often necessitates erasing existing data.
• Garbage Collection: Unpredictable interruptions are caused by the garbage collection process, which clears outdated data to free up storage space.

The system masterfully manages these challenges by optimizing task distribution through a global scheduler and employing local schedulers to react swiftly to events by shifting operations from congested devices. Sandook also mitigates read-write interference by alternating the SSDs used for read and write operations. It further enhances performance by profiling SSD performance to detect and reduce workload during garbage collection. Additionally, the global controller assigns workloads in a weighted manner based on each device's unique characteristics and capacity, ensuring optimal utilization.

Performance and Impact

During extensive testing, which included demanding tasks like AI model training and image compression, Sandook nearly doubled the performance of traditional approaches. The system demonstrated remarkable improvements, increasing individual application throughput by 12 to 94 percent compared to static methods and improving overall SSD capacity utilization by 23 percent. Remarkably, it enabled SSDs to achieve 95 percent of their theoretical maximum performance without requiring specialized hardware or application-specific updates.

"Our goal is to maximize the longevity of data center resources while significantly boosting performance."
— Gohar Chaudhry, Lead Author of the Research

The research findings will be formally presented at the esteemed USENIX Symposium on Networked Systems Design and Implementation.

Future Development

Looking ahead, researchers plan to integrate new protocols available on current SSDs to provide more granular control over data placement. They also aim to leverage the inherent predictability of AI workloads to further increase the efficiency of SSD operations, pushing the boundaries of data center performance even further.