Backblaze is transitioning from using 4TB hard drives to 6TB hard drives in the Storage Pods we will be deploying over the coming months. With over 10,000 hard drives needing to be purchased over the next several months, the choice of which 6TB hard drive to use is critical. Let’s take a look at how we’re navigating this transition.
Getting Started
We started the process in September when we purchased and deployed our first 6TB hard drives. We purchased Western Digital (WD60EFRX) and Seagate (STBD6000100) hard drives. We deployed two Storage Pods, each with 45 drives from each brand. The Western Digital Storage Pod was designated UL796 and the Seagate Storage Pod was designated UL800. Each Pod was identical in its design and configuration except for the hard drives used.
Set Up
In a previous post, we described how we set up and load tested our Storage Pods. Each of the 6TB drive Pods passed their set up and load testing without incident and were deployed, first the Western Digital Pod and a few days later the Seagate. For comparison purposes, we also deployed Storage Pod UL838, which had 4TB HGST drives installed.
Into the Fire
Backblaze currently receives 130TB of data from our customers to store each day. Data arrives in similarly sized encrypted blocks of data. On any given day, there are 20-40 Storage Pods accepting these data blocks as they arrive. As a block arrives, it is passed to a Storage Pod. If that Pod is busy, the data block is passed to the next Pod in line. Over the course of each day, this results in all the available Storage Pods being given the same opportunity to accept data at the same rate.
| 130,000 GB The amount of customer data Backblaze stores each day. |
Newly installed Storage Pods accept data for the most part unencumbered until they reach 80% full. At that point a Storage Pod will reduce the amount of data it receives each day. New Storage Pods come online on a regular basis so that arriving data always has a place to go without delay.
When UL796, UL800, and UL 838, came online, we recorded various statistics with regards to how much data they loaded. Here’s a sample of the relevant storage data collected on one of those days.
| Date | Pod ID | Data Stored | Free Space | Total Space | Status |
| 9/17/2014 | UL796 | 48.87TB | 162.27TB | 211.14TB | Active |
| 9/17/2014 | UL800 | 29.16TB | 181.98TB | 211.14TB | Active |
| 9/17/2014 | UL838 | 6.81TB | 133.95TB | 140.76TB | Active |
The amount of data each of these Storage Pods received each day of the period of observation is shown below. We stopped recording when a given Storage Pod reached 80% full.
Since the three Storage Pods did not start accepting data on the same day, there could be some differences in data stored by day. The following chart aligns the three Storage Pods over the same calendar days, e.g. day one is September 19th, day two is September 20th, etc.
Observe that the different Storage Pods filled up at different rates. Here is how quickly each Pod reached 80% capacity.
| Pod ID | Drive Brand | 80% Full | Total Space |
| UL796 | Western Digital | 33 days | 169TB |
| UL800 | Seagate | 42 days | 169TB |
| UL838 | HGST | 23 days | 113TB |
This translates to the following average data stored per day for each Storage Pod.
| Pod ID | Drive Brand | Data Loaded |
| UL796 | Western Digital | 5.12TB/day |
| UL800 | Seagate | 4.02TB/day |
| UL838 | HGST | 4.91TB/day |
The Western Digital hard drive loaded data faster than the Seagate and even edged out the HGST.
Evaluation
Let’s review the Seagate and Western Digital drives so far:
- Initial Reliability (how many drives failed): No failures.
- Running Reliability (three months): No failures
- SMART Stats (three months): No error conditions recorded for the five stats that we utilize.
- Hard Drive Cost: About the same.
- Energy Use: The Seagate drives were 7200 rpm and used slightly more electricity than the Western Digital drives which were 5400 rpm. This small difference adds up when you place 45 drives in a Storage Pod and then stack 10 Storage Pods in a cabinet.
- Loading Speed: Edge to Western Digital, by a little over 1TB per day on average.
Our goal is to find hard drive models that are reliable and cost effective in our environment. The “one Pod” test has proven over the years to be a good starting point to eliminate those hard drives models that are obviously incompatible with our environment.
Next Step: Scaling the Test
Based on the results, we have ordered 230 Western Digital drives to fill five Storage Pods (with five spare drives). These will be installed, load tested, and deployed shortly. Assuming the Western Digital drives continue to perform as well in the five Storage Pods, we’ll move forward with using the Western Digital 6TB drives in our Storage Pods over the coming months.
Is Seagate Shut Out?
The Seagate 6TB drives performed well, albeit they loaded a little slower. They also use a little more electricity than we’d like. Still, they are not shut out. We really like to have multiple qualified hard drives to order and use in our Storage Pods. Diversification is good. To that end, we expect to order additional Seagate 6TB hard drives over the coming months, build them into Storage Pods, and monitor how they perform. Assuming they perform well, the 6TB Seagate hard drives would be added to our list of qualified drives.
Victory Is Fleeting
Today, the Western Digital hard drives are first in line to be our choice for 6TB drives. Of course, we just ordered 45 HGST 8TB Helium hard drives for testing. Their unit price is still a bit too high for cost effective deployment, but thanks to the return of a relentlessly decreasing hard drive pricing curve, it could be just a matter of time. In fact, Seagate is now beginning to ship their 8TB shingled magnetic recording (SMR) hard drives for a reported $260 a drive. Availability is spotty at the moment, but is certain to improve over the coming months. Using either the HGST or the Seagate 8TB drives means a 360TB Storage Pod is imminent—we can hardly stand the wait…
