HDD Remains Dominant Storage Technology

What’s the outlook for enterprise hard disk drives (HDD) in the data center?

Does the recent dip in shipped units mean that the end is nigh for hard disk drives? Are solid state drives (SSD), fueled by long-term declines in NAND flash prices, on track to eclipse enterprise HDD as the dominant storage technology?

Hold your horses. Despite supply chain turbulence which has rattled the tech industry, reports of the demise of the hard disk drive are overblown. Even given advances in flash, most data still resides on spinning platters, and the dawn of HAMR drives is ramping up areal density. For the foreseeable future, enterprise HDD will have a central role storing data cheaply at scale.

Hard Times for Hard Drives?

Before talking long term predictions, let’s look at short term trends. There’s no getting around it: the past few years have been rough for HDD manufacturers.

In 2Q2022, there was a 33% y-o-y sharp drop in shipped units due to reduced demand for consumer HDD. A year later, it was clear the decline was also coming for nearline drives as well, with 3.5” high cap drives seeing a stunning 45% drop in shipped units in 2Q2023.

Sounds bad, until given some context. Notably, flash was hurting too. In early 2023, Micron saw its worst downturn in 13 years, Samsung’s profits hit an 8-year low, and Intel posted the largest loss in its history!

But now the good news. Heading into 2024, analyst Tom Coughlin projected that HDD growth would resume and prices would rise. A more recent forecast from Coughlin Associates suggests that HDD sales will rise at least through 2029.

Call it the pandemic yo-yo. First, supply chain disruptions led to shortages. Storage manufacturers, eager to meet demand, ramped up production. This led to a glut, decreased demand, and lower prices, hurting the bottom line of manufacturers.

If a glut was the problem, one would expect demand for enterprise HDD to pick back up once companies utilize their backlog of excess capacity. And sure enough, Seagate and Western digital are now eyeing price hikes in the face of surging demand.

The Relentless Rise of Data Storage

While tech trends are hard to predict, there’s a helpful rule of thumb: keep things in perspective. And for good reason: data is being generated faster than ever, and attempts to monetize it will drive demand for storage in the long-term.

Unit shipments are a poor guide to exabyte trends, since drive capacity is always increasing. For example, while HDD unit shipments peaked in 2010, HDD shipped capacity increased by approximately 67.5% between 2018 (912 shipped exabytes) and 2021 (1.35 shipped zetabytes).

As for where things are headed, the numbers are revealing. The global installed base of storage capacity is expected to hit 13.2ZB in 2024, almost double the capacity in 2020. But by 2025, there may be more than 175 zettabytes of data generated, vastly outstripping our ability to store it.

The projected capacity jump is partly due to the explosive growth of the internet of things. According to Fortune business insights, by 2032 the IoT market is on track to grow more than five times its current value. Also playing a role is Generative AI, which will drive demand for data storage devices at a CAGR of up to 20%, according to IDC projections.

Returning our focus to enterprise business, while flash storage will make up a growing slice of the pie, the Coughlin Associates study suggests that for some time yet, the vast majority of data will live on hard drives.

No surprise then that the major hard drive manufacturers—Seagate, Western Digital, and Toshiba—are doubling down on their HDD product road maps. The prospect of relentless data growth clearly supports their confidence.

New HDD Technologies

There remain good grounds for the three main hard drive makers to feel optimistic about the medium-to long-term outlook for HDD.

HDD manufacturers have been squeezing in extra 2TB platters and widely promoting SMR technology (shingled magnetic recording) for data center drives. However, they are also upping the ante on areal density with the use of heat-assisted magnetic recording (HAMR)

Beginning with a 10 disk, 32TB drive, Seagate plans to move to 36TB and 40TB drives, each with greater areal density than the last. The firm also experimented with 5TB disks in the lab. While no firm dates were given, Seagate CFO Gianluca Romano expects that there will be a gap of 18-20 months between HAMR capacity increases.

These Seagate drives will also utilize dual actuator technology, which is designed to unlock additional IOPS for improved read/write performance.

Western Digital has also been going the HAMR route. CFO Wissam Jabre said the firm was around 12 to 18+ months away from volume production of HAMR drives. These drives may also utilize WD’s innovative OptiNand technology, which augments HDDs with a small flash controller that stores metadata.

While the technical approaches may vary, what appears certain is that HDD is poised for significant capacity growth in the years ahead.

HDD vs SSD

None of this technological development reflects poorly on solid state drives—the opposite in fact. Indeed, flash is already the default choice for storage outside of the data center. “HDD makers have long since recognized the inevitability of the notebook entirely transitioning to SSD”, remarks Stephen Buckler, chief operating officer at Horizon Technology.

However, this does not spell the end of the hard drive, Buckler insists. “Increasingly storage is all about the cloud. As AI and big data ever more play a role in our day-to-day lives, this increases the need for cheap storage. Storage leaders such as Seagate and Western Digital continue to invest in improvements to hard disk technology, signaling that the game is far from up for HDD.”

Advances in the NVMe protocol and a growing appetite for composable architectures built around flash will further upend the established order. Along with the rise of storage-class memory, these factors will lessen the focus on data center HDD.

Limits to manufacturing will also keep HDD in play. Pure Storage R&D VP Shawn Rosemarin argues there will be no more HDD sales after 2028. But Chris Mellor at Blocks & Files estimates that even given advances in flash capacity, NAND production in 2029 will fall about 405EB short of what would be needed to replace HDD capacity.

Cloud Storage Holds Strong

Speaking of the cloud, cloud storage as measured by spending is still going strong.

In late 2023, Gartner estimated that end-user cloud spending would increase 20.4% year-over-year to $678.8 billion. This reinforces an IDC report which says that public cloud spending will continue to outpace private cloud spending until at least 2027, when it will account for 69% of total storage infrastructure and compute spend. That spending means a lot of exabytes, the lion’s share of which will be on spinning platters.

Again, it’s not an either-or situation when it comes to storage technologies. Tape serves as a good analogy here. It remains unlikely that HDD will develop into a credible threat to tape as a medium for cold storage in the hyperscale data center. Just as tape still has its place in the storage mix, enterprise HDD will likely retain its place in the cloud as the data storage workhorse.

Catch a Rising Tide

The future storage mix is always tricky to predict, as unexpected innovations can lead to rapid change. But for the moment, with hard drive sales picking back up and HAMR delivering long-promised increases in areal density, it seems pretty clear HDD tech has life in it yet. HDD currently dominates as a share of data center storage, and this is unlikely to end anytime soon.

At the same time, flash continues to strengthen across storage use cases, and will grow as a proportion of an expanding pie. In a world of diverse storage uses, analysts would do well not to overemphasize the conflict between HDD and SSD. After all, the surge in data generated means there will be a massive amount to store and process, suggesting an all-hands-on-deck scenario where multiple varieties of storage work in tandem. A rising tide lifts all boats, as they say.

Whether you’re buying or disposing of data storage capacity, learn more about how Horizon Technology leverages its deep industry expertise to support your enterprise hard drive needs.