Getting your head around what edge computing comprises is a tall order even for the most hardened data architects. At its simplest level, the edge constitutes billions of devices functioning outside of a centralized infrastructure and throwing off data at a relentless pace. In today’s interconnected world, the flywheel of data generation never stops spinning.
While much of this data is fleeting in its usefulness, some of it requires storage. This, in turn, triggers a cascade of questions about the best storage architectures and technologies for such purposes.
So, with edge computing an ever-more pivotal element of IT infrastructure, let’s start by considering some of the main factors driving the growth in edge storage.
An Avalanche of Data
The breakneck growth in data is the primary driver of demand for commercial storage outside the large data centers. In the next few years, the data sphere will accelerate to levels beyond our current comprehension.
Just think of the smartphone in your pocket and how sophisticated its applications are compared with even five years ago. Then consider how we’re only at the dawn of self-driving vehicles relentlessly cranking out data on the road.
Commercially, the impact of industrial IoT is fast disrupting the balance between local and cloud data centers. Think of advanced manufacturing plants that require high concentrations of compute onsite. Those robotic arms shouldn’t wait a millisecond longer than necessary to deploy to the next action.
For its part, IDC projects that the unstructured datasphere will more than double between 2022 and 2026 in data generated by the Internet of Things (IoT).
“What people probably don’t quite understand is how big data is going to grow, and probably a lot of it is at the edge,” said Seagate CEO Dave Mosley on a call with analysts in November 2020.
“There are some natural use cases that are coming out, but it’s really about decentralized data. Not everything is going to be centralized in one cloud.”
Undoubtedly, there’s significant industry interest in the commercial opportunities associated with edge storage.
Smaller data centers offering content delivery networks and similar low-latency applications are poised to grow sharply in the next few years, supported by developments in 5G and accelerated demand for cloud.
The storage OEMs see a mass capacity play here, beyond simply the “micro edge” of ruggedized, industrial, and surveillance devices.
Hybrid as the New Normal
Over the past decade, commercial data has migrated in droves to the cloud, and that growth is set to continue—worldwide expenditure on public cloud services is projected to soar to $809 billion by 2025, according to research from IDC.
The dynamic, however, is no longer as one-directional as previously the case. Amid concerns over adequate visibility and resiliency, there’s persistent industry chatter about the repatriation of data from the public cloud to private data centers and edge locations.
This boomerang phenomenon makes sense. When dealing with the surging demands of real-time data applications and AI-driven workloads, the challenges inherent to cloud data storage makes edge and on-premise solutions increasingly attractive (once again) to organizations.
Recognizing this, the enterprise OEMs and hyperscalers have moved fast in recent years to better accommodate the on-premise and edge storage needs of businesses large and small. What’s the point in forcing organizations to choose between the public cloud and private solutions? ask industry leaders like Intel chief Pat Gelsinger.
He’s right, of course. Amid growing demand for data storage, enterprises can literally have the best of all worlds. This is smart positioning from the enterprise OEMs and hyperscalers, who continue to sign deals left, right, and center. In fact, according to recent research, the hybrid cloud market will grow at a CAGR of 20% through 2027.
By bringing the best of the cloud data center—from advanced analytics to machine learning capabilities—to bear on data at the edge, the industry giants have normalized hybrid with great commercial success.
The Pull of Data Gravity
The greater the mass of data and its surrounding infrastructure, the more difficult it can be to move quickly.
Retrieving the right data at the right time is often an expensive and troublesome proposition for firms already navigating a complex web of storage architectures.
For hedge funds crunching trades at lightning speeds or for media companies making real-time edits to an in-production video, keeping the data close to its point of origin simplifies user access and cuts latency. Netflix has long embraced this principle of proximity across its global content delivery network.
“Storage in remote clouds is playing an important role in enabling collaborative workflows, content distribution and in archiving,” finds a study from ResearchandMarkets into the media and entertainment industry.
Or consider the telecommunications giants. Not only do they have a vital role in delivering 5G, but their existing infrastructure yields countless opportunities to accelerate the development of the edge. Leading telecommunications firm American Tower already operates as a real estate company, and acquired 20 data centers in a 2021 acquisition.
Finding space for edge infrastructure isn’t without its challenges, especially in high-density areas. But where there’s a will, there’s a way. Even shopping malls are in on the act, offering up underutilized retail space for the development of local IT infrastructure.
Developing data center infrastructure for the edge is driving an increasingly expansive view of commercial real estate.
Find out about the outlook for Data Center HDD
Simplified Compute
One emerging feature of the rise of edge storage is the strong commitment from the drive makers to RISC-V.
The RISC-V instruction set, an open source initiative designed to simplify the instruction set on chips, is well suited to customization for edge storage applications, from computational storage and content delivery to optimizing the servo algorithms on hard disk drives in edge locations.
RISC-V, or Reduced Instruction Set Computer, is distinct from the Complex Instruction Set Computer (CISC) architecture integral to Intel and AMD chips dominant in enterprise and cloud computing.
Western Digital’s Richard New believes the adoption of RISC-V will be “as transformative for the computer hardware domain as the Linux operating system has been to the software domain.”
WD’s long-time rival Seagate agrees, deploying a custom RISC-V processor to build “a system on a chip” for use in hard drives optimized for the edge.
“We are beginning to see data shift to the edge of the network,” Seagate CTO John Morris remarks.
RISC-V has the potential to reconfigure the way that storage is accessed and processed both at the edge and in the core.
Stronger Data Security
RISC-V will also prove important to preserving data security for large data sets across edge devices. “Seagate is investigating how RISC-V security functionality supports an end-to-end data integrity and trust framework,” says Morris.
According to Gartner, over 50% of enterprise-managed data will be created and processed outside the data center or cloud by 2025.
Nonetheless, these edge environments will need to be connected reliably and securely to data center and cloud networks, says John Gray of networking security firm Aruba. “This distribution of data collection and analysis signals a move from data centers to centers of data, changing infrastructure, and operational strategies along the way.”
It’s not just the network security to consider—there’s the physical security of edge devices, too.
“We think the physical motion of data and its security is going to be an important trend in the edge for the next ten years,” Seagate CEO Dave Mosley remarked in his November 2020 call with investors.
Some of this edge data will have to move physically at one point or another, Mosley continued. He cited Seagate’s Lyve family of products as his firm’s response to the need for physically transferring data from the edge.
With the rise of the edge comes the need to transfer data from edge locations at scale. Companies are meeting this need with special services, such as Seagate’s Lyve Data Transfer.
The Demand for Video Surveillance
Wherever there’s a corner, there’s a camera—or so it seems these days.
With the advent of facial recognition technology and rapid improvements in AI’s ability to process large swathes of visual data, the opportunities for video surveillance in edge locations —both for good and for ill—appear endless.
Indeed, the market for surveillance hard drives is booming—and the storage OEMs are working hard to deploy solutions for different uses that emphasize cost-effectiveness while harnessing advanced analytics and compression technology.
The Effect of Data Sovereignty
Forget data gravity—what about data sovereignty?
With the wide-ranging implications of GDPR on data governance (and more recently the California Consumer Privacy Act), the appeal of storing data as locally as possible grows.
“Nations are putting regulations in place that are designed to help protect and control locally generated data, and assert countries’ rights to technological autonomy”, explains Ashok Iyengar, Executive Cloud Architect at IBM.
“As long as edge locations are still within the nation’s borders, they can be part of the distributed cloud paradigm and also satisfy the sovereignty requirement.”
National and State regulations also place constraints on how data centers are decommissioned. Familiarity with these constraints is essential to managing data security.
Customizing Hardware
So these are some of the reasons driving up demand for storage outside the security-clad perimeters of larger data centers.
The question remains how to handle the practicalities of remote data storage: how to improve data governance at the edge and what data to eventually transfer to more central locations?
Expect to see more of the following:
- The strengthening of data management tools ensuring that data can move with as little friction as possible between locations.
- The increasing sophistication of deduplication technologies within a unified data plane.
- A more robust approach to discarding transient data and better structuring the data that remains for storage.
- The continued “ruggedization” of hardware, such as hard drive enclosures and data transfer devices.
- The ongoing maturing of the edge computing stack, including a growth in containerization.
- An intense focus on security because, as Fortinet’s CMO John Maddison puts it, the edge increases risk “through an expanded attack surface.”
- The greater adoption of automation technology in edge data centers.
- A growing awareness of the environmental footprint of edge computing and the need to retire assets as rigorously and compliantly as in a main data center.
There will be many challenges ahead in better integrating edge facilities with central IT infrastructures, but the depth of the development roadmap for the coming years is abundantly clear.
A Shifting Landscape
Not only is the development of edge infrastructure changing the game for network computing, it is providing a welcome testing ground for newer storage technologies and fast-emerging interface protocols such as NVMe.
While hard disk drives will continue to play a dominant role in the major data centers, solid state drive and emerging storage-class memory technologies are well positioned to serve the primary needs of the data-intensive workloads associated with the edge.
“Expect to see a movement away from centralized storage to a more distributed model that results in smaller localized data centers,” explains Stephen Buckler, chief operating officer at Horizon Technology. “HDD will play a role at the edge, albeit a reduced one, resulting from the overall shift to SSD storage technologies that better meet the demands of application-specific workloads.”
The ongoing transition of SSD from the SATA interface to NVME (which currently allows up to twelve times the transfer speed of the latest SATA SSDs) will only hasten this transition, according to Buckler. Meanwhile, HDD—which continues to offer a cost advantage over SSD—looks set to carve out a significant role within the growing surveillance space, he says.
“Despite never being able to eliminate the performance gap with SSD, the HDD manufacturers are deploying new technologies, such as dual actuator arms, to support those edge use cases that require low-cost storage solutions, such as with surveillance.”
Driving the Green Edge
As with all data center technology, storage hardware for edge locations will only develop in the coming years—and with those performance improvements comes a growth in the inherent value of IT equipment under management.
Calculations around TCO apply equally wherever your IT hardware is located. Whether you are approaching a refresh cycle or preparing to decommission capacity, make sure your organization is recovering maximum value from its retiring assets.
Fully explore all options for the proper reuse, remarketing, or disposition of data center hardware across your IT asset portfolio, regardless of location.
This touches on not only the financial and regulatory aspects of IT asset disposition, but the environmental impact too. Research from industry consortia iNEMI demonstrated tremendous reductions in carbon impact associated with reusing rather than disposing of enterprise hard disk drives.
The edge is a core element of the future of enterprise computing, and storage is a key component of that equation. Is your organization prepared for the avalanche of data around the corner?
Work with a data center storage specialist like Horizon Technology and make sure your IT hardware—regardless of location—is working for you, not the other way around.
