As data generation proliferates, competition among edge data transfer services is hotting up.
The coming years will see explosive growth in data generated by the global Internet of Things (IoT). Whether it’s self-driving cars, advanced facial recognition, or the inexorable advent of 5G, new devices mean the accumulation of vast amounts of data at the edge.
This much is certain. What is uncertain is how effectively companies will consolidate this data in a usable—and profitable—form. Much hinges on how efficiently companies can transfer data from the edge to somewhere more centralized where they can extract the desired analytics. Increasingly, the site of such analysis is the ubiquitous cloud.
Transferring data efficiently is not such a straightforward proposition, though. To meet this challenge, a host of data transfer services have emerged to tackle network limitations. These services utilize everything from military-grade ruggedized drives to shipping containers that physically transport data by the petabyte.
The future is data in motion—literally. Welcome to the world of edge data transfer.
Bringing the Edge to the Cloud
Physical transport of data is more than convenient—for transporting data from the edge to the cloud at scale, it’s essential. Even with a 1 GB/ps connection, transferring a petabyte of data would take more than three months.
Meanwhile, the edge is growing quickly. Fortune Business Insights projects that the IoT market will grow from $381.3 billion in 2021 to $1,853.76 billion in 2028, with a staggering CAGR of 25.4%.
There are a number of trends which will drive this growth. The most notable is the spread of 5G, which will better enable apps that rely on low-latency data and machine learning. Another important factor is the rise of sensors of all sorts, used for everything from monitoring patient temperatures in hospitals and soil moisture on farms.
Since these devices throw off more data than current bandwidth could possibly handle, the best option for transferring data from the edge is often physically moving it in specialized storage devices.
Data Transfer In Practice
So, what qualities does a storage device need to transfer data at scale from the edge?
- The first is size. We’re talking about devices which move terabytes by the hundreds, or even petabytes.
- The second is “ruggedization”. High capacity storage is no good if the data gets harmed or illicitly accessed while en route. Devices vary from service to service in their degree of ruggedization, though many edge data transfer devices can take a beating. They are dust-tight, water resistant, and can handle extreme heat, cold, and the occasional fall. In addition, they are generally tamper resistant, and always heavily encrypted.
Edge data transfer services also differ in the level of flexibility in transferring the data. Do you buy or lease the transfer devices and related drives? Who takes care of transporting them—the client or the service provider? And is the service tailored to a specific cloud, or compatible with multiple final destinations?
Seagate’s Lyve Data Transfer
Leading drivemaker Seagate is making its own edge-to-cloud service, Lyve Data Transfer, a key part of its strategy going forward.
“There are people who are struggling with the data that they have on the edge, being able to move that into the cloud,” says CEO David Mosley. “The market is clearly out there.”
Lyve Data Transfer involves leasing ruggedized storage units to clients. The Lyve Shuttle device has a capacity of 8 to 16TB. For higher capacity, the Lyve Array lets you transfer 46-96TB, depending on the unit chosen. Clients can select between SSD and HDD-based transfer options.
What sets Lyve apart is that it aims to be “vendor agnostic.” While other services align to a particular cloud, Lyve allows you to transfer your data to any cloud, or multiple clouds, not just to its own cloud service. That said, the storage units offered through Lyve are lower capacity than competing services, and oriented towards physical data transfer.
Cloud Competitors
Lyve is going up against an array of competitors. One such incumbent is Amazon, with its “Snow” family of devices. Amazon sends devices to clients, who return them filled with data for Amazon to upload to its AWS cloud.
Snow devices are not vendor agnostic, since Amazon’s AWS tries, as ever, to encourage clients to do everything via Amazon. What it does offer is transparent pricing and, at the upper end, sheer scale. Its services include:
- The 8TB ruggedized AWS Snowcone at $60 for the first 5 days and $6/day after.
- The 42TB/80TB AWS Snowball at $300 for the first 10 days plus $30/day after. The Snowball comes in two varieties, one optimized for edge storage, and one optimized for edge computation. The latter boasts 208 GiB memory.
- The AWS Snowmobile, a 45ft ruggedized shipping container. This juggernaut can store a mammoth 100PB per unit. Pricing is $.005/gb/month.*
*prices correct at time of writing
Google and Microsoft are also playing in the same sandbox, offering services which, like Amazon’s, support their in-house cloud storage. Pricing follows the same model as Amazon, with a flat fee for a set period, followed by an added price per extra day.
Notably, Microsoft offers higher transfer capacity than any but Amazon, with 100TB and 1PB data boxes. However, there’s a limit to the number of boxes per order, leading to less flexible scalability. Google’s devices, all SSDs, hold 40TB and 300TB respectively.
For its part, Western Digital has taken a somewhat different path, at least for now. Rather than leasing storage devices for the transfer of data to a specific cloud, WD offers drives to clients through its pair of edge-optimized Ultrastar servers.
While both servers offer 60TB capacity, one is highly ruggedized, designed to comply with the MIL-STD-810G-CHG-1 military grade protocol.
Bringing the Cloud to the Edge
The name of the game here is getting edge data and cloud computing in the same place. The strategies examined so far involve feeding the cloud with data which is physically transported from highly localized, and possibly quite remote, locations—the so-called “micro edge,” where the IoT meets ruggedized storage devices.
This isn’t the only strategy, however. Instead of bringing the edge to the cloud, some services bring the cloud closer to the edge. Specifically, such services focus on the “metro edge,” consisting of the digital ecosystem in proximity to major population centers.
These initiatives go by many names. Amazon offers clients AWS Outposts, on-prem racks filled with AWS cloud infrastructure. These help customers transfer data to a “local zone”, AWS facilities established in the relevant Urban Area. AWS currently has seven local zones in major U.S. cities, and has announced the deployment of nine others. Amazon is particularly interested in Los Angeles, where demand for edge data services is high enough for AWS to construct a second local zone.
Google and Microsoft offer similar services, each tied to their own clouds. Microsoft has Azure Edge Zones in New York, L.A., and Miami. Companies purchase Azure Stacks, which function like AWS Outposts, allowing efficient edge computation and transfer operations.
For its part, Google has a huge number of “network edge locations”—146 globally, and 24 in the U.S. alone. It is also strategically positioning itself for the advent of 5G, pushing a number of initiatives, including Google Mobile Edge Cloud.
By any estimation, the space is hotting up.
Thinking Green
The growing importance of edge data transfer, and the host of companies stepping up to make such transfers more efficient, will significantly affect the way we store and move data.
But, are there potential downsides to these services? One is the financial and environmental impact if the devices used in the services, once retired, are shredded rather than recycled or repurposed. E-waste is a growing issue globally, and edge data transfer services could exacerbate the problem if the myriads of devices employed are simply left for scrap without a proper disposition plan.
Moving forward, it will be interesting to see what sort of model big players like Amazon, Microsoft, and Google use for repurposing edge-optimized drives and the hardware that supports them. High capacity storage devices don’t come cheap, and those who figure out creative ways to repurpose them could make significant savings, all while pushing the envelope environmentally.
Cutting down on e-waste would also give ethically minded consumers peace of mind. A good IT citizen should be curious about the environmental impact of the services they use.
As pressure for greener business grows, such concerns could be a deciding factor when a company is deciding between two edge data services.
A Burgeoning Marketplace
These concerns aside, the upshots are obvious. In the short term, edge data transfer services will drive demand for high-capacity and ruggedized storage devices, a boon for the storage industry.
Looking further ahead, these services may reshape the global flow of data, as the cloud becomes more and more accessible, and 5G enabled low-latency applications become increasingly prevalent.
Find our more about how Horizon Technology can support your IT hardware needs, both at the edge and in the core data center.
