Data Center Automation and the Personnel Shortage

The data center, as essential infrastructure, is more intricate than it’s ever been. With all this complexity, new tools are needed to help centers store data at scale efficiently and securely. Enter data center automation. 

Automation is multifaceted. Much current automation consists in cloud solutions which let you automate certain business operations. In this article, we’ll instead zoom in on what automation can do for managing data center infrastructure itself. 

Data centers could certainly use the extra help, as they are currently facing personnel shortages. Automation as it exists today can’t solve these shortages, though it eases the burden on existing staff. Understanding why can clarify the scope and limits of current technologies. 

Automation: The basics

Automation is the future.

According to projections by Gartner, 60% of organizations will deploy compute with infrastructure automation tools by 2025. By the same year, Gartner estimates that 50% of enterprises will have devised AI orchestration platforms, up from 10% in 2020. These platforms are crucial in helping non-experts utilize AI-based tech. 

Automation and artificial intelligence (AI) may be a match made in heaven, but it’s still important to distinguish between the two. 

Automation consists of procedures that can run themselves with little human interaction to accomplish various tasks. Examples can be as simple as using a program to perform mathematical or financial calculations, or as complex as trying to design a robotic employee. 

AI, on the other hand, is a broad set of techniques that let computers replicate aspects of human intelligence. Possible uses include reasoning, planning, categorizing, assessing, language processing, and more. 

While automation and AI are separate concepts, they can overlap. Every data center automates some tasks一you don’t arrange bits in a drive yourself, after all. The promise of AI is that it will allow the automation of more complex tasks. 

The cutting edge of AI is machine learning, which allows a machine to use data in order to gradually improve its ability to complete a task. Much of machine learning is deep learning, which makes use of neural networks modeled on aspects of the human brain. 

Some Examples of Automation in the Data Center

There’s a wide range of tasks automation can help with. As AI becomes more sophisticated, there are even more possibilities on the horizon. 

One important sort of automation technologies are configuration management tools (CMTs), which help speed up storage deployment while enabling scalability and predictability. These can greatly simplify the process of updating and patching your system, systematizing settings, and managing subsystems of data center hardware. 

The best CMTs are flexible, so when it comes to employing them in creative ways, the sky’s the limit. Different tools do, however, have various points of emphasis which may make them more suited to particular tasks. 

One great way to apply CMTs is as an aid to ensure compliance. Tools such as Puppet help maintain continuous compliance by letting you write your policies in code. They can even generate automated reports for auditors. 

Speaking of code一there’s no way around coding if you want full customizability. Thankfully, many automation tools, such as Ansible, emphasize simplicity and readable code, so it’s transparent to others just what tasks you’ve put your CMT up to. 

Other notable automation tools include 

• OpenStack, which gives you a dashboard that lets you manage local compute, storage, and networking as if it were a cloud. 
• Chef, which is particularly good at integrating disparate systems, treating them as one. 
• Network monitoring tools such as SolarWinds and Paessler PRTG can help you visualize network traffic, track most used appliances, and keep an eye out for link saturation. 

Automation 2.0: Rise of the Machines

Machine learning (ML) techniques are also playing a huge role in expanding the range of tasks which can be automated. 

For example, several companies are working to develop “self-healing” technology, automating aspects of data center maintenance. 

Google and Seagate have partnered up to use ML to predict drive failure. Their machine takes SMART (Self-Monitoring, Analysis and Reporting technology) data from drives, and uses it to spot commonalities in drive behavior just before failure. 

Here’s another cool application of ML一with the use of DeepMind AI, Google reduced energy used for data center cooling by up to 40%.

As data, DeepMind fed its machines historical data on temperatures, power consumption, pump speeds, and more in Google’s data centers. It also led to a 15% reduction in power usage effectiveness. 

The Personnel Shortage

One major perk of automation is how the resulting efficiency frees data center workers to do more complex tasks. But this brings up a pressing issue一finding and keeping those workers is becoming more difficult. 

According to a report by the Uptime Institute, labor shortages are a mounting obstacle to data center development, and the challenge is growing. 47% of data centers are having difficulty finding qualified applicants, up from 38% in 2018. 

Retention is also difficult. 32% of respondents said their company is having difficulty with staff retention, compared to only 17% in 2018. 

This trend isn’t limited to data center workers, and is found elsewhere in tech. According to a DigitalOcean report, 25% of developers with over a year of experience started a new job in the past year. Additionally, 42% of those who have not left their jobs, are considering leaving. Compensation and remote/flexible arrangements were the two biggest factors. 

Data center automation to the rescue?

The personnel shortage is making data center operations more difficult. Can automation fix the problem? Do those workers who have stayed on need to worry about a robot takeover in their workplace?

Not so fast. It’s less straightforward than that. 

While almost three in four operators questioned by Uptime believe AI will eventually reduce staffing levels, 50% of those surveys believe “eventually” is five years out, at least. Uptime even named the view that AI will replace the need for human knowledge as among their top 3 myths about AI. 

Interestingly, respondents see AI-induced staff reductions as further away than they did in 2019.

Three years ago, 29% believed AI would reduce staff levels in the next five years, compared to 23% who believed so in the latest report.

Current Limits of Automation

Of course, survey answers are not the same as an argument. But in this case, it makes sense. While tech for data center automation is surging, it’s a long way off from being able to replace the skills of data center workers. 

Neural networks have made AI vastly better at identifying and categorizing input. For example, AI can now identify objects in pictures and convert audible speech into text. But even cutting-edge AI has considerable difficulty identifying cause and effect, adapting to complex new situations on the fly, and using common sense and heuristics to spot patterns in situations.

 

AI’s difficulty in using heuristics and common sense in adapting to new situations is one reason why self-driving cars, despite repeated promises, are still rarely found in the wild. 

So while AI and automation are improving in tandem by leaps and bounds, they’re quite a ways off from filling in for data center workers. Turns out having someone on the ground with some common sense and creative problem solving ability still goes a long way. 

In fact, demand for data center workers is set to rise from 2 million in 2019 to 2.3 million in 2025. 

As data itself moves to the cloud, and a software-driven operating model, basic administrator roles一platform administrator, database administrator, network administrator一are being transformed into roles with software-development capabilities. The new jobs require an understanding of how to leverage the software-based technologies behind today’s data center automation. Examples of these types of roles include site-reliability engineers, cloud engineers, solution architects, and others that drive operational efficiency and cost-competitiveness of data centers.

Arthur Hu, Senior Vice president and Chief Information Officer at Lenovo. 

Simplicity, Efficiency, Security

Data Center Automation is still the future. True, it may not be the solution to personnel shortages. But to get hung up on that is to risk overlooking the amazing things data center automation makes possible. 

AI-equipped configuration management tools greatly simplify storage at scale, allowing staff to systematize, update, and utilize data center infrastructure in an efficient and secure way. Machine learning will undoubtedly lead to even greater efficiencies in areas such as failure detection and energy reduction. 

All this is good news in the long term for both data centers and their workers. The glass is more than half full一it’s overflowing. 

Automation is great, but needs well-managed hardware to function optimally. Find out how Horizon can help you get more out of your data center hardware .