It is sometimes hard to grasp just how much digital data we generate each day as a planet. From small contactless coffee purchases at a Starbucks to hours of closed-circuit television (CCTV) footage, the digital footprint is growing rapidly and almost exponentially. Back in 2012, a study by IBM estimated that 2.5 exabytes — that’s 2.5 billion GB — of data was generated every day. Jump forward just 5 years, and another study found that the data created on mobile devices alone had shot up to 8 exabytes per day and is still growing.
For enterprises of all sizes, digital data has always been useful, but it is increasingly the most important fuel to power line-of-business activities. And for some, an entire business model is based on how quickly data can be collected, processed and turned into action.
Historically, data storage was hived off into a silo in the enterprise IT department. The big four wings of applications, servers, networking and storage had distinctly different skill sets, suppliers and driving goals. The storage team talked of SAN and network-attached storage (NAS) with a focus on adding continually more capacity and performance to meet the demands of new applications — while bemoaning the lack of faster network connectivity.
This paradigm still endures in a few places, but the last decade has seen enterprises become more agile in an effort to capitalize on the benefits offered by the internet and to stave off rivals as markets have opened through globalization.
If you look in the most progressive of enterprises, the storage team is no longer an isolated outpost concerned with the minutiae of data management but is increasingly having to transition skill sets to integrate with the wider goals of the enterprise. The most pressing change is the move to the cloud, with the arguments about the benefits of the cloud becoming redundant. For many workloads, it is the most effective option — and especially for developers building the next generation of apps that focus on areas such as machine learning. The sheer flexibility on offer is hard to ignore.
For IT staff that have had various storage-related job titles, the next big transition is to the role of data architect. However, where the shift from direct attached storage to NAS and SAN had been technology focused, the role of a data architect encompasses not just learning new storage technologies but, more critically, how to deliver value through data to the rest of the organization.
This means gaining a fundamental understanding of the benefits and limitations of the cloud model. It also means being part of key projects such as information governance in relation to regulatory constraints like the European Union’s General Data Protection Regulation (GDPR). The data architect now gains a seat at the table for many more enterprise activities because data architects have been elevated from a traditional infrastructure role to a more intrinsically valuable position that enables a wider range of business-critical tasks.
The data architect now gains a seat at the table for many more enterprise activities because data architects have been elevated from a traditional infrastructure role to a more intrinsically valuable position that enables a wider range of business-critical tasks.
This also includes more grandiose projects that could well shape the direction of the business for the next decade.
For many, the starting point of the transition will be either replicating or moving large portions of data storage architectures from on-premises data centers into cloud equivalents that are fit for purpose to match the business need. In most cases, the skills and technologies that are well understood for an on-premises storage architecture can move seamlessly into the cloud with little effort. The skills needed for a transition to a data architect role should at least encompass gaining certifications in the organization’s preferred cloud platform, whether that’s Amazon, Microsoft, Google or some other platform. As somebody that has been down this journey, I can attest that it is less daunting than you would imagine. Best of all, the cloud platforms have very similar core architectures which makes becoming conversant across all the big public clouds a relatively easy progression.
Having skilled data architects is becoming one of the most critical requirements. However, the move towards a data architect role should not be viewed as just a personal desire. Enterprises have a vested interest in elevating the data team to a more front-and-center role to support an increasingly agile IT environment. There is a clear skills shortage alongside a fear that can be articulated in a quote often attributed to the entrepreneur Richard Branson: “Train people well enough so they can leave; treat them well enough so they don’t want to.”
With data considered by many enterprises as the most valuable commodity they possess, more so than gold and oil, the next decade will require both technology and people that are ready for the transition. In a world where cloud is not just an option, but the first port of call for many organizations, having skilled data architects is becoming one of the most critical requirements. The organizations that are ready to start the journey sooner rather than later are in the best position to reap the benefits.