We live in an era of data abundance. Every digital interaction creates a small dot that, when plentiful enough, paints a remarkably detailed picture of who we are and what we do. Data has become a central fuel of the 21st century global economy and if optimally managed, can provide competitive and operational advantages for organizations of every type.
However, despite significant investment and increasing focus, the true promise of data such as understanding customer needs and building better products has been elusive. In addition, the attendant risks of data such as privacy, security, and compliance requirements are still not adequately managed. Fortunately, new tools and processes may be turning the tide. Automation, and in particular artificial intelligence (AI), appears to hold the key to unlocking the full potential of data.
Applying DevOps and DataOps to Data Governance
Despite the notable efficiencies that the information age has enabled, in the background much of its creation has been historically manual in nature.
Consider software development. It starts with lengthy requirements gathering, then code is written and handed off to testers, and finally it’s deployed to production. These essential activities have largely been human tasks and have been subject to frustrating delays and complexity inherent to disparate teams attempting to work together.
Recognizing these limitations, as recently as the late 2000s, the emergence of DevOps, and later DataOps, both successfully incorporated deeper team collaboration and continuous integration and delivery in order to improve the speed, quantity, and quality of output.
Today, these same principles coupled with automation are now being applied to the larger and essential role of data governance; the processes, standards, and tools to unleash the value of data while also reducing the risks that come along with it. In fact, the novelty of this nascent approach has coined a relatively new term: DataGovOps.
DataGovOps Brings Collaboration and Automation To Data Efforts
Data governance is a proven approach to managing data well, but adoption by organizations remains relatively low, and positive outcomes across the enterprise have been spotty at best. The diligence and rigor to ensure that all stakeholders in the data supply chain consistently follow processes and standards is a bar that is often too high to reach.
For sure, many organizations are highly successful in their data governance efforts, but it can be costly and create stubborn bureaucracy. DataGovOps offers a solution to data governance that can share its promise more broadly and increase the chances it will achieve desired results.
A New Day Has Arrived For Data Governance
Kash Mehdi is the VP of growth, operations, and strategy at DataGalaxy, a vendor in the data governance marketplace. He’s spent over a decade helping organizations around the world with their data efforts, including notable stints at data solution providers Informatica and Collibra.
Mehdi thinks a lot about how to enable data value and particularly approaches to help organizations in high-profile industries such as banking successfully adopt and implement the benefits of data governance. In the last few years, much progress has been made in his profession and he’s optimistic about several recent developments.
First, he believes that there’s a growing recognition of the value of data governance by business leaders and the unfortunate perception that it is simply a topic about compliance and process enforcement is declining. Specifically, more organizations are able to tie their data governance efforts directly to desired positive business outcomes. Second, he is buoyed by the enhanced capabilities that automation and AI are bringing to data governance solutions.
Automation and Artificial Intelligence Propel Data Governance
As data flows through and between organizations–the data journey so-to-speak–it’s subject to many requirements including managing access and availability, quality and standards, security, and compliance. Rather than manual interventions that traditionally support these requirements, but also end up creating inefficiencies and errors, contemporary tools and platforms are elevating automation as the solution to eliminate bottlenecks and increase data quality and readiness for use.
Mehdi says he’s most excited by the role AI is playing to accelerate and optimize the advantages of automation. By using AI, for example, to easily look for patterns, and classify and tag data, these typically manually intensive activities go away, and data staff and leaders are able to focus on higher value work such as how data can help with developing new products and services.
The Road To Success With DataGovOps and Artificial Intelligence
Data governance is growing in priority year over year, much of it driven by a better understanding of its value, but also by increasing compliance, privacy, and security requirements. It’s also now seen as an enabler of success with AI initiatives, given that quality AI results are often dependent on AI-ready data.
The increasing use of automation in data governance is also reducing the reluctance factor, providing more comfort to leaders that the benefits of an investment will significantly outweigh the overhead of deployment.
Kash Mehdi offers three tips for leaders to consider in overcoming the roadblocks to success with DataGovOps:
- Identify, document, and communicate to the organization how data governance efforts connect directly to desired business outcomes.
- Train all employees in data use expectations, tools, and skills to support efforts, including embedding data governance into job descriptions.
- Develop a culture where staff understand that they are data creators, not just consumers. In other words, staff should see how the data they capture and enter is essential to the broader goals of the business.
Finally, while DataGovOps has the potential to be transformative to an organization’s data efforts, to entirely rely on technology would be a mistake. Humans must be in the loop, and it will be the right mix of automation and manual activities that deliver the best results.