AI-Assisted Active Data and AI
Governance

MODULE 1: WHAT IS ACTIVE DATA GOVERNANCE AND WHY DO WE NEED IT?

This module looks at what active data governance is, how ungoverned data can impact business operations, decision making and increase risk, and where most companies are in terms of implementing data governance. It also looks at how this challenge has grown to encompass multiple different governance disciplines including data quality, data privacy, data access security, data sharing, data retention, and data usage across a hybrid, multi-cloud distributed data estate. Finally, it looks at the fractured approach many companies have been taking to tackle this challenge, the problems caused by using best of breed tools and why it is no longer practical to expect people to do everything manually when AI can help automate tasks.

• The ever-increasing distributed data landscape – on-premises, multiple clouds, SaaS applications and the edge
• The impact of ungoverned data on compliance with legislation and regulations, business profitability and ability to respond to competitive pressure
• Data Governance – Where are we?
• Beyond data quality to multi-disciplinary data governance
• Problems caused by a fractured approach to data governance using multiple best of breed tools
• Why AI is needed, and a data catalog is not enough

MODULE 2: WHAT ARE THE REQUIREMENTS FOR AI-DRIVEN ACTIVE DATA GOVERNANCE TO GOVERN DATA ACROSS A DISTRIBUTED DATA ESTATE?

This module looks at what the requirements are to govern data in terms of people, process, technologies, policies and capabilities and what we need to change to move from manual data governance to AI-driven active data governance automation.

• Key requirements for governing data & content across a distributed data estate
• People

• Data governance roles, responsibilities and groups

• Core processes & tasks to standardise data governance activities, approvals and actions
  • Technology requirements

• Universal data governance platform

  • Data catalog

  • Trainable ML classifiers

  • Data Governance co-pilots & AI agents

  • Dynamic data masking

  • Universal data access control

  • Data marketplace

  • Data loss prevention software

  • Data governance observability

  • Data governance enforcement agents

• The Data Catalog Marketplace

  • Alation, Ataccama, Atlan, AWS Glue Data Catalog, BigID, Cambridge Semantics Anzo Data Catalog, Collibra Data Catalog, data.world, Databricks Unity Catalog, Google Data Catalog, Hitachi Vantara Pentaho Data Catalog, IBM Knowledge Catalog, Informatica IDMC Data Governance and Catalog, Microsoft Purview, Oracle, SAP DataSphere, Qlik (Talend) Data Catalog, TopQuadrant TopBraid

  • Data governance platforms

  • Data access security tools

  • Foundational AI-assisted active data governance services

• Data catalog business glossary

• Data source discovery scanners

• Data catalog data map and data relationship knowledge graph

• Data governance classifiers

• Generative AI for automated metadata enrichment and conversational data search
  • Active data governance applications, policies & policy types needed to govern:

• Data quality

• Data access security

• Data privacy

• Data retention

• Data loss prevention

• Data sharing

• Data use and maintenance
  • Data governance action framework

• Data governance health metrics to measure data health, how it’s processed, protected and used

• Critical data elements

• Example data governance events & incidents to be monitored

• Likely triggers of governance actions

• AI models for data governance alerting, recommendation, action and task automation

• Data governance observability agents to monitor & detect incidents

• Using metadata lineage to understand the impact of events and incidents

• Data Governance action services

  • Data governance verification service to check that governance & data curation activities have occurred

  • Data governance rectification services to rectify metadata

  • Data governance action invocation service

• Example actions to be performed on items (e.g. on a data source, a business term, a data asset, a policy, a data product)

• Data governance action processes to ensure actions are standardized and consistently executed

• Organising human actions using an in-box and human task monitoring

MODULE 3: FOUNDATIONAL ACTIVE DATA GOVERNANCE SERVICES — THE IMPORTANCE OF A BUSINESS GLOSSARY

• Data standardisation using a common business vocabulary
• The purpose of a business glossary in data governance
• Business glossary capability in data catalogs

• Alation, Amazon Glue, Collibra, Informatica IDMC Business Glossary, IBM Knowledge Catalog, Microsoft Purview, Qlik (Talend), TopQuadrant TopBraid
  • Planning for a business glossary
  • Glossary roles and responsibilities
  • Glossary term submission, voting approval and dispute resolution processes
  • Approaches to creating a common vocabulary
  • Organising data definitions in a business glossary
  • The role of a data concept model
  • Utilising a common vocabulary in BI tools, semantic layers, data modelling, data fabric, MDM and APIs

MODULE 4: FOUNDATIONAL ACTIVE DATA GOVERNANCE SERVICES – AUTO DATA DISCOVERY, CATALOGUING AND MAPPING TO A BUSINESS GLOSSARY

Having defined your data, this module looks at automatically discovering what data you have, where it is and how it maps to your business glossary to provide a business understanding of your data estate.

• Understanding your data estate — the critical role of AI-driven data catalog software
• Registering data sources for discovery
• Automated data discovery and data quality profiling using a data catalog
• Automating metadata enrichment using Generative AI
• AI automated suggestion of business glossary terms during data discovery
• AI-assisted mapping of physical data assets to business glossary terms
• Setting policies and health controls to monitor business glossary creation and data curation
• Using AI agents to monitor data curation activity and automate curation actions

MODULE 5: FOUNDATIONAL ACTIVE DATA GOVERNANCE SERVICES – AI-DRIVEN DATA & CONTENT CLASSIFICATION

This module looks at manually and AI-driven automatic labelling of data and content to know how to govern it using predefined AI classifiers, user-defined classification schemes and trainable AI classifiers. It also looks at how AI-driven automatically classified data shows up in a data catalog and how policies can be assigned to labelled data to govern it across your data estate.

• What is AI-driven data classification?
• Automatically detecting and classifying sensitive structured data using predefined AI classifiers in a data catalog
• Creating your own data confidentiality and retention classification schemes
• Manually classifying content using your own classification scheme (e.g. Office documents, SharePoint, Email, Chat, Microsoft Teams or Zoom Meetings)
• Training and using AI classifiers to auto label content across your data estate
• Using classification insights to understand sensitive data proliferation and data redundancy across your estate
• Understanding policies, policy groups and tag-based policy management

MODULE 6: IMPLEMENTING AI-ASSISTED GOVERNANCE OF DATA SECURITY ACROSS YOUR DISTRIBUTED DATA ESTATE

Having classified the data and content in your data estate, this module looks at protecting data and content, focusing on sensitive or confidential data. It looks at setting and enforcing policies to govern data access and usage security, as well as governing data loss prevention.

• Data security objectives
• Key technologies in governing data security

• Policy establishment

• Policy enforcement
  • Steps to implement data security
  • Setting health controls and enterprise-wide and domain-specific policies in your data catalog to govern data access across your data estate

• Attribute based access control
  • Unifying data access control across multiple data stores
  • Universal authorisation fabric software (e.g. IBM, Immuta, Databricks (Okera)) and how they integrate with data catalogs
  • Using cloud application security brokers

• Auto discovery of cloud app data usage

• Setting policies to govern access to and use of sensitive data and content from applications

• Monitoring cloud application activity
  • Dealing with insider risk management and internal information barriers

MODULE 7: IMPLEMENTING AI-ASSISTED GOVERNANCE OF DATA PRIVACY ACROSS YOUR DISTRIBUTED DATA ESTATE

This module looks at AI-assisted governance of personal and financially sensitive data across your data estate to remain compliant with legislation in multiple jurisdictions.

• Data privacy objectives
• Data privacy legislation — GDPR, CCPA, HIPAA and more
• Steps involved in enterprise-wide data privacy risk management
• Using AI to automatically identify where unprotected personal data is located
• Setting policies and health controls in your data catalog to govern data privacy across your data estate
• Data privacy insights on sensitive data location, how it moves and where you are at risk
• Data privacy policy enforcement across a distributed data landscape

• Linking your data catalog to other technologies

• Encrypting and de-identifying personal data

• Using data loss prevention (DLP) to avoid loss of personal data

• Protecting personal data in email, chat, documents, file shares, cloud storage and endpoints
  • Using AI agents to monitor data privacy violations and automate actions
  • Monitoring AI agent effectiveness
  • Managing subject action requests

MODULE 8: IMPLEMENTING AI-ASSISTED GOVERNANCE OF DATA RETENTION ACROSS YOUR DISTRIBUTED DATA ESTATE

This module looks at governing the lifecycle of data across your data estate and how AI can automatically classify data to label it for retention, set policies to control retention duration, and manage expiry and legal holds.

• Creating a data retention classification scheme
• Complying with country and region-specific legislation
• Training AI classifiers to label your data
• Automatically classifying data and content using AI to create retention labels
• Setting policies and health controls in your data catalog to govern data retention
• Using AI agents to monitor data retention expiration and automate actions to destroy, archive and hold data

MODULE 9: IMPLEMENTING AI-ASSISTED GOVERNANCE OF DATA SHARING ACROSS YOUR DISTRIBUTED DATA ESTATE

This module looks at producing trusted, compliant data products to be shared across the enterprise and beyond, and how data sharing can be governed.

• Data sharing objectives
• Key technologies to help produce high-quality, compliant data products for sharing
• Steps to creating data products using a data catalog business glossary, automated discovery/mapping and AI-assisted data engineering
• A unified approach to producing high-quality data products using Data Fabric and DataOps pipelines
• AI-assisted publishing of certified, high-quality, compliant data products in a data marketplace
• Potential metadata standards for data products e.g. DPROD
• AI-assisted governance of data sharing and consumption using data contracts in a data marketplace
• Consumer use of AI-assisted conversational data search in a data marketplace
• Creating a standard data sharing approval process for consumers
• Using AI agents to monitor and track shared data consumption and usage

MODULE 10: IMPLEMENTING AI-ASSISTED GOVERNANCE OF DATA QUALITY ACROSS YOUR DISTRIBUTED DATA ESTATE

This module looks at AI-assisted governance of data quality across your data estate.

• The business impact of bad quality data
• Common data quality health metrics
• AI-assisted creation of data quality validation, matching and survivorship rules in your data catalog using Generative AI
• Using your data catalog to automatically profile and validate your data quality
• Setting data quality health controls and thresholds in your data catalog to govern data quality
• Leveraging data catalog AI-suggested data quality rules
• Integrating data observability with data catalogs to monitor and report data quality issues
• Using AI to monitor data quality validation rules against health control thresholds
• Auto-generating validation rules when data quality thresholds are breached
• Using the data catalog for AI-assisted data cleansing and generation of data integration pipelines
• AI-assisted MDM and the data catalog

MODULE 11: IMPLEMENTING AI GOVERNANCE

This module looks at AI governance to manage AI models and AI risk.

• AI Governance best practices including:

• Creating an AI inventory and risk registry

• Setting up accountability for AI

• Evaluating and mitigating AI risk

• Governing AI development

• AI observability
  • Formalising processes and enabling auditing
  • Explainable AI
  • AI observability
  • Avoiding PII leakages when building vectors for Generative AI LLMs
  • Establishing AI guardrails for GenAI
  • AI governance tools