Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 3 Next »

Key Points


References

Reference_description_with_linked_URLs_______________________Notes______________________________________________________________








What_is_Data_Governance_and_Why_Does_it_Matter_updated.pdf  link 

What_is_Data_Governance_and_Why_Does_it_Matter_updated.pdf. file


dbx_data-and-ai-governance-6sept2023.pdf. - databricks   link 

dbx_data-and-ai-governance-6sept2023.pdf. - databricks  file






Key Concepts


Data governance (DG) is the process of managing the availability, usability, integrity and
security of the data in enterprise systems, based on internal data standards and policies that
also control data usage. Effective data governance ensures that data is consistent and
trustworthy and doesn't get misused. It's increasingly critical as organizations face new data
privacy regulations and rely more and more on data analytics to help optimize operations
and drive business decision-making.

A well-designed data governance program typically includes a governance team, a steering
committee that acts as the governing body, and a group of data stewards. They work
together to create the standards and policies for governing data, as well as implementation
and enforcement procedures that are primarily carried out by the data stewards. Ideally,
executives and other representatives from an organization's business operations take part,
in addition to the IT and data management teams.

While data governance is a core component of an overall data management strategy,
organizations need to focus on the expected business benefits of a governance program for
it to be successful,

Data Governance problems 

Without effective data governance, data inconsistencies in different systems across an organization might not get resolved. For example, customer names may be listed differently in sales, logistics and customer service systems. That could complicate data integration efforts and create data integrity issues that affect the accuracy of business intelligence (BI), enterprise reporting and analytics applications. In addition, data errors might not be identified and fixed, further affecting BI and analytics accuracy.

Data Governance programs

compliance with data privacy and protection laws, such as the European Union's GDPR and the California Consumer Privacy Act
(CCPA). An enterprise data governance program typically includes the development of
common data definitions and standard data formats that are applied in all business systems,
boosting data consistency for both business and compliance uses.


<<key < FACTUR3DT.IO - measure Data Governance Value in the organization against defined OKRs, KPIs in the Data Governance Program

<<key < key benefits possible include

data access controls

compliance with data management policies and regulations

easy data availability for users and services 

automation of data management tasks 

data valuation based on usage vs objectives 






Jim DG, EDM keys


My career is managing data anywhere:  goalie, janitor, whisperer, therapist


Role keys- alignment on IT architecture, plans, priorities, business partnerships internal & external


SDP - virtual teams > discovery > assessment > plan > design > test > train > rollout > support

From IT EA, EDM, Digital Transformation programs, create related program for EDG that is aligned, integrated w BUs, clients, vendor services to meet goals on EDM quality & services OKRs, DT, Compliance ( internal and external audits )


EDM - Enterprise Data Mgt - 4 R keys - data & services are: RIGHT, RELIABLE, RESPONSIVE, REACTIVE across the enterprise, clients and vendor services


DTCC DLT Architect part of EA services tower w other IT lines > governance support on DLT, data


EDM app services, infrastructure, support tools 


Collibra for MDM or ?

Couchbase for distributed NoSQL w JDBC tools, reporting

Pulsar & Solace on global events

DLT w Firefly, Fabric

MySQL, Postgres, Oracle, Snowflake, MongoDB, Aws Aurora

Messaging > Kafka, MQ, ActiveMQ, Artemis ? 

Tools - EA, SA, Archimate, Plantuml, MDM repo ( NOT commercial ones but maybe integrated w Oracle )



RajG on Data Governance 


Deloitte on Data Governance Challenges


https://www.linkedin.com/posts/rajkgrover_datagovernance-generativeai-dataprivacy-activity-7145740183915720704-FQyX?utm_source=share&utm_medium=member_desktop

Top 15 #DataGovernance Challenges in Context of the #GenerativeAI and How to Mitigate Them

 

1.    Bias and Fairness

2.    Ethical Use and Misinformation

3.    #DataPrivacy

4.    #DataQuality Assurance

5.    Explainability

6.    Transparency

7.    Verification and Authenticity

8.    Regulatory Compliance

9.    #DataSecurity Risks and Malicious Use

10. #IntellectualProperty and Copyrights Issues

11. End-User Accountability

12. #HumanAICollaboration Guidelines

13. Public Perception and Trust

14. Community and Stakeholder Engagement

15. Real Time Monitoring and Control

 

Addressing these data governance challenges requires multidisciplinary, holistic and proactive approach that involves collaboration between data scientists, ethicists, legal experts, and stakeholders.

 

How to Mitigate the Above Challenges:

 

1.    Bias Audits

2.    Diverse Training Data

3.    #ExplainableAI

4.    Transparency Guidelines

5.    Anonymization and Privacy Preserving Techniques

6.    Human Review of the Quality Assurance

7.    Encryption and Access Controls

8.    Regular Audits by Legal Expertise

9.    Clear Ownership Policies

10. Intervention Protocols for Real Time Monitoring

 

Image Source: Deloitte

 

Data Governance Challenges with Generative AI



RajG Data Governance Concepts for Generative AI

Why is #datagovernance important in #GenerativeAI?

Data governance plays a pivotal role in fostering #innovation in the evolving AI landscape by ensuring responsible data practices, mitigating biases, and safeguarding privacy. A robust data governance strategy is the key to unlocking the full potential of your Generative AI use cases.

Comment image, no alternative text available










Other DG and EDM Concepts

Who should own Data Quality?

https://www.linkedin.com/posts/rajkgrover_dataquality-datamanagement-datagovernance-activity-7139936738956812288-eEwd/?utm_source=share&utm_medium=member_desktop

Ownership of data quality is a critical aspect of effective #datamanagement within an organization. While the specific team or role that owns data quality may vary depending on the organizational structure, size, and industry, there are several common approaches:

 

#DataGovernance Team:

 

Role: A dedicated Data Governance Team or Office is responsible for defining and enforcing data management policies, including data quality standards.

Responsibilities:

-Establishing data quality policies and procedures.

-Defining data quality metrics and benchmarks.

-Monitoring and enforcing data quality standards.

 

#DataStewardship Team:

Role: Data stewards are individuals or a team responsible for the management and oversight of specific sets of data.

Responsibilities:

-Ensuring data quality at the operational level.

-Resolving data quality issues and discrepancies.

-Collaborating with business units to improve data quality.

 

IT or Data Management Team:

Role: The IT or Data Management Team, including database administrators and data engineers, may be responsible for technical aspects of data quality.

Responsibilities:

-Implementing data quality tools and technologies.

-Monitoring and optimizing data quality processes.

-Collaborating with business units to understand data requirements.

 

#BusinessAnalysts or #DataAnalysts:

Role: Business analysts or data analysts who work closely with business units and understand data requirements can play a role in ensuring data quality.

Responsibilities:

-Profiling and analyzing data to identify quality issues.

-Collaborating with data stewards to address data quality concerns.

-Participating in the definition of data quality rules.

 

Quality Assurance (QA) Team:

Role: In organizations with a strong QA function, the QA team may be involved in ensuring data quality for systems and applications.

Responsibilities:

-Applying QA principles to data-related processes.

-Conducting data validation and testing.

-Collaborating with data owners and stewards.

 

Business Units and Data Owners:

Role: In a decentralized model, business units or data owners may have ownership of data quality for the data they generate and use.

Responsibilities:

-Defining and maintaining data quality requirements.

-Taking ownership of data quality improvement initiatives.

-Collaborating with data stewards and IT teams.

 

#ChiefDataOfficer (#CDO) or Chief Analytics Officer (CAO):

Role: The CDO or CAO may have a strategic role in setting the overall vision for data quality and ensuring alignment with business goals.

Responsibilities:

-Setting the strategic direction for data quality.

-Advocating for data quality best practices.

-Collaborating with executive leadership to prioritize data quality initiatives.

 

Image Source: Eckerson Group

Data Governance Structure











Analytics COE Models for Enterprises


Sample Digital Transformation Project Roadmap Checkpoints








Potential Value Opportunities



Potential Challenges



Candidate Solutions



Step-by-step guide for Example



sample code block

sample code block
 



Recommended Next Steps



  • No labels