Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Who can see my data?

Who did see my data?


SAM - Smart Access Mgr for data



What the hell happened to my data?  Data Doctor

...

DMTT shows what happened to that data message


An example of a service for DMTT is https://debezium.io/

a streaming service for database change events with data 

Debezium is an open source distributed platform for change data capture. Start it up, point it at your databases, and your apps can start responding to all of the inserts, updates, and deletes that other apps commit to your databases. Debezium is durable and fast, so your apps can respond quickly and never miss an event, even when things go wrong.

Most commonly, you deploy Debezium by means of Apache Kafka Connect. Kafka Connect is a framework and runtime for implementing and operating:

  • Source connectors such as Debezium that send records into Kafka

  • Sink connectors that propagate records from Kafka topics to other systems

The following image shows the architecture of a change data capture pipeline based on Debezium:

Debezium ArchitectureImage Added



Who will fix my data ?  Data Janitor


SDM - Smart Data Manager

Knows metadata mgt policies and master data mgt policies are

detects anomalies in data vs policies

useful where standard data definitions for validation need to be dynamically responsive

Normally simple data validation by data type can cover many data validation needs prior to posting data

SDM really applies to complex or dynamic data policy enforcement

SDM can connect CDC services like Debezium through the standard CDC service interface and configuration file

A default CDC configuration for Debezium is provided but other CDCs can be added



What is my data telling me?  Data Whisperer

a semantic value problem - is it useful? how? why?


Interpreting data against scenarios for:

goals - sli - okrs

context

history

known or needed data system investments

forecasts

with impact forecasts before and after > FACTUR3DT.io





How can I feel good about my data ?  Data Therapist

...