References
The Problem: Trust is Low
What's the biggest problem we face today?
War, Hunger, Poverty, Clean air, Clean water, Crime, Social unrest, Education, Work, Lack of common values ??
Look at the 17 UN SDGs
They attempt to address those issues.
What do the SDGs overlook?
Population Growth negatively impacts many of the SDGs – not mentioned
Lack of Trust impacts all the SDGs - not mentioned
The Beatles song from 1968: Strawberry Fields defines the problem > "Nothing is Real"
'Cause I'm going to strawberry fields
Nothing is real
And nothing to get hung about
Strawberry fields forever
i>> With our technology connected world, the problem of what's true and trust have increased a 100 times compared to life 200 years ago
200 years ago, the only things you couldn't verify with your 5 senses easily were God, the Devil and germs
Today, what CAN you verify 100% even if you use your 5 senses?
Re-imagine DLT as STS - Smart Trust Services
can be implemented, consumed & operated as platforms, services, capabilities, features, components, extensions, integrations, interoperable communications
- solutions -
- ssi, did, vc, crypto, digital assets life cycle, custody, trading, markets, security, audits, surveys,
- capabilities – can be engineered as features of, integrated with, extensions of or interoperable with: other solutions and services
- verifiable history, counter-party trusts, identities, credentials, signatures, contracts, governance, consensus, policy management, trades, payments, proofs, votes, pogs, non-repudiation, tamper-proof, fault-tolerant system of record, faster reconciliations on proofs with chains, access controls (keys, ZKP ), observability & audits, version management
- services - smart trust services, like others, are composable
- platforms – platforms partially implement capabilities
- DLT: Fabric, Indy, Aries, Besu, Polygon, Corda, Firefly, DAML,
- technologies - technologies supporting STS
- encryption, secrets, mpc, confidential compute, wallets, signatures, tokens, ACID transactions, AI, analytics, DLT, cryptography, data management, learn, MDS - metadata services, Doc services ( see plantuml ), GEMS ( Apache Mesh ), distributed data,
- models & patterns
- async events ( GEMS ), CQRS, views, streams, workflows, replicas, replay, recovery, resiliency, GPMS,
Blockchain has some features that can add value to STS
s Enterprise Blockchain Concepts and Value# BUSINESS VALUE IMPACTS POSSIBLE with Blockchain
m TOIP Trust Over IP# Key DLT features create a proof system
Blockchain has limitations that STS should avoid
s Enterprise Blockchain Concepts and Value# Blockchain Challenges
STS Assessments
Key Questions
- What is the use case in scope?
- Who are the parties, roles and goals?
- What are the parties relationships, counter-party challenges?
- What are the operations and processes?
- How are DATES defined and used in the solution 3P ( policies, procedures, processes ) now? ( Decisions, Automations, Trusts, Events, Services )
- What are the proofs needed? Why? How?
- FACTUR3DT.i without STS
- FACTUR3DT.o with STS
Key Points
SLT:DS features and capabilities
- It's not DLT ( or DAML or Corda or ?? )
- has specific features for specific value impacts
- goals include: separation of producer / consumer roles in data services, add only key proofs to create trust for given use cases
- produce effective trust service models
- provides full ACID support
- favors throughput ( TPS ) over latency ( time to commit, finalization )
- favors async over sync processing where possible to improve throughput
- implements specific patterns to address specific problems for selected use cases
- intended to support VCE: Value Chain Economies for identified parties, governance and compliance services
- governance is independent of the technology
- deployment topologies are independent of the technology
- applies Trust engineering principles to deliver specific trust benefits not covered by Distributed DBs
- avoids some DLT features that limit performance and scalability or adds strategies to manage those challenges
- rethinks transaction life cycle and events as: request, capture, validate, process, approve, commit, finalize, response
- leverages open-source solutions wherever feasible
- uses interfaces and adapters to support multiple implementations of a feature with default implementation as an executable example
- uses adapters, containers and wrappers to minimize direct dependencies on 3rd party components and services
- minimizes cloud specific dependencies on client interfaces ( high priority ) and implementations where feasible
- support zero trust security where feasible
- support private, shared, aggregated data objects, messages and streams
- support data encryption in flight, at rest, in compute where needed
- leverages user defined policies to control design, development, test, deployment, operation and support where feasible
- maximizes performance for write and read use cases for high throughput transaction tables
- uses key - value stores for text, blobs for fast write for big data tables in addition to RDBs
- supports generated UIDs, primary auto-gen numeric or character format surrogate key IDs for primary keys with timestamps
- uses control tables to: create control blocks of related transaction records for data updates, transaction status
- uses control tables for data rollback, replay by table segment as needed
- uses data hashes to link transactions to improve tamper-resistance and data provenance
- data distribution to nodes: uses archivers and relayers to move data blocks from local to remote nodes
- provides directory services to search and locate data based on metadata
- provides sharing data blocks using efficient protocols: simple gossip, protobufs etc over VPN or MTLS connections
- uses shards when needed to speed search and improve data obfuscation when configured in policies
- provides client access support using compatible JDBC, ODBC, APIs, grpc and message interfaces when possible
- provides dynamically configurable logging capabilities based on information levels
- supports memory caches configurable for performance use cases
- provides processing priorities for different threads ( real-time vs batch ) to maximize resource utilization when configured
- provides procedure support directly or via request / response tables and threads for external languages
- allows for flexible node topologies for different decentralization variations and performance on physical data distributions
- allows for virtual nodes and multi-tenancy to provide logical access for organizations on the same physical nodes
Target Use Cases and Dependencies
There are many dependencies, key ones include:
- reuse or extension of existing database solutions ( SQL and NoSQL )
- leverage open-source data lake and data mesh technologies
- open-source solutions, tools and frameworks for data warehouses, aggregation, analytics services
- use open-source security frameworks wherever feasible
- Master Data Management synchronization ( centrally or distributed )
- Software metadata management repositories ( central and distributed )
- Secure remote privileged network access ( OpenSSH, Putty etc )
- Secure network, storage, compute services
- Kubernetes for virtual node management
- Containerization where feasible to improve portability, platform independence
Key Use Cases in Scope
- Big Data, high speed transaction life cycle management
- Data suspension events for input data failing validations based on data policies
- ACID transactions with definable commit boundaries
- Shared Ledgers of Tamper resistant data across virtual data networks
- Multiple, concurrent active - active data management services for nodes when use case fits
- Data finalization events for shared ledger updates
- Basic data work flows for multi-party review, signatures policies where needed
- Multi-tenant support and architecture for multiple parties sharing nodes or clusters as virtual nodes with private ledger transactions
- Selective Data rollback, replay capabilities
- Data processing prioritization policies
- Smart, dynamic log level management and event notification
- Smart schema management for SQL and NoSQL data platforms ( SQL schema management depends on Liquibase )
- Reduction of platform dependencies for client solutions and data implementation services where feasible
- Remote data solution maintenance fix pack life cycle management
- validate target ready for change, apply fix pack, test changes, when tests fail - rollback changes, retest target for initial data states
- Remote Automated test of data maintenance fix packs
- Ability to rollback, reapply data maintenance fix packs in sequence
- Basic data mapping and transformation services using SQL or Object scripts or external routines
- Data quality audit policies
Key Use Cases Out of Scope
- limits on data quality management
- limits on dynamic data access remediations based on policies
- limits on finalization performance to optimize throughput
- limits on capabilities based on dependent services
- extensive data analytics
- full data warehouse support
- AI data quality tools
- Quantum platform support
References
Reference_description_with_linked_URLs_______________________ | Notes______________________________________________________________ |
---|---|
Key Concepts
Words matter in technology - they set expectations good and bad
replace
blockchain
DLT
on-chain
off-chain
digital twin
use instead
VSLT - virtual smart ledger technologies
shared ledger? sharing happens multiple ways
shared current ledger
shared history ledger
shared data
shared current data
shared history data
private ledger
private data
backed digital asset
transaction life cycle events - created, authorized, validated, processed, approved, committed, posted, finalized
ledger life cycle events - created, updated, offlined, onlined, current, outdated, archived, purged, used, unused, locked, unlocked,
topology for SLT
- virtual DAO version 3 for operations, governance - ( like DTCC with trust, low collateral in efficient decentralized environment )
- virtual organization - VO - OO - Operational Organization
- access to organization data, operations on smart contracts, transaction workflows, transaction approvals, accounts, wallets, virtual ledger
- virtual organization node - VON - MO - Manager Organization = OO + more
- full node with all VO + node management, smart contract management, transaction endorsements, private ledger
- organization node - ON - NO - Node Organization = MO + more
- cluster - CL
Blockchains or Trust chains?
Almost all blockchain platforms have a performance issue on consensus methods and block ordering
Consider Trust data chains – trusted data chained for tamper-resistance as an alternative
Features of Trust data chains
- Write only ledgers
- All Parties can access ledgers via API and data services easily
- When required, parties can have read-only ledger access, write ledger access or both IF the governance model allows it
- Every ledger transaction has a permanent id and a version id.
- Ledger transactions have a life cycle with commit and finalization different life cycle events
- Ledger transactions can have linked off-ledger transactions paired ( my paycheck has amounts on ledger and a proxy id but no PII data ) by transaction and version ids
- Ledger transaction data and off-ledger transaction data are each hashed and hashes recorded in the matched transaction hash table
- Ledger transactions can be managed as a group using the transaction control table with a configurable mapping of transactions per control record
VSLN - Virtual Smart Ledger Networks
VSLN - each ledger type knows how to connect to the VSLN standard interface. VSLN can operate transaction protocols with optional capabilities to support many transaction use cases
VSLN key features - STS - smart trust services for trusts & decisions, privacy, multi-tenancy, virtual smart service layers, extensible component architecture, event-driven services, IAM for authentication & authorization services, pluggable services with interfaces and default service implementations, famous ABC parts
VSLN concepts
- vnet - a logical subnet of a VSLN network that is a named scope that parties can join directly through a vnode or indirectly through a vprovider
- vnode - a virtual node that implements a set of services for client roles. The vnode provides a variety of services for the ledger network including:
- multi-tenant capability
- management of the network within a vnet
- validators for actions, transactions on the vnet
- replicas that receive copies of events, transactions, trusts, decisions, requests, responses, procedures
- operators ( organizations, individuals with accounts ) operate and manage services in assigned roles on permissioned networks with access to assigned vnodes
- parties ( organizations, individuals with accounts ) use services in assigned roles on permissioned networks with access to assigned vnodes
- contracts - secured, digital, business contracts that are executed, signed, approved ( and notarized if required ), committed and finalized on VSLN
- transaction consensus - POA and other methods as appropriate
- client access to VSLN - rpc, api, vnode sdk
virtual smart ledger model - all parties create transactions on a virtual smart ledger network ( VSLN ) - made of multiple, dynamically coordinated ledgers
VSL supports different roles and optional vnode service types to fit use cases & organization needs
Manager role and optional vnodes - manage and govern the VSL
Manager nodes control and ( typically ) govern the operation of the VSL network
There may be more than 1 manager node based on the requirements
These nodes can also operate as shared nodes to host virtual nodes
Validator role and optional vnodes - validate transactions on the VSL
Validator nodes are defined by the use case and the needs of an organizations that will operate them
Validator nodes are involved in validating transactions of different types
Validator node roles are assigned by manager nodes
These nodes can also operate as shared nodes to host virtual nodesReplica role and optional vnodes - operate replicate nodes on the VSL
Replica nodes are defined by the use case and the needs of an organizations that will operate them
Replica nodes are receive replica updates of transactions of different types
Replica node roles are assigned by manager nodes
These nodes can also operate as shared nodes to host virtual nodesVirtual role and optional vnodes - provide virtual operations capabilities, identity, private transactions and data to organizations not directly hosting a node
Virtual nodes are defined by the use case and the needs of an organizations that will operate them
Virtual nodes are created for organizations to operate as private organizations on the VSL
Virtual nodes have their own organization identities, private transactions & private data and data access
Virtual nodes can operate methods and smart contracts that assigned to them
Virtual node roles are assigned by manager nodes
VSL global event management support - GEMS
Integrating the activities and results of operations on a VSL requires global event management support
Some DLT platforms provide basic, integrated GEMS ( eg Firefly Open-Source )
VSL services
Like many platform concepts, a VSL would support a variety of service types including REST apis and more
SGC - Smart Governed Contracts - VSL smart contracts - different than a blockchain smart contract
- Very different - goes beyond ERC3643 permissioned governed contracts for tokenized assets
- Adds more asset attributes: owner, controller, processors, observers,
- Adds more states with events to transaction life cycle: request > prepare > govern > approve > execute > commit > sign > finalize
VCE Solution Process for VSLN ( Virtual Smart Ledger Net )
Potential Value Opportunities
Potential Challenges
Candidate Solutions
Step-by-step guide for Example
sample code block