Key Points

1. from concept and architecture to delivery, engineering is the difference
2. there are core engineering themes that lead to better software solutions
3. architecture is the plans for "your house". engineering is what you are living in
4. "like food, software has no value unless it's consumed" Jim 

References

Key Concepts

information-solution-architecture-themes-jim-mason.pdf

BCECRCTSSSSMTVLAV

The architecture themes are overall approaches to problems, solutions and the process of delivering solutions, not a pattern to solve a specific context or set of forces

Map the themes to design tools and processes where they fit during solution design

BPE - business process engineering

• defines measurable business value, costs, risks delivered for a solution

• reduces all 3 risks – business, project and technical ( by scoping )

CIP - continuous improvement process

• ensures continuous improvement in operational performance across the organization unit and outside parties
   

• background at BWC West - 1984
  before LSS - Six Sigma existed, in 1984 BWC West, defined the CIP process
• the ERP implementation team had designed many business process improvements while implementing MANMAN ERP
• after the system was running successfully in production, the team of mid-level operations staff continued to meet every 2 weeks to focus on the next items they could improve as a business or operations priority based on their own assessments of impact, value, risk as well as top management goals
• when significant funding or business approvals were required for a change, they submitted the business decision to senior management
• the business results from the BWC West CIP were clear and we took that process and starting using it in other business units
  
  much later came the branding of CIP with Six Sigma and LSS
• LSS can reduce your costs and help you retain and even gain more customers. LSS prescribes an improvement process known as DMAIC (Define—Measure—Analyze—Improve—Control).
• https://www.purdue.edu/leansixsigmaonline/blog/dmaic-vs-dmadv/
  
  
• PDCA is another approach to CIP
• PDCA refers to a repetitive model that includes four stages – Plan, Do, Check, and Act. It is used to achieve continuous improvement in business process management. The concept was introduced by Dr. Edward Deming in 1950
  
  
• CIP overview
• CIP - Continuous Improvement Process - a business operations model focused on continuous improvement on business goals and operating plan was the driver for the best run company I was part of. It was always driving operations excellence while driving improvement in services, operations across all departments ( an integrated version of Agile iterative improvement ). With its focus on metrics, it was always looking to find the right data and improve data quality. With its focus on process integration across departments, clients and vendors it was a prototype for todays blockchain consortiums that re-engineer and optimize business processes across communities and roles. It fit well in the "goals down, plans up" model driving operation change from the operations level. With both corporate strategic and tactical planning cycles driving goals down, it delivered consistently good results.

EOS - enterprise open-source

• low-risk, low-cost, normally high return software stack, framework and tools model in any environment

CDS - customer driven solutions

• customer continuously iterating executable use cases weekly to drive requirements, usage, design, quality

• working with customers, define the use cases, process requirements in scope for a release

RDD - Responsibility Driven Design for Requirements Definitions

• capture user requirements as a set of responsibilities for system actors from the use case list and process list

• iterate and refine the actor list and responsibilities as needed to provide conceptual level responsibilities with system actors
• actors defined by:
  • events consumed  ( responsibilities handled )
  • events produced  ( responsibilities delegated )
  • behaviors
  • data
  • relations:  collaborations, associations, dependents, dependencies
  • trust services: what trusts need to be established between objects, when, why, what proofs are needed for trusts
  • mapping: actors may be mapped to existing objects fully or partially where that applies using the object service catalog for existing systems to reduce "reinventing the wheel" where feasible
• Requirements are best expressed as a contract between clients that use a service and the service that implements the contract
• Service Contracts may evolve as the clients express more requirements or details on existing behaviors, events and data
• The consumption side of a service contract is expressed as XUCs typically
• The producer ( or implementation ) side of the contract is implemented by a service provider who is responsible for the behaviors, events and data
• Clients can invoke service behaviors, access exposed service data, and consume events produced by the service
• The service implements the behaviors, data interfaces and events defined in the service contract
• RDD requirements can be expressed as BDD stories or scripts
• some tools (  Spock, Karate etc ) make executable BDDs possible
• an implemented BDD that executes is one form of XUC

XUC - Executable Use Cases

• requirements are defined as executable use cases for a solution
• responsibilities are best defined as executable use cases as well
• all requirements should be testable as an executable use case
• if there isn't an XUC, don't build the solution or service that fulfills the requirement
• ideally, there is an E2M2 - Executable Enterprise MetaModel that XUCs are added to 
• the XUC approach automates SDP from requirements through QA ETE

CDD - Capability Driven Design

• given the RDD object model, define capabilities required from the RDD definitions and related NFRs ( Non Functional Requirements )

• what are the needs, priorities, problems, opportunities to solve? deliver? avoid?
  focus not on products but solution engineering
• CDD will enhance the actors and also add lower level services to implement many of the CDDs as needed
• mapping: actors may be mapped to existing objects fully or partially where that applies using the object service catalog for existing systems to reduce "reinventing the wheel" where feasible for the NFRs

Capabilities can be defined abstractly often

Use the 7 C's to make them concrete and executable through the ENTIRE SDP

most of all, make sure the capabilities realized are customer driven by executable use cases

7 C's of communications

clear, concrete, concise, courteous, correct, considerate, collaborative, cooperative, 

capabilities defined should support many of the ASCRUM concepts defined below

TDD - test driven delivery

• all phases driven by a library of executable test cases with matching masked data scenarios and automation

• tests are executable use case scenarios to implement and drive the entire delivery cycle.

SOE - service oriented engineering

• soa is a dream, soe is reality – it’s what got built and runs . it’s what matters. Not a paper design.

• soe delivers what soa promises

• multiple approaches to SOE design

SOE maturity model

>> Ensures well-engineered services that are easy to understand, manage, monitor and maintain by people and ARE - automated runtime environments

Services are not only separated into layers ( see Cloud service layers - IAAS, PAAS, SAAS ) but also segregated in Service Managers using SOLID principles

This improves on simple layering with better visibility and accountability

The SOE maturity levels can be defined and measured

SOE - Different Design Methods

3 views of a horse

1. structural 
   1. horse has a body, neck, head, 4 legs, tail
2. functional
   1. horse can walk, gallop, trot, canter, sleep
3. value
   1. horse can pull a plow, provide fun riding, pull a cart

Where to focus first?

Probably the value needed, then the function, then the structure

Wirfs-Brock on RDD - Responsibility Driven Design

http://www.wirfs-brock.com/PDFs/A_Brief-Tour-of-RDD.pdf

A_Brief-Tour-of-RDD.pdf

Smart Runtime Services can:

handle responsibilities, apply for jobs, learn new behaviors, negotiate relations and responsibilities, support policy management, document their models and behaviors and automate execution

Smart Service Interfaces - the job description for a set of Smart Service classes

Smart Service Classes - multiple classes can implement the same defintions differently for different behaviors to fit different contexts and test scenarios ( polymorphism )

Smart service objects are instance of Smart service class implementations

Smart service objects learn new behavior, produce new behavior and respond dynamically ( smart )

They are the foundation of decentralized smart systems that can support dynamic choreography of behavior and learn or create new behaviors

The same smart service objects CAN also be operated in the traditional centralized orchestration model as well

This makes digital transformations from centralized to decentralized systems much easier since the changes are normally well encapsulated with clear test scenarios that can automatically be executed in any environment as needed

Win - win - win

SRE - Smart Runtime Environment built using SOE principles and SRS

Given the capabilities of SRS automation ( they learn, adapt, disover, handle responsibilities, implement policies, manage events, trusts ), how complex do applications need to be that run on an SRE?  Probably not very complex

Cloud Service Layers - IAAS, PAAS, SAAS

Think of the standard cloud services models as a starting point for what features are managed by the customer vs the cloud provider

You can mix any combination of these models in solutions

The grey boxes show what the enterprise is responsible for in each layer

IAAS - OS, Middleware management - virtualization,servers, storage, networking, and data center

PAAS - Application, data management

SAAS - Application usage management

https://www.oracle.com/a/ocom/docs/dc/iaas-for-dummies-4thedition.pdf?source=:ad:pas:go:dg:a_nas:71700000086971898-58700007411666420-p66493910783:RC_WWMK170614P00033C0001:

Solutions define the Non-functional criteria required

• elastic scalability
• performance
• security
• predictability
• SLAs

Examples of Cloud layers

Dimensions for SOE delivery for clients

solutions

teams

technologies

SSM - smart service management. =.  ABC Architecture - Advanced Business Components manage value life cycle

• services should meet solution requirements through self management for operations, change management
• enable modularity, loose coupled processes with async transactions 
• leverage containerization for service independence and isolation

• ideally the services are self measuring and self managing

• where that’s not possible ( eg a broker lookup etc ), a serviceMgr performs the roles

• delegate responsibilities, intelligence, cooperation, support, alerts to services –
  they should be self managing and healing

• services are self documenting on all layers, service db

  • outputs uml, mmd files for review / analysis from tools logs events

• services execute in one or more modes ( run, debug, help, doc, test etc )
• services are aware of themselves, others
• services sense opportunities and challenges as events
• services communicate using events
• services understand their capabilities and the capabilities of their connected services
• services can search for others by capability and availability
• services understand advertising, search, negotiation, verification, validation, contracts, billing, payment
• see NCPS example - self instrumenting service that tracks its own performance

SDM - smart data management

• data should manage itself for operations, change management
• drive automation with metadata management for data and the environment
• SDM services fit data needs for:  volume, volatility, value, variety, variance, veracity, velocity, visualization
• variety includes: structured, unstructured documents and media
• SDM should support application needs for immutability, retention, reliability, replay, reuse

• the more intelligent the data, the less work for applications

• engineer intelligent data services that answer higher level questions

• expand process modeling to include decision modeling for each stage

• create agents that offer answers to the decisions at each stage

SDD - sprint driven delivery

• before you run and manage sprints, you need to prepare - Sprint 0 - the preparation sprint is task-based and critical to success
• all work for product releases, not just development, can be tracked in sprints
• the only exception is normally ongoing support for bug fixes etc

• break down deliverables into executable units of work that can be validated, ideally by users

• work sprint phases clearly EACH sprint – plan, deliver, test, review

• users involved at the start, end and testing of every sprint

• balance resource, deliverables and time in a sprint

• goals down, plans up

• daily standups

• sprint review reports

• sprint reviews

CDP - continuous incremental delivery process

• continuous incremental delivery process: from design through build, support, fix, test, deployment

• many small iterations built, tested, delivered and managed automatically provide much lower operational risks
• focus on evolutionary improvements using people and technology inputs

MDG - model driven generation engineering

• all development is model driven leveraging metadata where feasible

• models can be generated from an existing repository

• models can generate application artifacts to a repository

• round trip engineering ensures consistency between design and execution

• some portions of solutions, services require customization
• ideally customizations don't break the MDG metadata generation capability ( see Jim's UDX - logical user data extensions to core models )

3 ways to build solutions ( or grow corn )

1. Manually
   1. design, create the code and test
   2. manually plow the field, plant the seed, harvest
2. MDG - Model Driven Generation
   1. Automate development using model driven generation of code, tests from models
   2. use machines to plow, plant, harvest
3. SOM - Smart Object Managers
   1. Delegate the responsibilities to the SOMs and let them generate the code ( even dynamically when needed )
   2. use smart seeds and send the message "corn grow"

Use the "corn grow" example for farming

Use the horse example for 3 views of a horse - structure, function, value

Key examples for inheritance, encapsulation, polymorphism

Object responsibility concepts:  data, behavior, events   <<  only 2 have standard definitions commonly used in languages now

What's an object?  Hint: everything is an object —  explicit or implicit.  A serverless function always has an implicit object context

RT3 - right tools, training and talent to get the job done right

• if any of the 3 are less than correct, the whole delivery is at risk

• ever been on a project that had at least 1 of the 3 not 100% correct ? I thought not...

VE - validate everything = zero trust

• finally, trustless has become the accepted model for authentication and operation of systems, solutions
• Trustless should also be the model for quality - accept nothing as quality until claims are proven ( see Eiffel language pre, post assertions )

• clearly define ALL assumptions for usage, need, context, infrastructure, users etc

• validate ALL assumptions including support, training, volumes etc with expected and actual results

• add GAPS and OARS methods to improve process management

LNB - learning new behavior - teach people or objects - who has better retention?

• we build software from smart objects, not code
  we teach the objects new behavior using examples, use cases, metaphors, analogies and feedback

• when asked, the objects can learn new behavior sometimes on their own

• when asked, the objects can improve their performance using objectives, process and feedback

• when asked, the objects can be responsible for handling events

ARS - Application Recovery Services ( see DFR )

• any application can have issues: runtime failures, bad data etc
• applications have dependencies on many services and data sources
• need application recovery services ( typically used after support triage and problem support )
• pre-defined rules ( based on app configs ) to:
  • rollback to a point in time
  • issue corrective transactions
  • replay succeeding transactions to provide correct current state 

VCE - Value Chain Economies - Networks to optimize network value, efficiency, lower costs, risks

for more, see >> Modern Platform ODP supports VCE , DAN

• does not need to be a DLT network but DLT networks often focus on VCN outcomes
• Supports multiple parties in multiple roles on a network
• key design method for any DLT network
• See Solutions VCRS to decide how to realize
• See FACTUR3D.IO analysis for before and after impacts
  • s Blockchain Opportunity Assessment - BOA
• See Tokenomics for the models to incentivize network behaviors based on roles
  • m Token Economy Examples: Tokenomics

DFR - Design for Resiliency

• Rotation on environments resources assets where planned or needed
• Recovery Design:  BCP, RTO, RPO
• Reliability Design: What can't fill ( by design ) will >> Design Test for Failure, Performance vs Requirements
• Rerun: support transaction reprocess where needed across the VCN
• Replay: replay data transactions where needed
• Review: monitoring and alerting of events where needed
• Redesign: when the business requirements change or the technology operations capabilities change, consider redesign
• Re-engineer: where the business requirements haven't changed significantly but operations characteristics are no longer meeting needs

GOV = GTO > Governance, Trust, Outcomes

Better Governance = Better Trust = Better Outcomes

Proof of Governance is key

CBTP - Confidence Needs Understood, Belief the Right Resources Available, Trust in Delivery Priority for a Purpose in service provider relations

RME - Risk Management Engineering - Answer? - Consensus reached on "What's it take to do it right?"

Success and Failure outcomes have:

risks, impacts, recovery value and costs
what are the priorities to underwrite risk?
what are the risk premiums to pay for?
single or shared risk models?

Potential Value Opportunities

SDM - Smart Data Mgt 

dynamically defined data markets from datablox

build a market from datablox 

Data pipelines, data objects, data scripts ( groovy ) as peer streaming service for MySQL on request log
with data subscriptions for defined CDEs - critical data events, replays, pii, guikey, rbac, 

options ( MySQL, Postgres, SQLITE ??, Mongo ??  API agent ??  files ?? )

datablox - provides service, has requestSet, responseSet

#datablox have:

• duration
• type
• container blox
• contained blox
• configuration for data sources ( input, output, temp )
• ACL x role
• request map
• response map
• subscribers
• batch input
• stream input 
• request input
• validations with responses
• object schemas in and out
• pipeline operations
• error handler
• services: metadata mapper, directory, tempSpace

Datablox life cycle

1. Creation
   1. Datablox can be created automatically or manually from definitions ( metadata )
   2. create request can be defined by data gap: an output format for specific data sources 
   3. the datablox registry shows defined datablox definitions
   4. datablox can be: named, created as temporary or permanent, shared or dedicated by default

STS - Smart Trust Services Concepts

GEMS - Global Event Management Services

a pub / sub services for distributed event management across networks

flexible metadata definitions with shared repository

supports prioritizations, logging levels

has publishing, delegation and contracting capabilities in producer and consumer roles

allows for flexible push / pull configurations

configurable workflows that capture, filter, subscribe, publish and consume system and solution events flexibly across any landscape

dependencies include:

Kubernetes

Containers

Platform Agents for any platform environment

async message delivery

ASCRUM - Solution NFR Capabilities Metrics

Capabilities that higher value solutions deliver

for each of the capabilities listed below, how can your current solution design leverage these concepts?

ascrum NFRs - a set of capabilities that identify the value of different solution aspects - not related to solution functional requirements

• adaptable, automated, auditable, asynchronous
• secure, standards, supportable, serviceable, simple, smart, subscribeable, solid design principles 
• cost-effective, configurable, collaborative
• reliable, reusable, recoverable, risk-efficient, resource-efficient, reactive, resilient, replays 
• usable, useful
• modular, maintainable, manageable, monitorable, migrateable

Architecture Design Concepts

DATES - Decisions & Data, Automations, Trusts, Events, Services

• Decisions & Data are the decisions and the related policies made in the operation of the solution and data defines the inputs required for those decisions
• Automations defines the opportunity automate operations using Decisions, Policies and Procedures
• Trusts define the set of trusts needed by all stakeholders in each operating area of the solution ( beyond simple data and security trusts ) - policies define trusts and their handling
• Events define the events, producers, consumers, delivery systems that link all stakeholders in operating the solution - workflows are event-driven, dynamically created, found, subscribed
• Services define the services that all stakeholders provide to execute their responsibilities. These services are consumed by any authorized stakeholder in the solution

SOLID - Object-Oriented Design Principles

SOLID stands for:

• S - Single-responsiblity Principle
• O - Open-closed Principle
• L - Liskov Substitution Principle
• I - Interface Segregation Principle
• D - Dependency Inversion Principle

ABC - Advanced Business Component Architecture 

ABC provides consumable business components to provide services AND context for clients

ABC are defined by:

• ABC interface defining the ABC behaviors, attributes, events and relations
• ABC type ( or class ) that implements the interface. Using polymorphism, multiple types can implement the same interface differently to fit different use cases for similar services
• ABC instance ( or object ) is an instance of the type that has executable behaviors, session data

Typically, the client creates an ABC instance from an ABC type with a unique ID and assigns that instance to either a permanent or temporary session variable

For performance and efficiencies, the client may use a factory method that provides reusable instances of the ABC from a pool

ABC definitions are created and stored as part of the E2M2 ( see below )

For any requirement ( an executable test case ), the ABC is extended or customized to support the new test case

This allows validation of the E2M2 in the model runtime  environment with later generation and testing of runtime instances in other runtimes

ABC Component Definition = Smart Objects

an ABC Object is smart and:

• has metadata ( knows about itself )
• communicates with others ( shares info about itself )
• understands several "languages" for connectivity ( json messages ) and learns how to talk to them
• responds to: on-demand, batch and event requests in different modes
• is safely extensible
• can be extended as stereotypes ( datablox, processors, governors etc )
• is secure
• understands some specific packaging and deployment models based on type
• implements patterns as interfaces ( job descriptions )
• understands authorization roles as defined
• makes friends ( builds relationships ) of different types for different roles 
• has data ( knowledge )
• has behavior ( capability )
• has NFR capabilities that can be shared with friends
• has event support ( takes responsibility for events )
• can define events to notify others when requested on work done, state changes, exceptions
• has a set trusts with proofs that can be requested ( automated, digital verifications of trusts needed by clients - who signed this? who changed this ? etc )

Building executable ABC's they can operate on any platform and can be delegated responsibilities even at runtime

Consumers of ABCs never have to worry about how they are implemented

E2M2 - Executable Enterprise MetaModel - executable metamodel of the organization systems

>> Helps improve understanding and lower costs, risks, time to implement new and enhanced solutions and systems for a set of stakeholders and organizations

Beyond standared source repos and test tools, the EMM is an executable metamodel defining the goals, behaviors and outcomes of the organizations systems and solutions

An integrated EMM makes it far easier, faster with lower cost and risk to model potential changes to the organizations systems and solutions

The EMM execution capabilitiy makes it easy to understand the impacts of changes in one area on other areas of a system

It provides an executable framework where measurements can be made on effectiveness of solution designs, policies before they are put in operations in a production environment

E2M2 development

EMM - the Executable MetaModels

are version managed and executable interactively or in batch for any given test scope

for the execution scope, external systems, events, objects, data, trusts can all be mocked by default or customized with IOD - Interface Object Defintions

existing systems are harvested for metadata to add to the executable metamodel repositories

the executable metamodel can be harvested for it's interfaces, used to generate objects and services from the interaces to testing in a target test scope

MetaObjects - have behaviors, attributes, events and relations

the interactive metamodel is an object-oriented runtime with meta objects that implement behaviors, attributes, events and relations

Architecture Practice Concepts 

#business solution practice

success keys >>>

community building
business development
client success focus
factory production model
best non-functional features
standards based where it adds value
open-source where it adds value
architecture and engineering quality management
partnerships that add client value

#architecture practice

example ...
evaluate
is the process builder reusable now for tfin??

problems
shop for solutions

why?

can't afford to build without architecture
burning money faster than you can make

oti vs microsoft

machines to build components

Dave Thomas - software dividend
didn't need funding
highest quality software IBM

upgrade the place while the plane is flying
like PTP database re-engineering in prod

#non-functional requirements
dimensions

onboarding
education
consulting
security
scale
integration
reusability
quality management
operations RAS
maintenance
support
configuration management
policy management
process management
event management
customization
change management
community building
compliance with stds, regs
client success management
client usage kbase rewards

themes
rebuild the airplane while it's flying

Potential Challenges

Candidate Solutions

Step-by-step guide for Example

sample code block

Recommended Next Steps

Related articles