Key Points

1. service life cycle: definitions, design, plans, delivery, manage, operate, govern, support, maintain
2. service consumption life cycle for a consumer:  key questions, assumptions on consumer service capabilities
3. services have flexible mapping to containers and platforms
4. services are accessed from a service registry ( broker ) that provides meta management of services and governance by id, name etc
5. clients connect to a service through the broker and bind to the service with a configuration
6. clients connect with an identity that maps to a role and set of authorizations and receive a JWT for the client session
7. a good service broker can work with multiple service types ( API, data, message, http, ftp etc )

References

Key Concepts

Declarative vs Imperative Coding

https://stackoverflow.com/questions/1784664/what-is-the-difference-between-declarative-and-imperative-programming

A great C# example of declarative vs. imperative programming is LINQ.

With imperative programming, you tell the compiler what you want to happen, step by step.

For example, let's start with this collection, and choose the odd numbers:

List<int> collection = new List<int> { 1, 2, 3, 4, 5 };

With imperative programming, we'd step through this, and decide what we want:

List<int> results = new List<int>();
foreach(var num in collection)
{
    if (num % 2 != 0)
          results.Add(num);
}

Here, we're saying:

1. Create a result collection
2. Step through each number in the collection
3. Check the number, if it's odd, add it to the results

With declarative programming, on the other hand, you write code that describes what you want, but not necessarily how to get it (declare your desired results, but not the step-by-step):

var results = collection.Where( num => num % 2 != 0);

Here, we're saying "Give us everything where it's odd", not "Step through the collection. Check this item, if it's odd, add it to a result collection."

Integration Model Concepts

APIs and Messaging patterns work reasonably well for modularity, independence, async processing for service consumers and producers for many use cases

There are multiple ways to integrate systems or services. APIs is just one method BUT it has strong advantages as a method because it provides better modularity and independence between the consumer and the provider than many other options and supports chain of responsibility pattern. Messaging also provides some added flexibilities for some use cases where modularity, independence and async processing are important. Both APIs and Messaging can benefit from state ledgers for operations management, governance as well.

G4 Pattern Summary

https://springframework.guru/gang-of-four-design-patterns/

Creational Design Patterns

• Abstract Factory. Allows the creation of objects without specifying their concrete type.
• Builder. Uses to create complex objects.
• Factory Method. Creates objects without specifying the exact class to create.
• Prototype. Creates a new object from an existing object.
• Singleton. Ensures only one instance of an object is created.

Structural Design Patterns

• Adapter. Allows for two incompatible classes to work together by wrapping an interface around one of the existing classes.
• Bridge. Decouples an abstraction so two classes can vary independently.
• Composite. Takes a group of objects into a single object.
• Decorator. Allows for an object’s behavior to be extended dynamically at run time.
• Facade. Provides a simple interface to a more complex underlying object.
• Flyweight. Reduces the cost of complex object models.
• Proxy. Provides a placeholder interface to an underlying object to control access, reduce cost, or reduce complexity.

Behavior Design Patterns

• Chain of Responsibility. Delegates commands to a chain of processing objects.
• Command. Creates objects which encapsulate actions and parameters.
• Interpreter. Implements a specialized language.
• Iterator. Accesses the elements of an object sequentially without exposing its underlying representation.
• Mediator. Allows loose coupling between classes by being the only class that has detailed knowledge of their methods.
• Memento. Provides the ability to restore an object to its previous state.
• Observer. Is a publish/subscribe pattern which allows a number of observer objects to see an event.
• State. Allows an object to alter its behavior when its internal state changes.
• Strategy. Allows one of a family of algorithms to be selected on-the-fly at run-time.
• Template Method. Defines the skeleton of an algorithm as an abstract class, allowing its sub-classes to provide concrete behavior.
• Visitor. Separates an algorithm from an object structure by moving the hierarchy of methods into one object.

Microservices.io Pattern repository

https://microservices.io/patterns/index.html

text starts here

Monolithic Application Pattern

Microservices Application Pattern

Please see the example applications developed by Chris Richardson. These examples on Github illustrate various aspects of the microservice architecture.

Resulting Context

Benefits

This solution has a number of benefits:

• Enables the continuous delivery and deployment of large, complex applications.
  • Improved maintainability - each service is relatively small and so is easier to understand and change
  • Better testability - services are smaller and faster to test
  • Better deployability - services can be deployed independently
  • It enables you to organize the development effort around multiple, autonomous teams. Each (so called two pizza) team owns and is responsible for one or more services. Each team can develop, test, deploy and scale their services independently of all of the other teams.
• Each microservice is relatively small:
  • Easier for a developer to understand
  • The IDE is faster making developers more productive
  • The application starts faster, which makes developers more productive, and speeds up deployments
• Improved fault isolation. For example, if there is a memory leak in one service then only that service will be affected. The other services will continue to handle requests. In comparison, one misbehaving component of a monolithic architecture can bring down the entire system.
• Eliminates any long-term commitment to a technology stack. When developing a new service you can pick a new technology stack. Similarly, when making major changes to an existing service you can rewrite it using a new technology stack.

Drawbacks

This solution has a number of drawbacks:

• Developers must deal with the additional complexity of creating a distributed system:
  • Developers must implement the inter-service communication mechanism and deal with partial failure
  • Implementing requests that span multiple services is more difficult
  • Testing the interactions between services is more difficult
  • Implementing requests that span multiple services requires careful coordination between the teams
  • Developer tools/IDEs are oriented on building monolithic applications and don’t provide explicit support for developing distributed applications.
• Deployment complexity. In production, there is also the operational complexity of deploying and managing a system comprised of many different services.
• Increased memory consumption. The microservice architecture replaces N monolithic application instances with NxM services instances. If each service runs in its own JVM (or equivalent), which is usually necessary to isolate the instances, then there is the overhead of M times as many JVM runtimes. Moreover, if each service runs on its own VM (e.g. EC2 instance), as is the case at Netflix, the overhead is even higher.

Issues

There are many issues that you must address.

When to use the microservice architecture?

One challenge with using this approach is deciding when it makes sense to use it. When developing the first version of an application, you often do not have the problems that this approach solves. Moreover, using an elaborate, distributed architecture will slow down development. This can be a major problem for startups whose biggest challenge is often how to rapidly evolve the business model and accompanying application. Using Y-axis splits might make it much more difficult to iterate rapidly. Later on, however, when the challenge is how to scale and you need to use functional decomposition, the tangled dependencies might make it difficult to decompose your monolithic application into a set of services.

How to decompose the application into services?

Another challenge is deciding how to partition the system into microservices. This is very much an art, but there are a number of strategies that can help:

• Decompose by business capability and define services corresponding to business capabilities.
• Decompose by domain-driven design subdomain.
• Decompose by verb or use case and define services that are responsible for particular actions. e.g. a Shipping Service that’s responsible for shipping complete orders.
• Decompose by by nouns or resources by defining a service that is responsible for all operations on entities/resources of a given type. e.g. an Account Service that is responsible for managing user accounts.

Ideally, each service should have only a small set of responsibilities. (Uncle) Bob Martin talks about designing classes using the Single Responsibility Principle (SRP). The SRP defines a responsibility of a class as a reason to change, and states that a class should only have one reason to change. It make sense to apply the SRP to service design as well.

Another analogy that helps with service design is the design of Unix utilities. Unix provides a large number of utilities such as grep, cat and find. Each utility does exactly one thing, often exceptionally well, and can be combined with other utilities using a shell script to perform complex tasks.

How to maintain data consistency?

In order to ensure loose coupling, each service has its own database. Maintaining data consistency between services is a challenge because 2 phase-commit/distributed transactions is not an option for many applications. An application must instead use the Saga pattern. A service publishes an event when its data changes. Other services consume that event and update their data. There are several ways of reliably updating data and publishing events including Event Sourcing and Transaction Log Tailing.

How to implement queries?

Another challenge is implementing queries that need to retrieve data owned by multiple services.

• The API Composition and Command Query Responsibility Segregation (CQRS) patterns.

Related patterns for Microservices

There are many patterns related to the microservices pattern. The Monolithic architecture is an alternative to the microservice architecture. The other patterns address issues that you will encounter when applying the microservice architecture.

• Decomposition patterns
  • Decompose by business capability
  • Decompose by subdomain
• The Database per Service pattern describes how each service has its own database in order to ensure loose coupling.
• The API Gateway pattern defines how clients access the services in a microservice architecture.
• The Client-side Discovery and Server-side Discovery patterns are used to route requests for a client to an available service instance in a microservice architecture.
• The Messaging and Remote Procedure Invocation patterns are two different ways that services can communicate.
• The Single Service per Host and Multiple Services per Host patterns are two different deployment strategies.
• Cross-cutting concerns patterns: Microservice chassis pattern and Externalized configuration
• Testing patterns: Service Component Test and Service Integration Contract Test
• Circuit Breaker
• Access Token
• Observability patterns:
  • Log aggregation
  • Application metrics
  • Audit logging
  • Distributed tracing
  • Exception tracking
  • Health check API
  • Log deployments and changes
• UI patterns:
  • Server-side page fragment composition
  • Client-side UI composition

Microservice Orchestration vs Choreography

https://www.softobiz.com/microservice-orchestration-vs-choreography/

Microservice Orchestration Vs Choreography-softobiz.com.pdf

How to stitch these service modules together?

Orchestration = central controller

In an orchestra, we have a central man called the “orchestrator”. Therefore, he is responsible for invoking all the sounds.

Orchestration Benefits

Easy to maintain and manage as we unify the business process at the center.
In synchronous processing, the flow of the application coordinates efficiently.

Limitations

Creating dependency due to coupled services. For example, if service A is down, service B will never be called.
The orchestrator has the sole responsibility. If it goes down, the processing stops and the application fails.
We lose visibility since we are not tracking business process.

Microservice Choreography

It is the other way to achieve microservice interaction.

We want to avoid dependencies in a microservice architecture. So, that each service can work independently. Choreography solves this issue which was the main challenge in orchestration approach.

You can imagine Microservice Choreography as a belle dance (as shown in the above picture). In it, every individual performs steps independently.

Similarly, in microservice Choreography, every microservice performs their actions independently. It does not require any instructions. It is like the decentralized way of broadcasting data known as events. The services which are interested in those events, will use it and perform actions. This is also known as reactive architecture. The services know what to react to, and how-to, which is more like an asynchronous approach.

Benefits

Enables fast processing. As no dependency on the central controller.
Easy to add and update. The services can be removed or added anytime from the event stream.
As the control is distributed, there is no single point failure.
Works well with the agile delivery model, as teams work on certain services rather than on entire application.
Several patterns can be used. For example, Event sourcing, where events are stored, and it enables event replay. Command-query responsibility segregation (CQRS) can separate read and write activities.

Limitations

Complexity is the concerning issue. Each service is independent to identify the logic and react based on the logic provided from the event stream.
The Business process is spread out, making it difficult to maintain and manage the overall process.
Mindset change is a must in the asynchronous approach.

Most of the times, these approaches don’t work well in architecture. So, what is the solution in these use cases?

Hybrid

That is to say, the hybrid approach can solve the problem. For example, if we have a mix of synchronous and asynchronous blocks of activity. So consequently, one or more hybrid pattern can add value to the projects.

Hybrid is the combination of the orchestration approach and choreography. In this approach, we use orchestration within the services whereas we use choreography in between the services.

A Hybrid approach like others is two-edged. Let’s discuss:

Benefits

• The Overall flow is distributed. Each service contains its flow logic.
• Services are decoupled (but to an extent).

Limitations

• The coordinator is coupled with the services.
• If the coordinator goes down, it impacts the entire system.

RDD - Responsibility Driven Design - Wirfs-Brock

Why RDD?

more flexible than reactive services design

doesn't force everything to an API service model - just what should be an API vs a service vs a library vs a function

more natural way to find events produced and consumed by starting with business or client use cases

provides an easier way for business users to drive requirements specifications using natural language ( vs powerpoint logic )

Reactive Microservices based on event listeners

https://sites.google.com/a/mammatustech.com/mammatusmain/reactive-microservices

event maps, event storming in the context of microservices

effective really for software engineers but not so much for others

Benefits of Real OOP using Smalltalk before other languages improved

medium.com-The Forgotten History of OOP.pdf

https://medium.com/javascript-scene/the-forgotten-history-of-oop-88d71b9b2d9f

Interoperable Blockchain ACID transaction pattern

blockchain interoperability on atomic transactions using escrow transactions for DvP

context

assume 2 different orgs on 2 different blockchains on 2 different platforms want to execute a sale using an order (any type of asset )

blockchains implement escrow transaction interface services in a smart contract

2 independent chains A and B can send messages using a common protocol ( gRPC or ??? )

party A wants to buy X from party B for Y currency amount

problems

how to ensure only authorized parties participate in the transactions for settlement of a trade agreement

how to ensure both parties will execute their transaction once and only once to complete the purchase transaction

how to ensure that the purchase is cancelled if either party fails to complete their part of the purchase transaction within the designated time limit

how to prevent either party from failing to deliver or accept delivery of assets or funds per the terms of the trade agreement

pre-requisites

network connectivity and interoperability across all related networks

blockchain directory services and access

identity, registration and access management services for all parties in a transaction

each party implements the escrow transaction interfaces as buyer, seller and their agents

each party communicates using one of the supported protocols: gRPC or REST api or ?

each party digital certificates to sign and execute transactions identifying the authorized party

gateway nodes connect 2 different logical blockchain networks directly or indirectly ( blockchain networks normally not physical but virtual overlay networks across multiple provider networks )

solution

use an escrow transaction with smart contracts that

records each parties performance of the obligation

This follows a saga pattern to implement the escrow transaction (eg a purchase ) as a set of linked transactions in smart contracts

Singe Use Tokens are used to guarantee that only this single reservation transaction can be executed once by the designated party

step 0 - party A agrees to buy X from party B for Y currency amount agreeing to a set of policies to execute the transaction for the buyer and the seller roles

step 1 - reservation for payment from A to B with a token,  reservation for delivery from B to A with a token

step 2 - when both reservations complete, the reservations are executable as smart contracts by each party

each party executes their reservation transaction via smart contract using the supplied single use token created for just this transaction

A executes B's reservation using the token from B

B executes A's reservation using the token from A

alternate scenarios

both reservations are not completed within the agreed time limit, all reservations and tokens are cancelled and resources revert to current owner control

the purchase policy agreed to may include automated contracts to execute actions to reverse one or both reservation executions or cancel the reservations updating the blockchain

step 3 - if both reservations are not executed as transactions within an agreed time limit, each party is notified to manage the purchase based on agreed policy

step 4 - when both reservations have been executed by each party, all parties are notified of the completion of the escrow transaction

step 5 - any dependent actors are notified of the completion of the purchase which may trigger additional transactions ( eg start an automatic purchase warranty etc )

issues with simple saga pattern above - why escrow manager adds value

IF the trade settlement is executed simple on 1 network, real-time using a synchronous model it may work well using simple HTLC 

AND both parties fulfill all their obligations to deliver and accept delivery of assets and funds

OTHERWISE 1 or both parties may fail to deliver or accept delivery of the assets or funds for settlement after the trade agreement is set

P2P Escrow Transaction Pattern

goals

1. 2 parties complete an atomic swap of asset 1 for asset 2 without risk
2. if swap can't meet completion criteria, swap transaction is reversed for all parties returning initial states for each
3. optionally, a 3rd party notary service may need to approve the transaction
4. optionally, a 3rd party observer service may record the transaction
5. optionally, the 2 parties may be on different networks

model

a chain manager ( on a chain ) for other chain transactions ?? ( or a configurable chain model ? )

specific validators only for that transaction type and parties are active

1. multi-party contract with roles where config defines terms of the swap
2. each party is identified and claims a role ( eg buyer, seller etc )
3. define multi-step transaction types
4. create transaction instance using the shared transaction manager with workflow from the contract config for that transaction type
5. each transaction has a unique ID
6. set the parties who validate the transaction on each ledger
7. contract takes control over both assets from each party using locks and confirms locks ( if funds or asset limited, policy may allow partial transactions )
   1. obligation set for each asset in the contract locking a portion of the assets for a time period
   2. after lock done, the lock status for all assets in the contract is shared
   3. if all assets in the contract locked, next step executes ( normally the trade )
   4. funds transferred to new party,  assets transferred to new party
   5. after delivery completed, obligations removed
8. after each step, the contract instance is updated for both parties and confirmed
9. signatures and consents required by all parties for each step
10. state changes are validated  as complete on each ledger for the current step before moving to next step - 100% endorsement of each update by relevant parties
11. if completion criteria for a step not fulfilled by X time, the transaction is cancelled and all parties are restored to initial states
12. completion messages recorded and events published on success or failure of the transaction

context

assume 2 different orgs on 2 different blockchains on 2 different platforms want to execute a sale using an order (any type of asset )

a blockchain with 2 parties as nodes each running a ledger copy and a contract

assume direct payment ( no margin loan ) on purchase transaction

blockchains implement escrow transaction interface services in a smart contract

2 independent chains A and B can send messages using a common protocol ( gRPC, API, messages, db transactions or ??? )

party A wants to buy X from party B for Y currency amount

problems

how to ensure only authorized parties participate in the transaction

how to ensure both parties will execute their transaction once and only once to complete the purchase transaction

how to ensure each party will accept delivery of assets and funds

how to ensure that the purchase is cancelled if either party fails to complete their part of the purchase transaction within the designated time limit

pre-requisites

network connectivity

blockchain directory services and access

identity, registration and access management services for all parties in a transaction

each party implements the escrow transaction interfaces

each party communicates using one of the supported protocols: gRPC or ?

each party digital certificates to sign and execute transactions identifying the authorized party

gateway nodes connect 2 different logical blockchain networks directly or indirectly ( blockchain networks normally not physical but virtual overlay networks across multiple provider networks )

solution

use an escrow transaction with smart contracts that

records each parties performance of the obligation

This follows a saga pattern to implement the escrow transaction (eg a purchase ) as a set of linked transactions in smart contracts

Singe Use Tokens are used to guarantee that only this single reservation transaction can be executed once by the designated party

step 0 - party A agrees to buy X from party B for Y currency amount agreeing to a set of policies to execute the transaction for the buyer and the seller roles

step 1 - reservation for payment from A to B with a token,  reservation for delivery from B to A with a token

step 2 - when both reservations complete, the reservations are executable as smart contracts by each party

each party executes their reservation transaction via smart contract using the supplied single use token created for just this transaction

A executes B's reservation using the token from B

B executes A's reservation using the token from A

alternate scenarios

both reservations are not completed within the agreed time limit, all reservations and tokens are cancelled and resources revert to current owner control

the purchase policy agreed to may include automated contracts to execute actions to reverse one or both reservation executions or cancel the reservations updating the blockchain

step 3 - if both reservations are not executed as transactions within an agreed time limit, each party is notified to manage the purchase based on agreed policy

step 4 - when both reservations have been executed by each party, all parties are notified of the completion of the escrow transaction

step 5 - any dependent actors are notified of the completion of the purchase which may trigger additional transactions ( eg start an automatic purchase warranty etc )

Hashed Timelock Contract  |  Hashed Timelock Agreement

https://www.investopedia.com/terms/h/hashed-timelock-contract.asp

https://academy.binance.com/en/glossary/hashed-timelock-contract

https://docs.lightning.engineering/the-lightning-network/multihop-payments/hash-time-lock-contract-htlc

Hash Time Locked Contracts (HTLCs) Explained

https://medium.com/liquality/hash-time-locked-contracts-htlcs-explained-e88aa99cc824

Refund scenario for HTLC

 leverage HTLCs in Atomic Swaps, but also lead industry-wide efforts to standardize their implementation across different applications and blockchains.

Bitcoin UXTO model

https://river.com/learn/bitcoins-utxo-model/

• Bitcoin does not have accounts with balances. Instead, individual coins are owned by Bitcoin users.
• An Unspent Transaction Output (UTXO) is a discrete piece of bitcoin. UTXOs are used as the inputs of every Bitcoin transaction.
• The UTXO model makes Bitcoin more auditable, transparent, and efficient than traditional financial systems, which rely on accounts, balances, and third parties.

Hash time lock cross network example from EU Stella DeFi settlement report

https://www.ecb.europa.eu/pub/pdf/other/stella_project_report_march_2018.pdf

Potential Value Opportunities

Potential Challenges

Candidate Solutions

Step-by-step guide for Example

sample code block

Recommended Next Steps

Related articles