Firefly: Solution Review
- jimm1
Key Points
References
| Reference_description_with_linked_URLs_______________________ | Notes______________________________________________________________ |
|---|---|
| Firefly: Web3 Blockchain framework swt | |
| Firefly-v1-demo-p1 gdoc *** | |
| Firefly wiki | |
| Nicko Guyer's article on Firefly getting started - 2022 | |
| Firefly discord channel | |
Key Concepts
Web Solutions are Complex
Firefly Core - Hyperledger Open Source Project
Firefly Enterprise Version with Proprietary Extensions
a>> firefly deep dive
enterprise clients in production w similar resiliency models?
enterprise feature docs
alignment to our capabilities set
token mgt types, life cycle - ERC types, custom TTF types
transaction mgt life cycle customization ( commit, finalization, exception)
transaction consensus by contract
GEMS - global event mgt system
BPMN and JEPL work flows
off-chain db options w replication
private data options
zkp, zkevm,
DLT interfaces and connector updates for DLT currency - roadmap now?
fabric connector support etc? k8s, k8 package builder, api gateway
accelerators - openapi gen, sandbox ( for fabric ? )
customization levels w interfaces, policies
solution portability ( vs vendor lockin, enterprise, eos on eos )
https://www.planetcrust.com/whats-vendor-lock-in-and-how-to-avoid-it?utm_campaign=blog
a>> BRD xchain
use cases - PvP, DvP,
solutions
a>> firefly
ff inventory
map 2 capabilities
define set of use cases
evaluate use cases - functional, nfr
build business case
our delivery plan - get learnings from Kaleido, Swift
a>> get tier 2 components list
layers
- ingestion > routing, api, grpc, messages, DLT, files
- processors > engines
- connectors > services connectors
- services > persistence, analytics,
management
- < containers
- < kubernetes
- < prometheus, grafana
Kaleido Platform running Digital Asset Solutions
Kaleido Platform extends Firefly Enterprise
Kaleido’s Digital Asset Platform is designed to meet the demands of this evolving market. The most connective, customizable, and compliant platform available, it’s built to grow and adapt.
On May 10th at 10am ET, Lana and Nick, Kaleido Solution Architects, will show how it simplifies asset tokenization at an institutional scale with next-gen functionality, including the ability to:
- Connect to any chain, public or private
- Quickly deploy asset templates for leading and emerging token standards
- Manage the full smart contract life cycle, including through regular upgrades
- Generate APIs for any smart contract at the click of a button
- Plug in your custody solution of choice
- Automate governance with a configurable policy engine
- Manage multiple apps from the same console
- And more
Kaleido - Digital Assets Lessons Learned - Digital Asset Tokenization is primary focus everywhere in 2024
slides on line
streamlines the management of smart contracts throughout the contract lifecycle webinar
https://www.kaleido.io/resources/kaleido-asset-platform-in-depth-smart-contract-management
Nick Gaski, Kaleido Solution Architect shows you how the platform simplifies smart contract management at any scale by:
- Efficiently deploying smart contracts across diverse blockchain networks
- Handling tricky problems around the smart contract lifecycle, such as upgradeable smart contracts
- Automatically generating easy-to-use APIs from any smart contract
- Providing a robust API surface to automate builds and actions
- Enabling automated notifications and alerts for contract events
Platform In-Depth: Platform Overview
https://www.kaleido.io/resources/kaleido-asset-platform-in-depth-platform-overview
The tokenization of real-world assets is a quickly expanding opportunity for businesses in every industry.
The Kaleido Asset Platform has been designed from the ground up to help businesses seize this opportunity in the multi-chain, multi-standard world of web3. It's the most open, pluggable digital asset and tokenization platform in the industry.
This webinar is the first of a series of webinars covering the Kaleido Asset Platform in depth, including key features like:
- Advanced connectivity: Connect to any chains, public or private, or launch your own chain on the protocol of your choice
- Click button asset creation: Quickly create and tokenize digital assets with templates for leading and emerging token standards
- Pluggable custody: Deploy a range of native custody options or integrate with the third-party custody providers of your choice
- Robust policy engine: Automate employee access and asset controls to simplify compliance and oversight
- Unmatched smart contract management: Manage smart contracts at scale, including through regular upgrades
- Open source engine: Avoid vendor lock-in with an open-source orchestration engine powered by Hyperledger FireFly
- Powerful data indexing: Track assets across multiple chains with simple indexing of wallet balances and other on-chain data
- Flexible deployment options: Tailor hosting to your needs with dedicated cloud or on-premises deployments
- And more!
In the first talk, Nick Gaski will give an overview of the platform, and subsequent episodes will highlight all the key functionality of the platform—from tokens to blockchain connectors.
Hyperledger Firefly Distributed Ledger Event Management
https://hyperledger.github.io/firefly/
https://hyperledger.github.io/firefly/reference/events.html
Hyperledger FireFly Event Bus
The FireFly event bus provides your application with a single stream of events from all of the back-end services that plug into FireFly.
Applications subscribe to these events using developer friendly protocols like WebSockets, and Webhooks. Additional transports and messaging systems like NATS, Kafka, and JMS Servers can be connected through plugins.
Each application creates one or more Subscriptions to identify itself. In this subscription the application can choose to receive all events that are emitted within a namespace, or can use server-side filtering to only receive a sub-set of events.
The event bus reliably keeps track of which events have been delivered to which applications, via an offset into the main event stream that is updated each time an application acknowledges receipt of events over its subscription.
Your back-end application should be structured for this event-driven paradigm, with an Event Handler constantly listening for events, applying a consistent State Machine to those events and applying the changes to your Application Database.
FireFly comes with a built in event processor for Token transfers & approvals, that implements this pattern to maintain balances, and transaction history in a rich query off-chain data cache.
Decentralized Event Processing
In a decentralized system, you need to consider that each organization runs its own applications, and has its own private database.
At any given point in time different organizations will have slightly different views of what the most up to date information is - even for the blockchain state.
As well as the agreed business logic, there will be private data and core system integration that are needed to process events as they happen. Some of this data might be received privately from other parties, over a secure communications channel (not the blockchain).
The system must be eventually consistent across all parties for any business data/decision that those parties need to agree on. This happens by all parties processing the same events in the same order, and by applying the same business logic (for the parts of the business logic that are agreed).
This means that when processing an event, a participant must have access to enough historical data/state to reach the same conclusion as everyone else.
Let’s look at a couple of examples.
Example 1: A fungible token balance transfer
You need to be able to verify the complete lineage of the tokens being spent, in order to know that they cannot be double spent anywhere in the network.
This means the transaction must be backed by a blockchain verifiable by all participants on the network that could hold balances of that token.
You might be able to use advanced cryptography (such as zero-knowledge proofs) to mask the participants in the trade, but the transaction themselves must be verifiable to everyone in a global sequence that prevents double spending.
Example 2: A step in a multi-party business process
Here it is likely you want to restrict visibility of the data to just the parties directly involved in the business process.
To come to a common agreement on outcome, the parties must know they are processing the same data in the same order. So at minimum a proof (a hash of the data) needs to “pinned” to a blockchain ledger visible to all participants involved in the process.
You can then choose to put more processing on the blockchain, to enforce some critical rules in the business state machine that must be executed fairly to prevent one party from cheating the system. Such as that the highest bid is chosen in a competitive bidding process, or a minimum set of parties have voted agreement before a transaction is finalized.
Other steps in the process might include human decision making, private data from the core systems of one member, or proprietary business logic that one member is not willing to share. These steps are “non-deterministic” - you cannot predict the outcome, nor be guaranteed to reproduce the same outcome with the same inputs in the future.
The FireFly event bus is designed to make triggering these non-deterministic steps easy, while still allowing them to be part of the overall state machine of the business process. You need to take care that the system is designed so parties cannot cheat, and must follow the rules. How much of that rule enforcement needs to be executed on-chain vs. off-chain (backed by a deterministic order through the blockchain) is different for each use case.
Remember that tokens provide a great set of building blocks for on-chain steps in your decentralized applications. Enterprise NFTs allow generation of a globally unique ID, and track ownership. Fungible tokens allow value transfer, and can be extended with smart contracts that to lock/unlock funds in “digital escrow” while complex off-chain agreement happens.
Privacy groups and late join
If a new participant needs to join into a business transaction that has already started, they must first “catch up” with the current state before they can play their part. In a real-world scenario they might not be allowed to see all the data that’s visible to the other parties, so it is common to create a new stream of communications that includes all of the existing parties, plus the new party, to continue the process.
If you use the same blockchain to back both groups, then you can safely order business process steps that involve different parties across these overlapping groups of participants.
Using a single Ethereum permissioned side-chain for example.
Alternatively, you can create dedicated distributed ledgers (DLTs) for communication between these groups of participants. This can allow more logic and data to go on-chain directly, although you still must consider the fact that this data is immutable and can never be deleted.
Using Hyperledger Fabric channels for example.
On top of either type of ledger, FireFly provides a private Group construct to facilitate secure off-chain data exchanges, and to efficiently pin these communications to the blockchain in batches.
These private data exchanges can also be coordinated with most sophisticated on-chain transactions, such as token transfers.
Event Types
FireFly provides a number of different types of events to your application, designed to allow you to build your application state machine quickly and reliably.
All events in FireFly share a common base structure, regardless of their type. They are then linked (via a reference) to an object that contains detailed information.
The categories of event your application can receive are as follows:
See the Core Resources/Event page for a full list of event types, and more details on the data you can expect for each type.
Blockchain events
FireFly allows your application to subscribe to any event from a blockchain smart contract.
In order for applications connected to the FireFly API to receive blockchain events from a smart contracts, a ContractListener fist must be created to instruct FireFly to listen to those events from the blockchain (via the blockchain plugin).
Once you have configured the blockchain event listener, every event detected from the blockchain will result in a FireFly event delivered to your application of type blockchain_event_received.
Check out the Custom Contracts Tutorial for a walk-through of how to set up listeners for the events from your smart contracts.
FireFly automatically establishes listeners for some blockchain events:
Events from the FireFly BatchPin contract that is used to pin identities, off-chain data broadcast and private messaging to the blockchain.
Events from Token contracts, for which a Token Pool has been configured. These events are detected indirectly via the token connector.
Token events
FireFly provides a Wallet API, that is pluggable to multiple token implementations without needing to change your app.
The pluggable API/Event interface allows all kinds of technical implementations of tokens to be fitted into a common framework.
The following wallet operations are supported. These are universal to all token implementations - NFTs and fungible tokens alike:
Mint
Burn
Transfer
Approve
FireFly processes, indexes and stores the events associated with these actions, for any Token Pool that has been configured on the FireFly node.
See Token Transfer and Token Approval for more information on the individual operations.
The token connector is responsible for mapping from the raw Blockchain Events, to the FireFly model for tokens. Reference token connector implementations are provided for common interface standards implemented by tokens - like ERC-20, ERC-721 and ERC-115.
A particular token contract might have many additional features that are unique to that contract, particularly around governance. For these you would use the Smart Contract features of FireFly to interact with the blockchain API and Events directly.
Message events: on-chain / off-chain coordinated
Event aggregation between data arriving off-chain, and the associated ordered proof/transaction events being confirmed on-chain, is a complex orchestration task.
The universal order and additional transaction logic on-chain must be the source of truth for when and how an event is processed.
However, that event cannot be processed until the off-chain private/broadcast data associated with that event is also available and verified against the on-chain hash of that additional data.
They might arrive in any order, and no further events can be processed on that business transaction until the data is available.
Multiple parties might be emitting events as part of the business transaction, and the outcome will only be assured to be the same by all parties if they process these events in the same order.
Hyperledger FireFly handles this for you. Events related to a message are not emitted until both the on-chain and off-chain parts (including large binary attachments) are available+verified in your local FireFly node, and all previous messages on the same topic have been processed successfully by your application.
Your application just needs to:
Choose a suitable topic for your messages that determines the ordered stream it is part of. Such as a business transaction identifier.
Make sure the application does not acknowledge a message, until it has finished processing it.
See Message for more information
Transaction submission events
These events are emitted each time a new transaction is initiated via the Firefly API.
These events are only emitted on the local FireFly node that initiates an activity.
For more information about FireFly Transactions, and how they relate to blockchain transactions, see Transaction.
Potential Value Opportunities
Potential Challenges
Candidate Solutions
Hyperledger workshops list
https://www.hyperledger.org/use/tutorials
Hyperledger webinars
https://www.hyperledger.org/learn/webinars
Github firefly
https://github.com/hyperledger/firefly
Firefly docs
https://hyperledger.github.io/firefly/
Connecting to remote chains
https://hyperledger.github.io/firefly/tutorials/chains/
Firefly Tutorials
Article on Firefly Getting Started - Nicko Guyer
https://github.com/hyperledger/firefly
The best place to learn about FireFly is in the documentation.
There you will find our Getting Started Guide, which will get you a running FireFly network of Supernodes on your local machine in a few minutes.
Your development environment will come with:
| FireFly CLI | FireFly Explorer UI | FireFly Sandbox |
|---|---|---|
 |  |  |
Demo: Making Fabric Easy To Use: FireFly FabConnect - Jim Zhang - 20 mins
https://www.youtube.com/watch?v=sNqc0hOV5xo
FireFly FabConnect is a standalone microservice that provides a RESTful gateway to cover all the needs of a client application to interact with a Fabric network, including registering, enrolling, modifying and re-enrolling identities against a Fabric CA backend, submitting transactions, sending queries, requesting network information such as channel configuration, chain info, etc. It also provides an event stream that can reliably deliver events via websocket connections or webhook endpoints.
Sessions Details & Slides (if available): https://sched.co/15Am9
Access All Sessions: https://events.linuxfoundation.org/hy...
Demo: How to Create, Transfer and Track Tokens with Hyperledger Fabric - 20 mins
https://www.youtube.com/watch?v=PYBApoR277I
You’ll learn how to use the FireFly Sandbox to test and iterate on tokens, and how to track token activity using the FireFly Explorer. Sessions Details & Slides (if available): https://sched.co/1AFpa Access All Sessions: https://events.linuxfoundation.org/hy... ---- Hyperledger – Open Source Blockchain Technologies Hyperledger is an open source community focused on developing a suite of stable frameworks, tools and libraries for enterprise-grade blockchain deployments. It serves as a neutral home for various distributed ledger frameworks including Hyperledger Besu, Fabric, Sawtooth, Indy, as well as tools like Hyperledger Avalon, Cactus and libraries like Hyperledger Aries, Ursa. Learn more about Hyperledger projects: https://www.hyperledger.org/use Discord: 
/ discord Case Studies: https://www.hyperledger.org/learn/cas... Training & Certification: https://www.hyperledger.org/learn/tra... Tutorials: https://www.hyperledger.org/use/tutor... Webinars: https://www.hyperledger.org/learn/web... Events: https://www.hyperledger.org/events Vendor Directory: https://www.hyperledger.org/use/vendo...
Connecting to Public and Private Chains with Hyperledger FireFly - Lab
https://www.youtube.com/watch?v=PMohgbJ8G2M
You’ll learn how Hyperledger FireFly makes it easy to: * Connect to popular public chains including Ethereum, Polygon, Avalanche, Optimism, Arbitrum, and more. * Handle the complex concerns of public chains, including gas estimation and resubmission policies. * Access both public and private chains simultaneously, while building in guardrails specific to the enterprise space that ensure new apps meet both user and compliance requirements. With live coding and easy-to-follow steps, you’ll have everything you need to build and run your own blockchain-powered app. This session is targeted to developers of intermediate skill level and above, with some prior amount of blockchain knowledge. New developers or those without any prior blockchain knowledge will still find it informative, as Hyperledger FireFly is very approachable. At the same time, it is also flexible and customizable to meet the needs of experienced blockchain developers
Workshop: Using Hyperledger FireFly to Launch an NFT Collection on Public Chains - Lab - 2 hrs
https://www.youtube.com/watch?v=OMI-xuysiwI
We’ll provide an overview of Hyperledger FireFly features including: * Web3 Gateway Mode - for connecting to popular public chains like Ethereum, Polygon, Avalanche, Optimism, Arbitrum, etc. * Public Blockchain Tools - with support for managing and signing transactions * Enterprise-friendly functionality - to ensure new apps meet both user and compliance requirements. * Integration capabilities - for connecting your NFT app to existing enterprise systems. From there, we’ll take you through the steps to create and launch an NFT collection on Polygon using Hyperledger FireFly. With live coding and easy-to-follow steps, you’ll have everything you need to launch your own NFT collection.
Firefly NFT w Fabric notes from discord
Step-by-step guide for Example
sample code block