Feature Maturity
Definitions and processes around how features mature or are deprecated
Key Points
References
| Reference_description_with_linked_URLs_________________________ | Notes___________________________________________________________________ |
|---|---|
| Build tools | |
| Jenkins | |
| https://jenkins-x.io/about/ | Jenkins-x |
| https://circleci.com/dashboard | Circle CI dashboard |
| Azure Pipelines | |
Key Concepts
Deployment targets
Server
Cloud
Container
Kubernetes
Fabric
CICD tools compared
Jenkins with Blue Ocean - visual pipeline builder
https://jenkins.io/doc/book/blueocean/
Blue Ocean rethinks the user experience of Jenkins. Designed from the ground up for Jenkins Pipeline, but still compatible with freestyle jobs, Blue Ocean reduces clutter and increases clarity for every member of the team. Blue Ocean’s main features include:
Sophisticated visualizations of continuous delivery (CD) Pipelines, allowing for fast and intuitive comprehension of your Pipeline’s status.
Pipeline editor - makes creation of Pipelines approachable by guiding the user through an intuitive and visual process to create a Pipeline.
Personalization to suit the role-based needs of each member of the team.
Pinpoint precision when intervention is needed and/or issues arise. Blue Ocean shows where in the pipeline attention is needed, facilitating exception handling and increasing productivity.
Native integration for branch and pull requests, enables maximum developer productivity when collaborating on code with others in GitHub and Bitbucket.
This chapter covers all aspects of Blue Ocean’s functionality, including how to:
get started with Blue Ocean - covers how to set up Blue Ocean in Jenkins and access the Blue Ocean interface,
create a new Pipeline project,
use Blue Ocean’s Dashboard,
use the Activity view - where you can access lists of your current and previously completed Pipeline/item runs, as well as your Pipeline project’s branches and any opened Pull Requests,
use the Pipeline run details view - where you can access details (i.e. the console output) for a particular Pipeline/item run, and
use the Pipeline Editor to modify Pipelines as code, which are committed to source control.
This chapter is intended for Jenkins users of all skill levels, but beginners may need to refer to some sections of the Pipeline chapter to understand some topics covered in this Blue Ocean chapter.
For an overview of content in the Jenkins User Handbook, see User Handbook overview
Blue Ocean video
https://jenkins.io/doc/book/blueocean/getting-started/
Jenkins-X
Concepts
Jenkins X is designed to make it simple for developers to work to DevOps principles and best practices. The approaches taken are based on the comprehensive research done for the book ACCELERATE: Building and Scaling High Performing Technology Organisations. You can read why we use Accelerate for the principals behind Jenkins X.
Principles
“DevOps is a set of practices intended to reduce the time between committing a change to a system and the change being placed into normal production, while ensuring high quality.”
The goals of DevOps projects are:
- Faster time to market
- Improved deployment frequency
- Shorter time between fixes
- Lower failure rate of releases
- Faster Mean Time To Recovery
High performing teams should be able to deploy multiple times per day compared to the industry average that falls between once per week and once per month.
The lead time for code to migrate from ‘code committed’ to ‘code in production’ should be less than one hour and the change failure rate should be less than 15%, compared to an average of between 31-45%.
The Mean Time To Recover from a failure should also be less than one hour.
Jenkins X has been designed from first principles to allow teams to apply DevOps best practices to hit top-of-industry performance goals.
Practices
The following best practices are considered key to operating a successful DevOps approach:
- Loosely-coupled Architectures
- Self-service Configuration
- Automated Provisioning
- Continuous Build / Integration and Delivery
- Automated Release Management
- Incremental Testing
- Infrastructure Configuration as Code
- Comprehensive configuration management
- Trunk based development and feature flags
Jenkins X brings together a number of familiar methodologies and components into an integrated approach that minimises complexity.
Architecture
Jenkins X builds upon the DevOps model of loosely-coupled architectures and is designed to support you in deploying large numbers of distributed microservices in a repeatable and manageable fashion, across multiple teams.
Conceptual model
Building Blocks
Jenkins X builds upon the following core components:
Kubernetes & Docker
At the heart of the system is Kubernetes, which has become the defacto virtual infrastructure platform for DevOps. Every major Cloud provider now offers Kubernetes infrastructure on demand and the platform may also be installed in-house on private infrastructure, if required. Test environments may also be created on local development hardware using the Minikube installer.
Functionally, the Kubernetes platform extends the basic Containerisation principles provided by Docker to span across multiple physical Nodes.
In brief, Kubernetes provides a homogeneous virtual infrastructure that can be scaled dynamically by adding or removing Nodes. Each Node participates in a single large flat private virtual network space.
The unit of deployment in Kubernetes is the Pod, which comprises one or more Docker containers and some meta-data. All containers within a Pod share the same virtual IP address and port space. Deployments within Kubernetes are declarative, so the user specifies the number of instances of a given version of a Pod to be deployed and Kubernetes calculates the actions required to get from the current state to the desired state by deploying or deleting Pods across Nodes. The decision as to where specific instances of Pods will be instantiated is influenced by available resources, desired resources and label-matching. Once deployed, Kubernetes undertakes to ensure that the desired number of Pods of each type remain operational by performing periodic health checks and terminating and replacing non-responsive Pods.
To impose some structure, Kubernetes allows for the creation of virtual Namespaces which can be used to separate Pods logically, and to potentially associate groups of Pods with specific resources. Resources in a Namespace can share a single security policy, for example. Resource names are required to be unique within a Namespace but may be reused across Namespaces.
In the Jenkins X model, a Pod equates to a deployed instance of a Microservice (in most cases). Where horizontal scaling of the Microservice is required, Kubernetes allows multiple identical instances of a given Pod to be deployed, each with its own virtual IP address. These can be aggregated into a single virtual endpoint known as a Service which has a unique and static IP address and a local DNS entry that matches the Service name. Calls to the Service are dynamically remapped to the IP of one of the healthy Pod instances on a random basis. Services can also be used to remap ports. Within the Kubernetes virtual network, services can be referred to with a fully qualified domain name of the form: <service-name>.<namespace-name>.svc.cluster.local which may be shortened to <service-name>.<namespace-name> or just <service-name> in the case of services which fall within the same namespace. Hence, a RESTful service called ‘payments’ deployed in a namespace called ‘finance’ could be referred to in code via http://payments.finance.svc.cluster.local, http://payments.finance or just http://payments, dependent upon the location of the calling code.
To access Services from outside the local network, Kubernetes requires the creation of an Ingress for each Service. The most common form of this utilises one or more load balancers with static IP addresses, which sit outside the Kubernetes virtual infrastructure and route network requests to mapped internal Services. By creating a wildcard external DNS entry for the static IP address of the load balancer, it becomes possible to map services to external fully-qualified domain names. For example, if our load balancer is mapped to *.jenkins-x.io then our payments service could be exposed as http://payments.finance.jenkins-x.io.
Kubernetes represents a powerful and constantly improving platform for deploying services at massive scale, but is also complex to understand and can be difficult to configure correctly. Jenkins X brings to Kubernetes a set of default conventions and some simplified tooling, optimised for the purposes of DevOps and the management of loosely-coupled services.
The jx command line tool provides simple ways to perform common operations upon Kubernetes instances like viewing logs and connecting to container instances. In addition, Jenkins X extends the Kubernetes Namespace convention to create Environments which may be chained together to form a promotion hierarchy for the release pipeline.
A Jenkins X Environment can represent a virtual infrastructure environment such as Dev, Staging, Production etc for a given code team. Promotion rules between Environments can be defined so that releases may be moved automatically or manually through the pipeline. Each Environment is managed following the GitOps methodology - the desired state of an Environment is maintained in a Git repository and committing or rolling back changes to the repository triggers an associated change of state in the given Environment in Kubernetes.
Kubernetes clusters can be created directly using the jx create cluster command, making it simple to reproduce clusters in the event of a failure. Similarly, the Jenkins X platform can be upgraded on an existing cluster using jx upgrade platform. Jenkins X supports working with multiple Kubernetes clusters through jx context and switching between multiple Environments within a cluster with jx environment.
Developers should be aware of the capabilities that Kubernetes provides for distributing configuration data and security credentials across the cluster. ConfigMaps can be used to create sets of name/value pairs for non-confidential configuration meta-data and Secrets perform a similar but encrypted mechanism for security credentials and tokens. Kubernetes also provides a mechanism for specifying Resource Quotas for Pods which is necessary for optimising deployments across Nodes and which we shall discuss shortly.
By default, Pod state is transient. Any data written to the local file system of a Pod is lost when that Pod is deleted. Developers should be aware that Kubernetes may unilaterally decide to delete instances of Pods and recreate them at any time as part of the general load balancing process for Nodes so local data may be lost at any time. Where stateful data is required, Persistent Volumes should be declared and mounted within the file system of specific Pods.
Helm and Draft
Interacting directly with Kubernetes involves either manual configuration using the kubectl command line utility, or passing various flavours of YAML data to the API. This can be complex and is open to human error creeping in. In keeping with the DevOps principle of ‘configuration as code’, Jenkins X leverages Helm and Draft to create atomic blocks of configuration for your applications.
Helm simplifies Kubernetes configuration through the concept of a Chart, which is a set of files that together specify the meta-data necessary to deploy a given application or service into Kubernetes. Rather than maintain a series of boilerplate YAML files based upon the Kubernetes API, Helm uses a templating language to create the required YAML specifications from a single shared set of values. This makes it possible to specify re-usable Kubernetes applications where configuration can be selectively over-ridden at deployment time.
Circle-CI
Azure Pipelines
Potential Value Opportunities
Potential Challenges
Candidate Solutions
OpenTofu
https://opentofu.org/docs/intro/
https://opentofu.org/faq/#why-use-opentofu
https://opentofu.org/docs/intro/vs/
https://opentofu.org/manifesto/
GitOps
GitOps Cheat Sheet
gitops-The Essentials of GitOps.pdf. link
gitops-The Essentials of GitOps.pdf. file
The "Essentials of GitOps" guide from DZone provides a comprehensive overview of GitOps principles and practices. Here are the key points:
1. **Definition**: GitOps uses Git as a single source of truth for declarative infrastructure and applications.
2. **Benefits**: It offers improved deployment consistency, faster delivery, and enhanced security.
3. **Core Principles**: GitOps is based on declarative descriptions and version-controlled infrastructure.
4. **Workflow**: Includes managing configurations in Git, automatic deployment, and continuous monitoring.
5. **Tools**: Common tools include Flux, Argo CD, and Jenkins X.
6. **Security**: Emphasizes secure Git repositories and access controls.
7. **Examples**: Includes YAML configurations for Kubernetes deployments.
8. **Best Practices**: Version control everything, use pull requests, and implement automated testing.
9. **Challenges**: Addresses potential issues like managing secrets and handling stateful applications.
10. **Future Trends**: Highlights the growing importance of policy-driven automation and AI integration.
11. **Case Studies**: Examples of successful GitOps implementations in various organizations.
12. **CI/CD Integration**: Explains how GitOps fits into the broader CI/CD pipeline.
13. **Metrics**: Monitoring and measuring success using specific metrics.
14. **Community**: Encourages participation in GitOps communities for support and collaboration.
15. **Resources**: Provides additional resources for learning and implementation.
For more details, refer to the full guide [here](https://dzone.com/storage/assets/16799903-sponsored-refcard-339-essentialsofgitops-2023.pdf).
Hashicorp Terraform License Change
Terraform was open-sourced in 2014 under the Mozilla Public License (v2.0) (the “MPL”). Over the next ~9 years, it built up a community that included thousands of users, contributors, customers, certified practitioners, vendors, and an ecosystem of open-source modules, plugins, libraries, and extensions.
Then, on August 10th, 2023, with little or no advance notice or chance for much, if not all, of the community to have any input, HashiCorp switched the license for Terraform from the MPL to the Business Source License (v1.1) (the “BUSL”), a non-open source license. In our opinion, this change threatens the entire community and ecosystem that's built up around Terraform over the last 9 years.
Hashicorp Consul
Consul alternatives
https://stackshare.io/consul/alternatives
Circle CI
NGrok - commercial SDN API gateway as a service
ngrok.com-Introducing ngroks Developer-Defined API Gateway Shifting the Paradigm of API Delivery.pdf link
ngrok.com-Introducing ngroks Developer-Defined API Gateway Shifting the Paradigm of API Delivery.pdf file
Step-by-step guide for Example
sample code block