Deploying µONOS micro-services with HELM
One of the goals of the µONOS project is to provide simple deployment options that integrate with modern technologies. Deployment configurations can be found in the
onos-helm-charts repository. Each
onos service has a directory containing its chart. As an example the
onos-config chart is in
Deploying on Kubernetes with Helm
Helm is a package manager for Kubernetes that allows projects to provide a collection of templates for all the resources needed to deploy on k8s. ONOS Config provides a Helm chart for deploying a cluster for development and testing. In the future, this chart will be extended for production use.
Individual charts or overarching (umbrella) chart
The individual components of µONOS may be deployed one at a time, or altogether through an overarching (unbrella) Helm chart, or some combination of both.
In all cases the prerequisites must be satisfied:
- Creation of a namespace
- deployment of Atomix controller(s) in the namespace.
The individual components in the umbrella chart are:
The choice of which of these is deployed can be chosen at deployment time with an option like:
In this way all, none or some of the components can be deployed together.
The Helm chart provides resources for deploying the config service and accessing it over the network, both inside and outside the k8s cluster:
Deployment- Provides a template for ONOS Config pods
ConfigMap- Provides test configurations for the application
Service- Exposes ONOS Config to other applications on the network
Secret- Provides TLS certificates for end-to-end encryption
Ingress- Optionally provides support for external load balancing
Local Deployment Setup with Kind
To deploy the Helm chart locally:
First, you will need to install Docker to build and deploy an image locally.
Second, install Kind.
Kind v0.9.0 at least is required, which provides the K8S API v1.17
brew install helm
For more information, please refer to Installing Helm page.
- Once Kind has been installed, start it with
kind create cluster
- Once Kind has started, export the configuration:
This needs to be refreshed if you delete and recreate the Kind cluster
kind get kubeconfig > ~/.kube/kind
- Once Kind has started, export the new environment to access the Kubernetes cluster:
This needs to be run in each terminal window that will access the cluster
Bare metal deployment
ONOS can also be deployed on a bare metal cluster provisioned with Rancher or equivalent.
For any deployment scenario a number of steps must be performed first.
The steps below assume the
KUBECONFIG environment variable to point
kubectl to your cluster.
Add the "CORD" Helm chart repo
The Prometheus and Grafana installations are derived from the CORD Helm charts. Run:
helm repo add cord https://charts.opencord.org
Add the "Atomix" Helm chart repo
helm repo add atomix https://charts.atomix.io
Add the "onosproject" Helm chart repo
helm repo add onosproject https://charts.onosproject.org
Update the local cache with charts from these repos:
helm repo update
Inspect the chart versions and app versions
To see the list of the latest chart versions and app versions, use the "search" command.
helm search repo onos
Configure the micro-onos namespace
onos services can be deployed to any
A "namespace" partitions the cluster in to independent islands. For consistency between documentation we use
micro-onosas the namespace here.
To create the
micro-onos namespace run:
kubectl create namespace micro-onos
Deploy Atomix Controller
onos services leverage Atomix as the distributed store for HA, scale and redundancy. The first thing that needs to be deployed in any
onos deployment is the Atomix go controller. To ensure the controllers are deployed in the correct place with the proper configuration, you can use the deployment manifests rather than the Helm charts:
kubectl create -f https://raw.githubusercontent.com/atomix/kubernetes-controller/master/deploy/atomix-controller.yaml kubectl create -f https://raw.githubusercontent.com/atomix/raft-storage-controller/master/deploy/raft-storage-controller.yaml kubectl create -f https://raw.githubusercontent.com/atomix/cache-storage-controller/master/deploy/cache-storage-controller.yaml
Deploy ONOS Operator
onos-operator ensures that ONOS Custom Resource Defintions (CRD) for
onos-config are deployed in to the cluster.
To ensure the controllers are deployed in the correct place with the proper configuration, you can use the deployment manifests rather than the Helm charts:
kubectl create -f https://raw.githubusercontent.com/onosproject/onos-operator/v0.4.1/deploy/onos-operator.yaml
Deploy the µONOS services
A complete set of µONOS services can be deployed with just the over-arching
Run the install:
helm -n micro-onos install onos-umbrella onosproject/onos-umbrella
this will deploy
onos-config (but not
onos-classic as it's not needed for µONOS - see Deploying ONOS classic with HELM).
To monitor the startup of the pods use
kubectl -n micro-onos get pods -w
giving a list like:
NAME READY STATUS RESTARTS AGE onos-cli-7b7868c7b8-t2n4r 1/1 Running 0 89s onos-config-84fd7476c7-44llj 2/2 Running 0 89s onos-consensus-db-1-0 1/1 Running 0 89s onos-gui-694bc898b7-m5xls 2/2 Running 0 89s onos-topo-c6975b7bf-x8nb7 1/1 Running 0 89s
Additionally the Controllers for Atomix and Onos-Operator can be seen in the
NAME READY STATUS RESTARTS AGE atomix-controller-d69c9d45c-fx9b7 1/1 Running 42 79d cache-storage-controller-65977dccc8-gdcb2 1/1 Running 42 79d raft-storage-controller-79649fc8c7-qcj44 1/1 Running 42 79d config-operator-56dc64df8d-h8cpb 1/1 Running 0 3h16m topo-operator-6f555cb86-54pdz 1/1 Running 0 3h16m
To see the list of installed charts:
helm -n micro-onos ls
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION onos-umbrella micro-onos 1 2020-08-12 16:12:40.010583704 +0100 IST deployed onos-umbrella-0.0.11 v0.6.4
To delete the deployment issue:
helm delete -n micro-onos onos-umbrella
Deploy single services services
Alternatively can deploy each service by itself. Please refer to each service's
deployment file to get the exact command for each helm chart. Example for onos-topo.
helm -n micro-onos install onos-topo onosproject/onos-topo
For individual services it is necessary to install Atomix first, as above:
Developers may want to run and deploy charts that have not yet been released. This must be done from the checked out charts folder
To use the latest version of an application without having to update the chart, an override like
--set image.tag=latestcan be used when individual charts. Alternatively when deploying the umbrella chart the override like
--set onos-topo.image.tag=latest. The individual applications can be updated in to
kindwith the command
Note that the source of the charts like
onosproject/onos-topowill use the chart from the helm repository (cached locally), where as a source like
./onos-topowill load the chart from the local folder. This is useful when editing charts.
Check out the Helm charts
The helm charts need to be present on your PC. Run:
git clone https://github.com/onosproject/onos-helm-charts && cd onos-helm-charts
Over-arching (umbrella) chart
Run the build of dependent charts to use the local
onos-umbrella over-arching chart:
make clean && make deps
If you make changes to one of the charts and want to re-deploy, please first issue:
helm dependency update onos-umbrella
Individual local charts
To deploy charts individually (from the
onos-helm-charts directory) for example:
helm -n micro-onos install onos-topo ./onos-topo