Page tree
Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 12 Next »

Rancher based Kubernetes Cluster Set-up on OpenStack VMs with NFS Storage class


This document describes how you can achieve PNDA Helm Deployment on Rancher based Kubernetes cluster over OpenStack based (Kilo) VMs.

There are two parts to this document,

  1. Rancher based K8S cluster set-up on OpenStack (along with NFS as Storage class)
  2. Helm-based PNDA deployment on K8S cluster

Rancher based K8S cluster set-up on OpenStack

In the first part, we need to set-up a Rancher based Kubernetes cluster with the necessary software components. Following is the high-level depiction of Rancher Kubernetes cluster components using OpenStack VMs,

Cluster Components and Versions

All Rancher Kubernetes cluster components are deployed across 6 OpenStack VMs. Resource configuration for each OpenStack VM is shown in the diagram.

Following are the versions of various components that can be used,


Docker Installation

Install Docker on all the Rancher Kubernetes Engine nodes including Load Balancer node. Execute the following commands on all the nodes. Following commands assume “CentoOS” as underlying OS, but you can also use Ubuntu.

$ yum install -y yum-utils device-mapper-persistent-data lvm2

$ yum-config-manager  --add-repo

$yum install docker-ce docker-ce-cli

$ systemctl start docker

$ systemctl enable docker

$ usermod -aG docker centos

Following are the cluster components that need to be installed on the respective node or VMs as depicted in the above diagram,

  • NGINX Layer 4 (TCP) Load Balancer
  • Rancher Kubernetes Engine Nodes
  • Helm and Tiller
  • Rancher Server
  • Network File System (NFS)

NGINX Layer 4 (TCP) Load Balancer

NGINX is configured as Layer 4 load balancer (TCP) that forwards connections to one of the Rancher nodes. The important point here is not to use one of the Rancher nodes as the load balancer. Following steps are executed on the Nginx node or VM.

Step 1: Create a repo file as below - 

Step 2: List the repo with the below command to check for any errors - 

Step 3: Install Nginx package with the below command - 

Step 4: Update the configuration file “/etc/nginx/nginx.conf” with the IP addresses of your Rancher Kubernetes nodes as shown below - 

Please refer to link for any additional details - Nginx Load Balancer Set-up

Rancher Kubernetes Engine Nodes

Next activity is to install Rancher Kubernetes Engine on all the designated Kubernetes nodes. Following steps can be executed from any node or VM (Installer node) that has access to all the Rancher Kubernetes Engine Nodes.

Step 1:

Create the rancher-cluster.yml file - 

Using the below sample file, create the rancher-cluster.yml file. Replace the IP Addresses in the nodes list with the IP address or DNS names of the nodes you created.

Step 2: Download and install ‘rke’ utility - 

Download the utility as shown in the below diagram based on your node hardware type.

To install ‘rke’ using the following command –

$ install rke_linux-amd64 /usr/bin/rke

Step 3: Run ‘rke’ utility on the Rancher Cluster configuration.

$ rke up --config ./rancher-cluster.yml

When finished, it should end with the line: Finished building Kubernetes cluster successfully.

Step 4: Install Kubectl utility – 

For managing your Kubernetes cluster you need to use Kubectl utility.

$ curl -LO`curl -s`/bin/linux/amd64/kubectl

$ install kubectl /usr/bin/kubectl

 Step 5: Testing Your Cluster

rke’ command execution in Step 3 above should have created a file kube_config_rancher-cluster.yml. This file has the credentials for kubectl and helm that you need to use. Update the .bash_profile file with as below -

 $ export KUBECONFIG=$(pwd)/kube_config_rancher-cluster.yml

$ source .bash_profile

 Test your connectivity with kubectl and see if all your nodes are in Ready state as shown below

Please refer to the link for any additional details such as connecting to and testing your Kubernetes cluster and for checking the health of cluster Pods - Rancher Kubernetes Engine Installation

Helm and Tiller 

Following steps need to be executed from any node or VM (Installer node) that has access to all Rancher Kubernetes Engine nodes.

Step 1: Install Helm and Tiller on the installer node

$ wget

$ tar zxvf helm-v2.14.3-linux-amd64.tar.gz

$ install linux-amd64/helm /usr/bin/helm

$ install linux-amd64/tiller /usr/bin/tiller

Step 2: Copy or Install Kubectl, Helm and Tiller binaries on all the Rancher Kubernetes Nodes to the same location as described in the above steps.

Step 3: Finishing Tiller installation on Cluster from the Installer node –

  • Create the ServiceAccount in the kube-system namespace.
  • Create the ClusterRoleBinding to give the tiller account access to the cluster.
  • Finally, use helm to install the tiller service

$ kubectl -n kube-system create serviceaccount tiller

$ kubectl create clusterrolebinding tiller   --clusterrole=cluster-admin   --serviceaccount=kube-system:tiller

$ helm init --service-account tiller

 Test your Tiller installation –

Important Files

  • rancher-cluster.yml: The RKE cluster configuration file.
  • kube_config_rancher-cluster.yml: The Kubeconfig file for the cluster, this file contains credentials for full access to the cluster.
  • rancher-cluster.rkestate: The Kubernetes Cluster State file, this file contains credentials for full access to the cluster.

Rancher Server

In this activity, Rancher server and UI gets installed as one of the Pods inside Kubernetes cluster.

Step 1: Add the Helm Chart Repository -

$ helm repo add rancher-stable

 Step 2: Install cert-manager from Kubernetes Helm chart repository -

$ helm install stable/cert-manager --name cert-manager --namespace kube-system --version v0.5.2

 Step 3: Wait for cert-manager to be rolled out -

$ kubectl -n kube-system rollout status deploy/cert-manager

 Step 4: Install Rancher Server – Stable version

$ helm install rancher-stable/rancher  --name rancher --namespace cattle-system --set --set ingress.tls.source=letsEncrypt --set

This should print a message – Rancher Server has been installed. This will be followed by the Rancher Admin UI URL that you can browse.

Important Note: The hostname specified in the above command needs to be registered with your organization’s DNS.

Step 5: Rancher Rollout

$ kubectl –n cattle-system rollout status deploy/rancher

This will roll out the Rancher deployment.

Step 6: Check Ingress

$ kubectl -n cattle-system get ingress

Step 7: Check Rancher Deployment Status

$ kubectl -n cattle-system get deploy rancher

Network File System (NFS) Server

Following are general steps to get the NFS server and share created -

Step 1: Set-up a NFS Server on one of the nodes or VMs.

Step 2: Create and export a NFS Share

Step 3: Mount the exported NFS share on all the Rancher Kubernetes Nodes

Kubernetes NFS-Client Provisioner

The nfs-client is an automatic provisioner that use your existing and already configured NFS server to support dynamic provisioning of Kubernetes Persistent Volumes via Persistent Volume Claims. You can use Helm Chart to deploy the nfs-client.

$ helm install stable/nfs-client-provisioner --set nfs.server=x.x.x.x --set nfs.path=/nfsshare

For additional details source code, please refer to -

For additional details please on Helm refer to -

Helm-based PNDA deployment on K8S cluster


  • No labels