Cloud Native PNDA development guide on AWS EKS:
This document describes how you can setup Cloud Native PNDA on AWS
Prerequisites:
AWS CLI Installation and Setup:
Use following link for AWS CLI setup
https://docs.aws.amazon.com/cli/latest/userguide/install-linux.html
Once setup is done then configure aws cli as per below,
$ aws configure
AWS Access Key ID [None]: Enter your access key ID
AWS Secret Access Key [None]: Enter your secret access key
Default region name [None]: Enter your region
Default output format [None]: json
Docker Installation:
Follow the steps as mentioned in an official documentation to install Docker,
https://docs.docker.com/install/linux/docker-ce/ubuntu/#set-up-the-repository
Kubernetes(kubectl) Installation:
In your terminal run the following commands:
EKSCTL Installation:
Run the following commands in your terminal
Helm client 2.14 Installation:
$curl -L https://git.io/get_helm.sh | bash -s -- --version v2.14.3
$helm version
EKS CLUSTER CREATION:
Step 1) Create EKS cluster on AWS:
You can create EKS cluster on AWS by using either CLI or AWS GUI
Creating EKS cluster through CLI by step a or b:
sudo eksctl create cluster --name pnda \
--region us-east-2 \
--version 1.14 \
--nodegroup-name pnda-nodes \
--node-type t3.xlarge \
--nodes 5 \
--nodes-min 2 \
--nodes-max 6 \
–managed
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
metadata:
name: pnda
region: es-east-1
nodeGroups:
- name: ng-pnda-1
instanceType: t3.large
desiredCapacity: 10
volumeSize: 80
ssh:
allow: true # will use ~/.ssh/id_rsa.pub as the default ssh key
- name: ng-pnda-2
instanceType: t3.xlarge
desiredCapacity: 2
volumeSize: 100
ssh:
publicKeyPath: ~/.ssh/ec2_id_rsa.pub
$ eksctl create cluster -f pnda_eks_cluster.yaml
Note: If you needed to use an existing VPC, you can use config file with VPC details,
vpc: subnets: private: eu-north-1a: { id: subnet-0ff156e0c4a6d300c } eu-north-1b: { id: subnet-0549cdab573695c03 } eu-north-1c: { id: subnet-0426fb4a607393184 }
Creating EKS cluster thru GUI:
Follow the steps as mentioned in following link,
https://docs.bitnami.com/aws/get-started-eks/
Note: Creating EKS cluster thru GUI is time taking and harder as compared to using CLI.
Step 2) Setup alb Load balancer for created cluster.
(https://docs.aws.amazon.com/eks/latest/userguide/alb-ingress.html)
Run the following commands to setup load balncer,
IAM Console --> Roles --> search for the NodeInstanceRole.
(Example: eksctl-pnda-eks-NodeInstanceRole-xxxxxx.) --> Attach policy select ingressController-iam-policy.
kubectl edit deployment.apps/alb-ingress-controller -n kube-system
spec:
containers:
- args:
- --ingress-class=alb
--cluster-name=pnda
--aws-vpc-id=vpc-05b0819933d5440cb
--aws-region=us-east-2
Step 3) Helm tiller Installation on EKS cluster:
$helm init
Create rbac-config.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: tiller
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: tiller
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: tiller
namespace: kube-system
$kubectl create -f rbac-config.yaml
$helm init --service-account tiller --upgrade
$helm version
Note: Default EBS General Purpose SSD(gp2) volume types only supports read-write-once persistent volume (PV) access-mode. The PV which we create for Deployment Manager requires read-write-many access-mode, as multiple pods (Deployment Manager, Spark Operatoretc..) share same PV. Hence, we have to create/setup EFS CSI driver which supports read-write-many access-mode for a PV.
Step 4) To deploy Amazon EFS CSI driver to an Amazon EKS cluster
(Above command will return VPC ID and use the same in below command)
(Above command returns VPC CODR range and use the same while adding rule for NFS inbound traffic)
Type: NFS
Source: Custom. Paste the VPC CIDR range.
Description: Add a description, such as "Allows inbound NFS traffic from within the VPC."
Step 5) Create AWS EFS
(https://us-east-2.console.aws.amazon.com/efs/home?region=us-east-2#/get-started)
Note: Use same region as EKS cluster created region in above URL
There are 2 ways to create AWS elastic file system
Note: If you don't see your VPC, at the top right of the console, make sure that the region that your VPC is in is selected.
(Or)
Follow the steps from following link,
https://docs.aws.amazon.com/efs/latest/ug/wt1-create-efs-resources.html
Step 6) Setup K8S-EFS provisioner
git clone https://github.com/kubernetes-incubator/external-storage
cd external-storage/aws/efs/deploy/
Kubectl apply -f rbac.yaml
See following attachment for manifest.yaml,
kubectl apply -f manifest.yaml
kubectl get pv, pvc à should return efs volume with aws-efs storage class
Step 7) Setup Console/Kafka/Grafana/etc.. service types as load balancer for Helm charts
Console: ~/pnda/pnda-helm-chart/cloud-pnda/values.yaml
Kafka: ~/pnda/pnda-helm-chart/cloud-pnda/charts/kafka-manager
Grafana: ~/pnda/pnda-helm-chart/cloud-pnda/charts/grafana/values.yaml
Step 8) Add storage class as gp2 for all other pvc and aws-efs for deployment manager pvc.
storageClass: aws-efs
storageClass: gp2
Step 9) Install PNDA on eks with helm
(helm install --name pnda --namespace pnda cloud-pnda)
Step 10) Setup DNS alias for external IP’s in Route 53 to access the DNS like console.pnda.io within private network
Step 11) Access the console front end from any one of the ec2 instance.
PNDA UI: http://console.pnda.io/
Step 12) Testing the deployment of SparkStreaming
kubectl -n pnda port-forward service/pnda-package-repository 8888
curl -XPUT "http://localhost:8888/packages/literary-word-count-app- wf-0.3.2.tar.gz?user.name=" --data-binary "@literary-word-count-app- wf-0.3.2.tar.gz"
Sample Spark Streaming example is in following Git repo,
https://github.com/sreenivasa-xor/SparkStreaming-Example-Application.git
Step 13) Clean-up
Delete PNDA on eks cluster
% helm del –purge pnda
Delete EKS Cluster:
% eksctl delete cluster --region=us-east-2 --name=pnda