Page History

...

 Step 6) Setup K8S-EFS provisioner

1. Download k8s-efs repository from git,
   1. git clone https://github.com/kubernetes-incubator/external-storage
2. Switch to the deploy directory
   1. cd external-storage/aws/efs/deploy/
3. Apply rbac permissions
   1. Kubectl apply -f rbac.yaml
4. Modify manifest.yaml. In the configmap section change the system.id: and aws.region: to match the details of the EFS you created. Change dns.name if you want to mount by your own DNS name and not by AWS's *file-system-id*.efs.*aws-region*.amazonaws.com.

...

1. See following attachment for manifest.yaml,

5. Apply the manifest

kubectl apply -f manifest.yaml

6. Check PV and PVC created properly

kubectl get pv,

...

pvc  (should return efs volume with aws-efs storage class)

Step 7) Setup Console/Kafka/Grafana/etc.. service types as load balancer for Helm charts

...

Console: ~/pnda/pnda-helm-chart/cloud-pnda/values.yaml

...

          Image Added

                Kafka: ~/pnda/pnda-helm-chart/cloud-pnda/charts/kafka-manager                                

                Image Added                              

               Grafana: ~/pnda/pnda-helm-chart/cloud-pnda/charts/grafana/values.yaml                            

   Image Added

Step 8) Add storage class as gp2 for all other pvc and aws-efs for deployment manager pvc.

...

Step 9) Install PNDA on eks with helm

Image Added

(helm install --name pnda --namespace pnda cloud-pnda)

Step 10) Setup DNS alias for external IP’s in Route 53 to access the DNS like console.pnda.io within private network 

Step 11) Access the console front end from any one of the ec2 instance.

...

Step 12) Testing the deployment of SparkStreaming

...

1. To test the deployment of SparkStreaming we have created an example app literary-word-count-app-wf-0.3.2.tar.gz.

...

1.  To upload the package to the cloud-pnda platform: 
   1. kubectl -n pnda port-forward service/pnda-package-repository 8888
2. It is possible to temporarily expose the package repository API with kubectl port forwarding:
   1. curl -XPUT "http://localhost:8888/packages/literary-word-count-app-     wf-0.3.2.tar.gz?user.name=" --data-binary "@literary-word-count-app- wf-0.3.2.tar.gz"
3. You can see the uploaded package details under available packages in Package Management and then deploy the package by clicking Deploy button.

            Image Added

5. After successfully deploying the package, create/install the application in the Apps Tab

           Image Added

6. Verify the application on K8S cluster(Tested on Rancher cluster).

           Image Added

7. To delete the application, you have to click on delete button of particular application under Apps Tab,

     Image Added

Sample Spark Streaming example is in following Git repo,

https://github.com/sreenivasa-xor/SparkStreaming-Example-Application.git

Step 13) Clean-up

 Delete PNDA on eks cluster

% helm del –purge pnda

Delete EKS Cluster:

% eksctl delete cluster --region=us-east-2 --name=pnda