PNDA forum discussion

Motivation

Proposal

Design

The following section discusses the changes required to each PNDA component.

PNDA Mirror

Druid resources and any other dependencies will be hosted on the PNDA mirror. The mirror build script will need to include these in the appropriate mirror section.

Druid Components in PNDA

For Druid cluster, will launch new nodes for Druid borker, Historicals and MiddleManagers,  Coordinator and Overlord processes and use the existing nodes of PNDA cluster for kafka, zookeeper, and mysql for druid metadata storage.


Support will be added for deploying and configuring Druid components in heat templates and  salt configuration files respectively.

Deployment Manager:

A Druid component plugin will be created that will run druid applications. A supervisor will be set up on the PNDA edge node that will call the druid CLI to process the durid query operation.

Console:

The PNDA console dashboard page will be modified to include add Druid blocks under data storage.

Logging

Each druid component will have a specific log file for debugging purpose.

Example applications

The community druid example applications will be created that demonstrates use of druid.

PNDA Guide

Sections of guide will need creating or updating to reference Druid

Plan

Phase 1 - Integrate of Druid with single node deployment using Openstack Pico flavor.

 (Refer http://druid.io/docs/0.12.1/tutorials/quickstart.html for Druid single node deployment.)

Phase 2 - Integration of Druid cluster in Openstack Standard flavor & AWS Standard flavor

Phase 3 - Druid stand alone (lambda integration) vs server cluster deployment

Notes:

Interfaces

Compatibility

Alternatives