Motivation

PNDA supports creation of different applications using different data-processing platforms/tools like spark, flink, oozie.

Currently, PNDA doesn't have the support to do any other action on the application apart from start/stop. As these data-processing platforms/tools evolves, there would be a need to perform various actions on these applications. For example, in Flink, a user would want an easy way for Triggering, Listing, Resuming & Disposing of Savepoints for a chosen application.

PNDA may have to enhance applications of spark, oozie, etc in future to support different kinds of actions (not necessarily related to Savepoints alone). 

This leads to a generic design of handling actions based on the application type without impacting the user experience of PNDA console UI.

Proposal

Flink Savepoints related actions (Triggering, Listing, Resuming & Disposing) would be implemented with a generic design with actions & the required information is driven through deployment manager API. console-front-end code goes through minimum/no changes for every new action type.

Flink Savepoints is a feature provided by flink for streaming applications to take snapshot of current running state of application and store the metadata in specified directory. Using the metadata user can create a new instance of running application. 

Design

console-frontend would be getting the list of additional actions for an application based on application and each action will be handled by separate API. 

• GET /applications/application/action/list
  This returns below values for each action
  • id 
  • category 
  • name
  • description
  • input(s)
    
    • name
    • type
      • single-select (to indicate that UI can use radio buttons for display)
        • options (set of values)
      • multi-select  (to indicate that UI can use check boxes for display)
        • options (set of values)
      • text (to indicate that UI can use a text box for display)
  • http-method (GET/POST/PUT to be used when initiating the action)

For a category of actions related to savepoint, each of them to have a separate API like below:

• GET /applications/application/savepoint/list
• POST /applications/application/savepoint/trigger
• PUT /applications/application/savepoint/dispose
• PUT /applications/application/savepoint/restore

List of More actions:

List Savepoints:

Restore/Dispose Savepoint:

JSON template for action list is here.

User Experience

Action agnostic implementation at console front end to have a seamless user experience for all actions across flink, spark, oozie, etc.

Alternatives

console-front-end to have action specific code.