Page tree
Skip to end of metadata
Go to start of metadata

Overview


For the first phase, we will focus on this target. Essentially this focuses on securing user access through a gateway, while ops access continues to be carried out over SSH. Ingest access to secured separately from either of these. Thus, this could be deployed on three VLANs, with the core PNDA nodes not requiring more than one interface.



  • Integrate upstream Knox version 1.0.x as part of PNDA deployment in order to cover main HDP services
    • We won't use the (older) bundled HDP/Ambari Knox integration 
    • We will reuse the default Knox HDP/Ambari configuration with adjustment (service name) -
      • WebHDFS (HDFS)
      • Yarn RM
      • Stargate (Apache HBase)
      • Apache Oozie
      • Apache Hive/JDBC
      • Apache Hive WebHCat (Templeton)
      • Name Node UI
      • Job History UI
      • Yarn UI
      • Apache Oozie UI
      • Apache HBase UI
      • Apache Spark UI

    • We will add other user facing PNDA services into Knox configuration -
      • Deployment manager API (for PoC DM-Knox integration see here)
      • Package Repository API
      • OpenTSDB API
      • PNDA UI 

  • Links to services (for example quicklinks) will be configured to direct to the gateway, rather than to the individual services
  • Bring user facing services not covered by Knox under consistent access regime via HAProxy -
    • JupyterHub
    • Grafana

  • Operations access will be via SSH and will still require a SOCKS proxy for UIs not listed above. That list includes -
    • Ambari 
    • Kafka Manager
    • ELK
    • Data Service
    • Console back end
    • Some Hadoop APIs/UIs
  • We will introduce HAProxy on OpenTSDB ingest

  • We will disable the OpenTSDB UI and rewire the PNDA UI element for OpenTSDB to an ELK filter of OpenTSDB logs.

Authentication


  • PNDA will provide a configuration point to set up an external LDAP. This will be used to configure Knox, Grafana and Jupyter authentication.
  • On the ingest interfaces, it will be left to the operator to configure authentication via TLS and certificates.
  • Server authentication of the gateway services will be accomplished via TLS and certificates.
  • PNDA will provide a mechanism to supply a tree of security material (certificates, keys) that is then used to configure TLS on the services mentioned above.
  • Operations access will be authenticated via PAM, on connection over SSH. 

Authorization


  • Authorization will be decentralized -

    • We will add a configurable authorization feature based on ABAC principles to some user facing PNDA services, including the Deployment Manager and Package Repository.
    • The PNDA UI will rely on sensible handling of authorization errors from these services to control access - it will not provide role or authorization filtered views.
    • Authorization for Grafana and Jupyter will be configured on those services by the operator
    • Authorization on ingest will be configured on those services, with respect to principles based on certificates used in authentication, by the operator
    • Authorization on operations interfaces will be configured on those services by the operator


  • No labels