Motivation

Backwards compatibility:

KiteSDK? No (Is not used by any of our examples or the community as far as we know)

Path structure?

Use original Envelop for output? No

Consistency on the output schema? Yes.

Conclusion: This change require a major release number increment as the output schema will be altered.

Proposal

Plan

    1. Simple externalized configuration of ingest (PNDA-4553)
      • User assigned Family ID
      • Use as fallback when no configuration is found in the Registry Service.
    2. Externalized ingest configuration registry service (PNDA-4465)
      • Organize data in suitable registry (use the integrated consul, Extending https://github.com/confluentinc/schema-registry, ...)
      • Mechanism to create data in registrar, issue family IDs (a controlled list per deployment), map to topics, etc
      • Fill out the client implementation to use registry instead of stubbed data
      • Expose with REST API

Interfaces