[ANSIENG-3807] | Rbac over mTLS #1804

rrbadiani · 2024-10-18T13:47:28Z

Description

This Pr aims to add support for RBAC over mTLS.
The includes changes for

MDS
Broker
Controller
SR
ERP
RP
Connect
KSQL
C3
Using cert based token retrival
Extracting Principal from certs to assign role bindings
Molecule tests
Setting Principal Mapping Rules
Setting impersonation super users for principal propogation

Replicator changes will be raised in seperate PR

Fixes # (issue)

Type of change

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
This change requires a documentation update

How Has This Been Tested?

zookeeper

kraft

Checklist:

Any variable/code changes have been validated to be backwards compatible (doesn't break upgrade)
I have added tests that prove my fix is effective or that my feature works
If required, I have ensured the changes can be discovered by cp-ansible discovery codebase
My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
My changes generate no new warnings
Any dependent changes have been merged and published in downstream modules

* [ANSIENG-4229] | adding new user facing variables for rbac over mtls * [try-mtls] | modified default listener and get auth token * [ANSIENG-4229] | adding 2 small scenarios for testing * [try-mtls] | [ANSIENG-4229] | modifying the default values of ssl_client_authentication and ssl_mutual_auth * [try-mtls] | fixing issues in set principal for mtls and health checks * [try-mtls] | fixing mds client auth properties * [try-mtls] | adding ldap+mtls sceanrio * [try-mtls] | changing os to rhel9 and java to 17 for mtls tests * [try-mtls] | fix listener auth issue * [try-mtls] | fix health checks for broker * [try-mtls] | adding impersonation super users in bk customer properties * [try-mtls] | unmasking secrets for better logging * [try-mtls] | fix mds health check and config validations to include auth mode mtls * [try-mtls] | fix get authorization tokens * [try-mtls] | temporary removal of ssl endpoint identification * [try-mtls] | fix verify of new molecule scenarios for mtls * [try-mtls] | chaning erp listener to oauthbearer listener for impersonation token * [try-mtls] | fixing qoutes in erp listener name * [try-mtls] | removing ssl client auth verification from controller as it is only mds server property * [try-mtls] | adding impersonation super and protected users in mds properties * [try-mtls] | modify ssl client authentication for listeners based on inventory file * [try-mtls] | principal mapping rules on listeners * [try-mtls] | overriding principal mapping rules in molecule * [try-mtls] | fix impersonation protected user and remove config override for impersonation super user * [try-mtls] | remove principal mapping rules * [try-mtls] | fixing qoutes around client auth mode in listeners * [try-mtls] | adding sr in mtls only setup * [try-mtls] | fixing sr mtls * [try-mtls] | fix ldap detection and add extra $ in molecule for escape reasons * Fix SR RBAC (#70) * [ANSIENG-4229] | adding new user facing variables for rbac over mtls * [try-mtls] | modified default listener and get auth token * [ANSIENG-4229] | adding 2 small scenarios for testing * [try-mtls] | [ANSIENG-4229] | modifying the default values of ssl_client_authentication and ssl_mutual_auth * [try-mtls] | fixing issues in set principal for mtls and health checks * [try-mtls] | fixing mds client auth properties * [try-mtls] | adding ldap+mtls sceanrio * [try-mtls] | changing os to rhel9 and java to 17 for mtls tests * [try-mtls] | fix listener auth issue * [try-mtls] | fix health checks for broker * [try-mtls] | adding impersonation super users in bk customer properties * [try-mtls] | unmasking secrets for better logging * [try-mtls] | fix mds health check and config validations to include auth mode mtls * [try-mtls] | fix get authorization tokens * [try-mtls] | temporary removal of ssl endpoint identification * [try-mtls] | fix verify of new molecule scenarios for mtls * [try-mtls] | chaning erp listener to oauthbearer listener for impersonation token * [try-mtls] | fixing qoutes in erp listener name * [try-mtls] | removing ssl client auth verification from controller as it is only mds server property * [try-mtls] | adding impersonation super and protected users in mds properties * [try-mtls] | modify ssl client authentication for listeners based on inventory file * [try-mtls] | principal mapping rules on listeners * [try-mtls] | overriding principal mapping rules in molecule * [try-mtls] | fix impersonation protected user and remove config override for impersonation super user * [try-mtls] | remove principal mapping rules * [try-mtls] | fixing qoutes around client auth mode in listeners * [try-mtls] | adding sr in mtls only setup * [try-mtls] | fixing sr mtls * [try-mtls] | fix ldap detection and add extra $ in molecule for escape reasons --------- Co-authored-by: Mansi Jain <[email protected]> * [ANSIENG-4233] | added mtls configs for connect * [ANSIENG-4233] | comment fix * [try-mtls] | fix sr changes * [ANSIENG-4233] | config fix for listener authentication * [ANSIENG-4233] | add config for connectors * [ANSIENG-4233] | code fix * [ANSIENG-4233] | code revert * [try-mtls] | modify molecule scenario to add super user and principal mapping rules * [ANSIENG-4233] | add connector mtls config * [ANSIENG-4236] | add connect replicator mtls config * [ANSIENG-4233] | delegate token fetch to broker for connector * [ANSIENG-4236] | property fix * [ANSIENG-4235] | mtls configs * [ANSIENG-4235] | mtls configs * [ANSIENG-4236] | mtls configs * [ANSIENG-4236] | mtls configs * [ANSIENG-4235] | mtls configs * [ANSIENG-4235] | property fix * [ANSIENG-4236] | add test for replicator * [pm-rules] | handle default principal mapping rules * [pm-rules] | principal mapping rules in mds, erp, listeners, sr * [pm-rules] | removing config overrides from mtls onnly scenario as principal mapping rules are added by variables * [pm-rules] | principal mapping rules fix to get proper super user * [pm-rules] | adding listener level control over principal mapping rules * [pm-rules] | fix principal mapping rules in listeners * [mtls-rp] | add mtls support in erp,rp * [mtls-rp] | fix kafka rest license issue * [ANSIENG-4235] | add eol * Connect mtls ansieng 4233 (#72) * [ANSIENG-4233] | added mtls configs for connect * [ANSIENG-4233] | comment fix * [ANSIENG-4233] | config fix for listener authentication * [ANSIENG-4233] | add config for connectors * [ANSIENG-4233] | code fix * [ANSIENG-4233] | code revert * [ANSIENG-4233] | add connector mtls config * [ANSIENG-4233] | delegate token fetch to broker for connector * Revert "Connect mtls ansieng 4233 (#72)" (#76) This reverts commit 57cc5e6. * [try-mtls] | c3 mtls support * [try-mtls] | adding c3 in mtls scenario * [try-mtls] | fix c3 bugs * [try-mtls] | adding impersonation users to molecule scenarios * [try-mtls] | fix kafka rest listener and conlfuent.license config * [try-mtls] | file based login in mtls only scenario added * [try-mtls] | fix oauth and ldap scenarios and confluent.license * [try-mtls] | send certs in kafka broker tasks for register cluster * [try-mtls] | fixing register cluster to run on internal token listener so it has sasl_ssl protocol instead of ssl for rbac over mtls * [mtls-connect] | making retries in get auth token configurable and increasing the default wait time as mds takes more time in upgrades * [mtls-connect] | adding ksql connect in molecule scenarios * [mtls-connect] | fix erp pm rules * [mtls-connect] | remove cyclic dependency in ssl_client_authentication and ssl_mutual_auth_enabled * [mtls-connect] | remove set fact for mtls old var * [mtls-connect] | add when conditions for extract principal --------- Co-authored-by: Mansi Jain <[email protected]>

… rules

CLAassistant · 2024-10-18T13:47:37Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

confluent-cla-assistant · 2024-10-18T13:47:41Z

🎉 All Contributor License Agreements have been signed. Ready to merge.
_{Please push an empty commit if you would like to re-run the checks to verify CLA status for all contributors.}

…abled and ssl_client_authentication and askign for user confirmation for setting requested

…instead of oauth_enabled and ldap_with_oauth_enabled varaibles

rrbadiani and others added 14 commits October 16, 2024 10:48

[mtls-combined] | fix bugs in delegate and getting pm rules

a5a9316

[mtls-combined] | fix connect rbac send certs

376ccaa

[mtls-combined] | fix connect rbac, ksql rbac send certs

9165607

[mtls-combined] | fix ksql send certs

48af80e

[mtls-combined] | fix get token bug, missing delegate and molecule pm…

067d35c

… rules

[mtls-combined] | handle provided ks/ts

0da75e7

[mtls-combined] | fix verify of 2 scenarios

91a0970

[mtls-combined] | dont delete cert in auth mode mtls

b6430cf

[mtls-combined] | verify fixes

90a980c

[mtls-combined] | adding old var for mtls

bd1689c

[mtls-combined] | remove debug logs

5332335

[ANSIENG-3807] | revert replicator mtls changes

069ef31

[ANSIENG-3807] | adding more comments

c25dc50

rrbadiani requested a review from a team as a code owner October 18, 2024 13:47

[ANSIENG-3807] | fixing linting in filters.py file

ca89017

rrbadiani changed the base branch from 7.8.x to master October 19, 2024 09:53

rrbadiani changed the base branch from master to 7.8.x October 19, 2024 09:53

rrbadiani added 2 commits October 20, 2024 10:28

[ANSIENG-3807] | removing cyclic dependency of old ssl_mutual_auth_en…

197c02d

…abled and ssl_client_authentication and askign for user confirmation for setting requested

[ANSIENG-3807] | fixing the variables/vars.yml file to use auth_mode …

149140c

…instead of oauth_enabled and ldap_with_oauth_enabled varaibles

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ANSIENG-3807] | Rbac over mTLS #1804

[ANSIENG-3807] | Rbac over mTLS #1804

rrbadiani commented Oct 18, 2024

CLAassistant commented Oct 18, 2024

confluent-cla-assistant bot commented Oct 18, 2024

[ANSIENG-3807] | Rbac over mTLS #1804

Are you sure you want to change the base?

[ANSIENG-3807] | Rbac over mTLS #1804

Conversation

rrbadiani commented Oct 18, 2024

Description

Type of change

How Has This Been Tested?

Checklist:

CLAassistant commented Oct 18, 2024

confluent-cla-assistant bot commented Oct 18, 2024