Resource: Add LBAC for datasources data_source_config_lbac_rules

[eleijonmarck]

why/what
We have seen that users would love to provision their lbac rules using terraform. this implements a experimental feature that will be highly developed. this is the first part of LBAC rules resource for terraform.

dependancies:

• Data source: Add delete teamHttpHeaders in the json read path for data source #1857 ✅
• Data source: Add check for write path of teamHttpHeaders in the json encoding #1856 ✅
• https://github.com/grafana/grafana-enterprise/pull/7449 ✅
• Chore: go.mod Update to latest openapi spec  #1905 ✅
• https://github.com/grafana/grafana-enterprise/pull/7469

ability to run terraform plan, terraform apply and terraform destroy of the rules resource

Example terraform:

resource "grafana_data_source" "test" {
  type = "loki"
  name = "loki"
  url  = "http://localhost:3100"
  basic_auth_enabled  = true
  basic_auth_username = "username"

  lifecycle {
    ignore_changes = [json_data_encoded, http_headers]
  }
}

resource "grafana_team" "test" {
	name = "test"
}

resource "grafana_data_source_config_lbac_rules" "test" {
    datasource_uid = grafana_data_source.test.uid
    rules = jsonencode({
        "${grafana_team.test.uid}" = [
            "{ foo != \"bar\", foo !~ \"baz\" }",
            "{ foo = \"qux\" }"
        ]
    })
}

why is this difficult

lbac for datasources terraform resource, why is it a bit tricky?

the bidirection of lbac_rules -> data_source:jsonData, meaning we are updating the jsondata with another resource impacting another resource

---

proposals and thought process

there is no ID for the json field teamHttpHeaders

• so the state is only “stored” in the datasource as a whole, therefore leaning on copying the data_source_config resource more
  this is a resource that will be heavily used
• i would like to have the internal state for terraform (some customers create about 500 rules and more), which clogs the datasource, i would like to copy the setup from grafana_data_source_permission_item for this. as it suits well for how we would like to setup this resource
• instead we make use of the api as it is. it is bulk update anyway. so we assume that all of the teams will be provided in the resource of resource_data_source_config_lbac_rules

proposal of all rules combined into one resource

benefits

• one api call to grafana
• mimicing the get/update apis

tradeoffs

• clonky json encoding (could be fixed)
• not ideal for teams managing their own set of rules

resource "grafana_data_source_config_lbac_rules" "test" {
    datasource_uid = grafana_data_source.test.uid
	org_id = grafana_team.test.org_id
    rules = jsonencode({
        "${grafana_team.test.id}" = [
            "{ foo != \"bar\", foo !~ \"baz\" }",
            "{ foo = \"qux\" }"
        ]
    })
}

proposal of each rule being a resource

benefits

• each team rules, would be able to be owned by a team. this is preferred from a user perspective

tradeoffs

• too many api calls to grafana, if we have 500 rules we would have 500 api calls to the datasource endpoint

resource "grafana_data_source_config_lbac_rule" "test" {
    datasource_uid = grafana_data_source.test.uid
    org_id = grafana_team.test.org_id
    team_id = "1"
    rules = [
            "{ foo != \"bar\", foo !~ \"baz\" }",
            "{ foo = \"qux\" }"
        ]
}

proposal of embedding it into a resource and then add it to the json

benefits:

• easier to implement

tradeoff

• management of lbac rules would lie inside of the json and we would not be able to move the storage of the lbac rules

It's a tradeoff of where you want the complexity to be. In Grafana with a new way to store this, or in the provider extracting it from json data, and possibly running into versioning issues.

I guess the less complex option is to leave everything as-is, isn't it? Have people manage those new fields in the existing json data? Perhaps provide a datasource helper for this, if the value is not trivial to build/represent

data "grafana_datasource_team_headers" "helper" { ... }

resource "grafana_datasource" "test" {
  ...
  json_data_encoded = jsonencode({
     teamHeaders = data.grafana_datasource_team_headers.helper.value
     otherfields
  })
}

run tests via

GRAFANA_VERSION=11.1.0 
GRAFANA_URL=http://localhost:3000 
GRAFANA_AUTH=admin:admin 
TESTARGS="-run TestAccDataSourceLBAC_inOrg" make testacc-enterprise

The teamHttpHeaders resource is a datasource json field.

epic

basically this introduced a way to add/update a datasource json field
lbac-api

here is the doc for it. https://grafana.com/docs/grafana/latest/administration/data-source-management/teamlbac/

---

next steps

improvements:

• API of the resource could be made so that there is no need of escaped JSON. To enforces types and schema

1. Create a template file (logql_query.tmpl):

{f="alice", b="bob", a="abc"}

resource "grafana_data_source_config_lbac_rules" "test" {
    datasource_uid = grafana_data_source.test.uid
    rules = jsonencode({
        "1" = [
            "{ foo != \"bar\", foo !~ \"baz\" }",
            "{ foo = \"qux\" }"
        ],
        "2" = [
            templatefile("${path.module}/logql_query.tmpl", {})
        ]
    })
}

```[tasklist]
- [ ] fix the lifecycle of the jsondata within grafana
  - [ ] make sure you cannot set teamhttpheaders from the json api (updateDatasource), ONLY through lbac/team api
  - [ ] magic diffsuprressfunc to ignore changes on json_data_encoded for the specific field teamHttpHeaders

[github-actions]

In order to lower resource usage and have a faster runtime, PRs will not run Cloud tests automatically.
To do so, a Grafana Labs employee must trigger the cloud acceptance tests workflow manually.

[undefinedhuman]

Hey there, is there an ETA when this feature will be shipped? Or is there anything that still needs to be done?

[eleijonmarck]

@undefinedhuman we are working on the backend side of things for this. So this will wait for clear resource handling

[undefinedhuman]

Thank you very much!

[Robsta86]

What's the status of this? I was using teamHttpHeaders to configure lbac via the grafana_datasource_config, but that seems to be broken after these 2 are merged:

#1857
#1856

Error: teamHttpHeaders is a reserved key and cannot be used in JSON data. Use the data_source_config_lbac_rules resource instead

Now I am no longer able to manage my loki lbac rules via terraform, resulting in new onboarded teams to have full query access to my log datasource.

[IevaVasiljeva]

Mostly looks good, but I've added a few questions.

[IevaVasiljeva]

This commended out code block also appears in the docs, so we should really remember to uncomment the code before we do the next Terraform release.

Or maybe just track this as an action item in an issue rather than having it here.

[eleijonmarck]

il track it in an item instead 👍

[eleijonmarck]

https://github.com/grafana/identity-access-team/issues/1047

[IevaVasiljeva]

Nice, I've added some small comments, but once those are fixed it looks ready to me 👌

[IevaVasiljeva]

I think we actually need to set it to 11.5.0 - 11.4.0 will not include the changes that we need (see this internal thread).

[IevaVasiljeva]

Is this line needed?

[eleijonmarck]

it's not needed at all. just showing that you need to have a password to connect with the datasource from basicAuth

[IevaVasiljeva]

Is this actually used?

[eleijonmarck]

wait it is unused, i should really start to make my structs private on first creation. how do you spot this? with the linter

[IevaVasiljeva]

Suggested change

	Description: "JSON-encoded LBAC rules for the data source. Map of team IDs to lists of rule strings.",
	Description: "JSON-encoded LBAC rules for the data source. Map of team UIDs to lists of rule strings.",

[IevaVasiljeva]

Should also be 11.5.0.

[eleijonmarck]

Tested feature fully ✅ :

• rejects the teamHttpHeaders field in json when trying to update a json field in the datasource resource.
• do not change state when data source was provisioned via terraform and users manually adds a rule / removes rules
• correctly adds resource
• correctly delete resource
• correctly updates/changes the resource, if user adds more rules

---

This feature would be issued together with the release of Grafana version 11.5 in 28th of January