Add policies

[brieger-atlassian]

Currently the IAM account change class only allows for a single policy. We needed multiple policies to exist for GCP Account Change log, so we're adding a policies attribute.

[mikeradka]

Is the intent for this to simply be a plural of the existing policy attribute? Has it been considered to make this an array of the policy object?

[zschmerber]

@mikeradka We considered this but Policies already existed in the schema for this use-case.
see: http://0.0.0.0:8080/objects/agent?extensions=

[brieger-atlassian]

Is the intent for this to simply be a plural of the existing policy attribute? Has it been considered to make this an array of the policy object?

policies is an array of the policy object. We are currently planning on leaving both (policy and policies) so logs that will always have exactly one policy can be put under the policy object. We could deprecate policy in favor of policies but this would be a breaking change and require much more effort to coordinate the rollout. Even if we did go this route there would need to be an intermediate state where we have both policy and policies so every instance of using policy can be moved to policies before dropping the policy attribute.

[mikeradka]

Is the intent for this to simply be a plural of the existing policy attribute? Has it been considered to make this an array of the policy object?

policies is an array of the policy object. We are currently planning on leaving both (policy and policies) so logs that will always have exactly one policy can be put under the policy object. We could deprecate policy in favor of policies but this would be a breaking change and require much more effort to coordinate the rollout. Even if we did go this route there would need to be an intermediate state where we have both policy and policies so every instance of using policy can be moved to policies before dropping the policy attribute.

Thanks @zschmerber and @brieger-atlassian that is pretty straightforward.

[alanisaac]

We are currently planning on leaving both (policy and policies) so logs that will always have exactly one policy can be put under the policy object. We could deprecate policy in favor of policies but this would be a breaking change and require much more effort to coordinate the rollout.

I think this change could be confusing for log publishers. If I were creating one of these objects new, even if I had one policy, I would wonder why there seem to be two fields to represent it.

We have the ability to deprecate fields without removing them from the schema to aid in backwards compatibility. See, for example,

ocsf-schema/objects/cve.json

Lines 14 to 20 in 178bc27

	"cwe": {
	"@deprecated": {
	"message": "Use <code>related_cwes</code> attribute instead.",
	"since": "1.4.0"
	},
	"requirement": "optional"
	},

Should we aim to do that here, to more explicitly indicate that policy will be removed in a future release?

[mikeradka]

We are currently planning on leaving both (policy and policies) so logs that will always have exactly one policy can be put under the policy object. We could deprecate policy in favor of policies but this would be a breaking change and require much more effort to coordinate the rollout.

I think this change could be confusing for log publishers. If I were creating one of these objects new, even if I had one policy, I would wonder why there seem to be two fields to represent it.

We have the ability to deprecate fields without removing them from the schema to aid in backwards compatibility. See, for example,

ocsf-schema/objects/cve.json

Lines 14 to 20 in 178bc27

	"cwe": {
	"@deprecated": {
	"message": "Use <code>related_cwes</code> attribute instead.",
	"since": "1.4.0"
	},
	"requirement": "optional"
	},

Should we aim to do that here, to more explicitly indicate that policy will be removed in a future release?

@alanisaac this is a good point. Deprecation is certainly an option, though I think we have been trying to avoid deprecation if we can. I am not sure the correct answer here, but I do want to highlight another option that was brought up in the past and I don't think a decision was reached - a while back, a proposal for an array profile was raised to provide an easier way to enable plurals where only singletons were available. Here is a link to the discussion: #952 @zschmerber @floydtree do you recall this discussion, and do we think this is another scenario to consider within that discussion?

Deprecation is certainly the path of least resistance, but I didn't want to overlook the array profile discussion that was in flight. I suppose my question to this approach would be: is this one of those instances where we set ourselves up for more deprecations in the future when arrays are needed over singletons?

[floydtree]

I agree with @alanisaac here. Leaving both policy and policies in would be a bit cumbersome for all types of users of OCSF, be it a consumer(they will have a fragmented query/analytics experience) or the mapper/producers(they will have multiple choices to populate the same information.) I think we should streamline it and make it more guided for our users. Deprecating policy in favor of policies in the event would be the way to go.

---

Schema modelling aside, a separate question. How do existing Activity IDs correlate to the target policy? For example, for Attach/Detach Policy activity, which policy in the policy array is the target of this attachment/detachment? Or is it the case that GCP populates multiple policy objects only when multiple policies were affected and a singleton otherwise?

[floydtree]

@mikeradka regarding the profile for arrays topic, adding a few thoughts. Interested to hear more perspectives on it. @pagbabian-splunk @zschmerber @alanisaac @Aniak5

I am guessing the array profile idea is similar in spirit to the datetime profile -> A shadow array attribute added for a singleton attribute on profile selection.
I don't know if profile for "shadow" array attributes would be feasible in the framework as is today. Primarily because, our current naming structure of attributes is not fully uniform for singletons vs arrays. We have multiple conventions, from simply being plural to a _list suffix. Can get quite messy to automate using a profile.

Additionally, thinking about downstream consumption, it will be very tricky for consumers if producers apply an array profile for one event and not for the other in the same stream of events. Overall, I think making array decisions explicitly wherever required makes sense, instead of an automated profile addition. We can deprecate singletons along the way as necessary.

[zschmerber]

We have many places where we use this methodology of adding an array to a singleton field in the past. @floydtree should we consider a framework adjustment to fix this PR and all the others that have worked this way ?

[floydtree]

We have many places where we use this methodology of adding an array to a singleton field in the past. @floydtree should we consider a framework adjustment to fix this PR and all the others that have worked this way ?

Not sure what you mean here, mind elaborating based on the series of other conversations above? Because there's a ton of context in those that I don't want to lose.

[zschmerber]

We must have posted at about the same time or i missed the above. I agree with the statement:
Additionally, thinking about downstream consumption, it will be very tricky for consumers if producers apply an array profile for one event and not for the other in the same stream of events. Overall, I think making array decisions explicitly wherever required makes sense, instead of an automated profile addition. We can deprecate singletons along the way as necessary.