add fluentbit to opensearch log shipping #19
Conversation
…gke-cluster into fluentd-opensearch
jaketf
requested review from
chrisghill and
WillBeebe
and removed request for
WillBeebe
| properties: | ||
| observability: |
There was a problem hiding this comment.
If we like this structure we can make this a type and ref it from each cloud's cluster.
There was a problem hiding this comment.
I like that idea. I'll back it out to a type once I confirm it's working as expected in staging
There was a problem hiding this comment.
I would advise not extracting this to a separate type. EKS will have log options the others won't (Cloudwatch, managed Opensearch), GKE same (Stackdriver) and same with AKS.
We should use the same structure, but not force them to use the same type.
| - collection | ||
| - destination | ||
| properties: | ||
| collection: |
There was a problem hiding this comment.
We don't need to ask them about the "collector". If they select a log destination that requires a collector, it will be fluentbit. This is an implementation detail that users shouldn't need to care about as long as the logs arrive where they want.
This also becomes a logic issue when we support datadog, where their agent does collection. What happens if they select datadog collector and opensearch destination?
All we need to ask is where they want their logs. We'll handle the internals to get it there.
| properties: | ||
| observability: |
There was a problem hiding this comment.
I would advise not extracting this to a separate type. EKS will have log options the others won't (Cloudwatch, managed Opensearch), GKE same (Stackdriver) and same with AKS.
We should use the same structure, but not force them to use the same type.
both services are running still working on getting them to talk properly.
Note, revised the scope on this with @xpositivityx to remove SSL / custom users to a future PR. See massdriver-cloud/terraform-modules#31