index templates other than logs and metrics when using data streams #752
+9
−13
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
index templates other than logs and metrics when using data streams #751
Problem
In most real world scenarios a user is not in control over the default index templates "logs" and "metrics". There are many usecases where the user wants to define a custom template. In my case I want to use the data retention feature to delete messages stored in elasticsearch after a certain amount of time.
Currently this connector is able to use the predefined index templates "logs" and "metrics" when data streams are enabled. User created templates are not supported yet.
Solution
Defining the type of the data stream as String to make it flexible for custom inputs. The enum will not be removed to still be able to use the recommender when deploying the connector via control-center.
Does this solution apply anywhere else?
Test Strategy
Started a local cluster via docker-compose (cp-quickstart and elastic+kibana) and installed the connector.zip in the cluster. Both connectors with "logs" and connectors with "custom" as data stream types were running perfectly fine after creating the "custom" index template in Kibana.
Testing done: