Hello, we are currently employing the Elastic agent on Kubernetes, managed by Fleet, and have incorporated various Elastic integrations such as Kubernetes, Auditd, Audit_manager, and System. Our primary goal is to monitor both the logs and metrics originating from our server and Kubernetes cluster.
For testing purposes, we have established a straightforward ILM (Index Lifecycle Management) policy, which initiates index rollover after 10 minutes and proceeds to delete indices after 20 minutes.
resource "elasticstack_elasticsearch_index_lifecycle" "im_ilm" {
name = "ilm-policy-${var.cluster.name}"
hot {
set_priority {
priority = 0
}
rollover {
max_age = "10m"
}
}
delete {
min_age = "20m"
}
}
Additionally, we have crafted a custom component template and assigned the ILM policy to it.
resource "elasticstack_elasticsearch_component_template" "custom_template_playground" {
name = "metrics-${var.cluster.name}@custom"
template {
alias {
name = "metrics-playground@custom"
}
settings = jsonencode({
index = {
lifecycle = {
name = elasticstack_elasticsearch_index_lifecycle.im_ilm.name,
rollover_alias = elasticstack_elasticsearch_index_lifecycle.im_ilm.name
},
number_of_replicas = "0"
number_of_shards = "1"
}
})
}
}
Lastly, an index template has been created for all indices following a pattern similar to "metrics-*-playground6" as we aim to aggregate metrics specifically from the playground6 environment.
resource "elasticstack_elasticsearch_index_template" "custom_index_template" {
name = "metrics-${var.cluster.name}"
priority = 250
index_patterns = ["metrics-*-${var.cluster.name}-*"]
composed_of = [
elasticstack_elasticsearch_component_template.custom_template_playground.name,
"metrics-mappings",
"data-streams-mappings",
"metrics-settings",
]
data_stream {
}
template {
settings = jsonencode({
index = {
number_of_replicas = "0"
number_of_shards = "1"
}
})
}
}
It's worth noting that there are currently no linked index templates or attached linked indices in our ILM policy.
As evident from the default metrics ILM policy, there are 54 linked index templates and 69 linked indices. How can we automatically incorporate these indices and index templates into the custom ILM?
