Hello,
I am attempting to setup custom indexes for winlogbeat via logstash using ILM settings in the elasticsearch output. I would like to use the default template for winlogbeat (winlogbeat-7.3.0) but have different custom indexes for each organization. The purpose of the separate indexes is to have users be able to login to the kibana instance with roles that can only view their particular indexes for winlogbeat. I would like to use the following base structure for the indexes
winlogbeat-{agent.version}-organization
I figured appending the organization to the end would allow me to use the default template for winlogbeat according to each version as well as to use the built in dashboards and visualizations for winlogbeat-*. Also, it would allow me to use a default ILM policy for all organization indexes just by specifying the index pattern in a specific ILM template similar to how it is described here
My problem is mainly when specifying the output for ilm_rollover_alias in logstash's elasticsearch output it is not appending the date and increment number like I would hope. Here is the logstash config:
input {
beats {
port => 5044
}
}
output {
elasticsearch {
hosts => "elastic_cloud_url"
user => "user"
password => "password"
ilm_enabled => true
ilm_rollover_alias => "%{[@metadata][beat]}-%{[@metadata][version]}-miles"
ilm_policy => "hot-warm-delete-30days"
}
}
So it is creating the index and applying the correct ILM policy but the index name is just what is in the rollover alias without the ilm_pattern appended to the end:
winlogbeat-7.3.0-miles
It also seems to be using the correct winlogbeat-7.3.0 template from what I can tell on the index settings since I configured that one in particular to have 5 shards and 1 replica. Below is snippet pulled from index settings via kibana:
{
"settings": {
"index": {
"lifecycle": {
"name": "hot-warm-delete-30days",
"rollover_alias": "winlogbeat-7.3.0"
},
"routing": {
"allocation": {
"require": {
"data": "hot"
}
}
},
"mapping": {
"total_fields": {
"limit": "10000"
}
},
"refresh_interval": "5s",
"number_of_shards": "5",
"provided_name": "winlogbeat-7.3.0-miles"
I noticed in the snippet above also that the rollover_alias is the default from the winlogbeat-7.3.0 template so something doesn't seem correct. I am getting the error in kibana
illegal_argument_exception: index.lifecycle.rollover_alias [winlogbeat-7.3.0] does not point to index [winlogbeat-7.3.0-miles]
Am I able to structure my data and policies in this way or am I way off base here?