Dear All.
I have following configuration.
Elasticsearch Version 7.4.0
We have several beats in use.
filebeat, metricbeat, packetbeat, auditbeat and heartbeat
We have following to communication streams from the beats to elasticsearch.
beat -> elasticsearch
beat -> logstash (with input beat port 5044 and output elasticsearch) -> elasticsearch
logstash beat configuration:
Beats -> Logstash -> Elasticsearch pipeline.
input {
beats {
port => 5044
}
}
output {
elasticsearch {
hosts => ["${ES_HOSTS}"]
index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
}
}
The beats that are communicating direct to elasticsearch are writing their data into the
filebeat-7.4.0-YEAR.MONTH.DAY-000001
metricbeat-7.4.0-YEAR.MONTH.DAY-000001
packetbeat-7.4.0-YEAR.MONTH.DAY-000001
auditbeat-7.4.0-YEAR.MONTH.DAY-000001
heartbeat-7.4.0-YEAR.MONTH.DAY-000001
The beats that are communicating to logstash creating every day a new index
filebeat-7.4.0-YEAR.MONTH.DAY
metricbeat-7.4.0-YEAR.MONTH.DAY
packetbeat-7.4.0-YEAR.MONTH.DAY
auditbeat-7.4.0-YEAR.MONTH.DAY
heartbeat-7.4.0-YEAR.MONTH.DAY
I have following problems with the ILM
Index lifecycle error:
illegal_argument_exception: index.lifecycle.rollover_alias [filebeat-7.4.0] does not point to index [filebeat-7.4.0-2020.01.28]
illegal_argument_exception: index.lifecycle.rollover_alias [metricbeat-7.4.0] does not point to index [metricbeat-7.4.0-2020.01.27]
illegal_argument_exception: index.lifecycle.rollover_alias [packetbeat-7.4.0] does not point to index [packetbeat-7.4.0-2020.02.09]
illegal_argument_exception: index.lifecycle.rollover_alias [auditbeat-7.4.0] does not point to index [auditbeat-7.4.0-2020.01.31]
illegal_argument_exception: index.lifecycle.rollover_alias [heartbeat-7.4.0] does not point to index [heartbeat-7.4.0-2020.01.27]
I think this means that the beats that are communicationg over logstash are not able to use the Index Lifecycle Policies, because the indexes have not the extension -000001.
I will like to see that also the beats that are communicating over logstash are writing there data into the indexes with the extension -000001.
How can I fix this problem?
best regards
Alexander