Hello,
I'm trying to achieve a complex action where we currently have a common index filebeat-%{beat version}-%{YY-mm-dd} type of index with multiple log.file.path
different. The idea is to have them in other index depending on this log.file.path.
Ex:
From:
filebeat-7.17.1-2022.01
log.file.path: /var/log/log1
log.file.path: /var/log/log2
log.file.path: /var/log/log3
To:
filebeat-log1-2022.01
log.file.path: /var/log/log1
filebeat-log2-2022.01
log.file.path: /var/log/log2
filebeat-log3-2022.01
log.file.path: /var/log/log3
Here I tried with the following setup in logstash to try
input {
elasticsearch {
hosts => "http://localhost:9200"
index => "filebeat-7*"
query => '{
"query":{
"match":{
"log.file.path" : "/var/log/log1"
}
}
}'
size => 10000
scroll => "20s"
}
}
output {
elasticsearch {
hosts => "http://localhost:9200"
index => "filebeat-log1-%{+YYYY.MM}"
}
}
input{
elasticsearch {
hosts => ["http://localhost:9200"]
}
}
filter {
elasticsearch {
hosts => ["http://localhost:9200"]
index => ["filebeat-7*"]
query => "log.file.path:/var/log/log1"
}
}
output {
elasticsearch {
hosts => "http://localhost:9200"
index => "filebeat-log1-%{+YYYY.MM}"
}
}
In my first setup, index are created correctly for other log comes into picture and in the second case, nothing is done.
Do you have any suggestion on this topic ?
Thank you in advance for your help.
Best regards,
Benjamin