Apologies I do not know what is not working with your setup
I just did this ... this is literally all I did.
- Completely fresh default install of Elasticsearch / Kibana 7.12.0
- Edited and combined filebeat.yml and nginx.yml into single minimal file see below.
- $ ./filebeat setup -e
- $ ./filebeat -e
Result
GET _cat/indices/file*?v
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open filebeat-7.12.0-2022.08.02-000001 Lg5TuGtcRwKYc7DN7YeXqQ 1 1 0 0 208b 208b
yellow open filebeat-7.12.0-nginx-access-2022.08.02 hiiPnYcIRSOe3BoYn1PYMQ 1 1 7 0 36.1kb 36.1kb
This is my entire filebeat.yml (I combined them which is perfectly valid, to reduce variables)
filebeat.modules:
- module: nginx
access:
enabled: true
input.index: "filebeat-%{[agent.version]}-nginx-access-%{+yyyy.MM.dd}"
input.tags: ["customer-a"]
var.paths: ["/Users/sbrown/workspace/sample-data/nginx/nginx-test.log"]
setup.template.settings:
index.number_of_shards: 1
setup.kibana:
output.elasticsearch:
hosts: ["localhost:9200"]
You have something going on... another .yml (this has happened to me before), your not using the .yml you think you are... bad syntax ... something ... or something is not default with the pipeline, alias something cluster etc.
I can provide my docker compose and test data if you like...
Something is also weird have you set some odd refresh rate I see even on the index you appear to be writing to 0 docs.. also 1 has 1 replica and the other 0... this leads me to believe there is something else going on... did you create your own templates or something with the same matching patterns... there could be conflict or the order they are applied.. something strange is going on
# curl -X GET "localhost:9200/_cat/indices?v"
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open filebeat-7.12.0-else02-httpd-access-2022.08.02 I_rWmCFZTMCr4f8TGcK4zg 1 0 0 0 208b 208b
yellow open filebeat-7.12.0-2022.08.02-000001 jUGcGI4hTX2pZF643ObQ2Q 1 1 0 0 71.5kb 71.5kb
technically looking very close there is 1 issue we would resolve and that is removing the ILM but that is not the cause of your issue...
it looks to me that filebeat is still writing to the write alias....
{
"filebeat-7.12.0-2022.08.02-000001" : {
"aliases" : {
"filebeat-7.12.0" : {
"is_write_index" : true
}
},
which means your filebeat is still writing to the default filebeat-7.12.0 why I am not sure.