Filebeat not creating new index in Elastic search?

I have a microservice users-ws for which i want to see logs getting pulled by filebeat and send to elasticsearch and logstash for searching and filtering. I have a problem that with
the configuration below its not creating new indexes ..
filebeat.yml -

filebeat.inputs:
  - type: log
     enabled: true
     paths:
       - C:\my_projects\PhotoAppAPIUsers\users-ws.log

   fields:
     log_type: users-ws-log

filebeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: false



setup.template.settings:
index.number_of_shards: 1

output.elasticsearch:
hosts: ["localhost:9200"]

output.logstash:
hosts: ["localhost:5044"]

logstash conf :

    input {
       beats {
         port => 5044
   }
   }

  filter {
  if [log_type] == "users-ws-log" {
    mutate { add_field => { "[@metadata][target_index]" => "users-ws-log-%{+YYYY.MM}" 
     } }
      } else if [log_type] == "albums-ws-log" {
       mutate { add_field => { "[@metadata][target_index]" => "albums-ws-log-% 
       {+YYYY.MM.dd}" } }
        } else {
         mutate { add_field => { "[@metadata][target_index]" => "unknown-%{+YYYY}" } }
    }
}

output {
  elasticsearch {
    hosts => ["localhost:9200"]
    index => "%{[@metadata][target_index]}"
  }
  stdout { codec => rubydebug}
}

Please let me know if i am missing any thing . Everytime i start my microservice though its generating new logs but i don't see new indexes getting created using filebeat.

Below are the indexes everytime created :
green open ilm-history-1-000001 g8nmTY7UQCiH7VDlSyQyQw 1 0 18 0 25.4kb 25.4kb
yellow open filebeat-7.6.2-2020.04.17-000001 WMENcfDbRsGKNLjwMYIIzw 1 1 7432 0 1.5mb 1.5mb

Also its not creating the indexes the way i have configured in logstash.

Hi @pracks1982, welcome to the Elastic community forums!

Looking at your filebeat.yml, I see two outputs enabled — elasticsearch and logstash. This is not supported by Beats; only one output can be enabled at any given time. Are you sure your Filebeat is running without any errors? Could you check the Filebeat log after restarting it?

Shaunak

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.