Filebeat not creating new index in Elastic search?

I have a microservice users-ws for which i want to see logs getting pulled by filebeat and send to elasticsearch and logstash for searching and filtering. I have a problem that with
the configuration below its not creating new indexes ..
filebeat.yml -

  - type: log
     enabled: true
       - C:\my_projects\PhotoAppAPIUsers\users-ws.log

     log_type: users-ws-log

path: ${path.config}/modules.d/*.yml
reload.enabled: false

index.number_of_shards: 1

hosts: ["localhost:9200"]

hosts: ["localhost:5044"]

logstash conf :

    input {
       beats {
         port => 5044

  filter {
  if [log_type] == "users-ws-log" {
    mutate { add_field => { "[@metadata][target_index]" => "users-ws-log-%{+YYYY.MM}" 
     } }
      } else if [log_type] == "albums-ws-log" {
       mutate { add_field => { "[@metadata][target_index]" => "albums-ws-log-% 
       {+YYYY.MM.dd}" } }
        } else {
         mutate { add_field => { "[@metadata][target_index]" => "unknown-%{+YYYY}" } }

output {
  elasticsearch {
    hosts => ["localhost:9200"]
    index => "%{[@metadata][target_index]}"
  stdout { codec => rubydebug}

Please let me know if i am missing any thing . Everytime i start my microservice though its generating new logs but i don't see new indexes getting created using filebeat.

Below are the indexes everytime created :
green open ilm-history-1-000001 g8nmTY7UQCiH7VDlSyQyQw 1 0 18 0 25.4kb 25.4kb
yellow open filebeat-7.6.2-2020.04.17-000001 WMENcfDbRsGKNLjwMYIIzw 1 1 7432 0 1.5mb 1.5mb

Also its not creating the indexes the way i have configured in logstash.

Hi @pracks1982, welcome to the Elastic community forums!

Looking at your filebeat.yml, I see two outputs enabled — elasticsearch and logstash. This is not supported by Beats; only one output can be enabled at any given time. Are you sure your Filebeat is running without any errors? Could you check the Filebeat log after restarting it?


This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.