Creating index depends on source

Elastic Stack Logstash
2

Hi @Kanthasamyraja,
To achieve this I have done the following.

In Filebeat I set some fields like

- type: log
  paths:
    - /var/log/sensu/*.log
  encoding: plain
  fields:
    log_prefix: dc
    log_idx: sensu-logs
  fields_under_root: false

In Logstash I create @metadata fields based on these.

# Adding @metadata needed for index sharding to Filebeat logs
mutate {
  copy => {
   "[fields][log_prefix]" => "[@metadata][log_prefix]"
   "[fields][log_idx]" => "[@metadata][index]"
  }
}

And my Elasticsearch output if Logstash can then be just

  elasticsearch {
        hosts => ["10.1.1.1:9200", "10.1.1.2:9200", "10.1.1.3:9200"]
        index => "%{[@metadata][log_prefix]}-%{[@metadata][index]}-%{+YYYY.MM.dd}"
  }

You can drop one of the @metadata fields if you just need the one "switch".

Hope that helps,
AB