How to improve Logstash configuration for outputs

Dear team,
can you advise on how to improve my configuration for outputs from Logstash?
In our environment, the only output is Elasticsearch, and we have around 30 sources which are sent to Elasticsearch from logstash based on tags to different indices, like below:

output {
  if "_grokparsefailure" in [tags] {
    stdout { codec => rubydebug }
    file {
      path => "/path/to/file/parse-errors.log"
    }
  }
  else if "some-logs" in [tags] {
      elasticsearch {
        id => "some-logs"
        index => "some-logs"
        action => create
        hosts => ["https://linktoelastic:9200"]
        ssl => true
        ssl_certificate_verification => true
        cacert => '/pathtocert/certs/ca.crt'
        user => logstash_internal
        password => "${LS_PWD}"
        }
  }

As above, we have around 30 more of "else if".
Is there a way to simplify it and possibly increase performance?

If the only thing that needs to change is the index then you can do the if-else-if-else-if... in the filter section to set a field (perhaps [@metadata][index]) in the filter section and then use a single Elasticsearch output with

index => "%{[@metadata][index]}

This will result in a single connection to Elasticsearch instead of 30 and should be a little more efficient.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.