Sending data to to different Elasticsearch instances from Logstash

[Ryan_Downey]

I'm looking to send data to two different Elasticsearch instances. If pretty sure that each one of these options will work but many months ago I had a similar setup in a different environment and the way I had implemented the split created some odd issues with the data in each environment. I don't remember all of the details so I figured I'd just post this here and double-check.

Along with sending data to two ES instances, I would like to send metricbeat data from a specific host to the second ES instance. Is one of the options below better than the other? Nested IF vs two different IF's? If it even makes a difference.

Option 1, nested IF statement:
output {
  if [agent][type] == "metricbeat" {
    elasticsearch {
      hosts => ["https://abc.gov:9243"]
      manage_template => false
      index => "metricbeat-%{[agent][version]}"
      user => logstash_internal
      password => Pleasework18_li
      ssl => true
      cacert => "/etc/logstash/cert.pem"
      index => "metricbeat-%{[agent][version]}"
      ilm_rollover_alias => "metricbeat-%{[agent][version]}"
      ilm_pattern => "000001"
      ilm_policy => "metricbeat-%{[agent][version]}"
    }
    if [agent][type] == "metricbeat" and [host][hostname] =~ "(host1|host2|host3)" {
        elasticsearch {
          hosts => ["https://def.gov:9243"]
          manage_template => false
          index => "metricbeat-%{[agent][version]}"
	  user => logstash_internal
	  password => Pleasework18_li
	  ssl => true
	  cacert => "/etc/logstash/cert.pem"
          index => "metricbeat-%{[agent][version]}"
	  }
	}
  }

Option 2, two different IF statements:
output {
  if [agent][type] == "metricbeat" {
    elasticsearch {
      hosts => ["https://abc.gov:9243"]
      manage_template => false
      index => "metricbeat-%{[agent][version]}"
	  user => logstash_internal
	  password => Pleasework18_li
	  ssl => true
	  cacert => "/etc/logstash/cert.pem"
      index => "metricbeat-%{[agent][version]}"
      ilm_rollover_alias => "metricbeat-%{[agent][version]}"
      ilm_pattern => "000001"
      ilm_policy => "metricbeat-%{[agent][version]}"
    }
  }
  
  if [agent][type] == "metricbeat" and [host][hostname] =~ "(host1|host2|host3)" {
    elasticsearch {
      hosts => ["https://def.gov:9243"]
      manage_template => false
      index => "metricbeat-%{[agent][version]}"
	  user => logstash_internal
	  password => Pleasework18_li
	  ssl => true
	  cacert => "/etc/logstash/certt.pem"
      index => "metricbeat-%{[agent][version]}"
	  }
	}
  }

[ptamba]

i would think that in both config, events will get written to both indices when {[host][hostname] =~ "(host1|host2|host3) . this is because both conditions are evaluated as true.

if i follow your intention, i would go with

output {
 if [agent][type] == "metricbeat" {
   if [host][hostname] =~ "(host1|host2|host3)" {
      [es1]
    } 
   else { 
      [es2]
   }
 }
}

[Ryan_Downey]

Appreciate the quick response. I seem to have missed the fact that I do want the events to go to both ES instances, sorry about that. With your recommendation though I now see how I would split the data to one and not the other if needed.