Ready to throw logstash config file out the window -- help please :)

Elastic Stack Logstash
1

I have the below file and I'm getting the completely useless error message:
Error: Expected one of #, => at line 85, column 5 (byte 1787) after output {
elasticsearch {
if
____ File below:
input {
beats {
port => 5044
ssl => true
ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
}
}
input {
udp {
host => "0.0.0.0"
port => 2055
codec => netflow { versions => [ 5, 9 ] }
type => "netflow"
}
udp {
host => "0.0.0.0"
port => 9995
codec => netflow { versions => [ 5, 9 ] }
type => "netflow"
}

    udp {
            host => "0.0.0.0"
            port => 1514
            type => "syslog-relay"
            tags => [ "netsyslog" ]
     }
    tcp {
            host => "0.0.0.0"
            port => 1514
            type => "syslog-relay"
            tags => [ "netsyslog" ]
     }

}

filter {

if [type] == "syslog-relay" {

grok { match => { "message" => "(?:<%{INT:priority}>)?%{SYSLOGBASE2} (?:\s?%{LOGLEVEL:log_level} )?(?:\s?%{WORD:log_format}: )?%{GREEDYDATA:syslog_message}" }}
date { match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ] }
syslog_pri { }

if ("_grokparsefailure" in [tags]) { 
    mutate { replace => ["@message", "TOPARSE: %{message}"] }
} else if [log_format] == "json" {
  mutate {
    gsub => ["syslog_message", "@timestamp", "syslog_timestamp"]
  }

  json {
    source => "syslog_message"
  }

  mutate {
    replace => ["@message", "%{message}"]
  }

} else {
  mutate {
    replace => [ "@message", "%{syslog_message}" ]
  }

}

mutate {
  remove_field => [
    "syslog_hostname",  "syslog_message", "syslog_timestamp",
    "syslog_severity_code",  "syslog_facility_code",
    "message" #facility_label", "severity_label"
  ]

}

}
}
-> part it's complaining about
output {
elasticsearch {
hosts => "localhost:9200"
if [type] == "syslog-relay" {
index => "logstash-%{+YYYY.MM.dd}"
} else if [type] == "netflow" {
index => "flowstash-%{+YYYY.MM.dd}"
}
}

}

2

You can not have conditionals within a filter block like you have for the Elasticsearch output. You will need to configure 2 completely separate Elasticsearch output blocks and select between these based on conditionals.

3

Thank you so much for the reply.

I was starting to think that may be the case.

So i just move the if's outside like so:
if [type] == "syslog-relay" {
output {
elasticsearch {
hosts => "localhost:9200"
index => "logstash-%{+YYYY.MM.dd}"
}
} else if [type] == "netflow" {
output {
elasticsearch {
hosts => "localhost:9200"
index => "flowstash-%{+YYYY.MM.dd}"
}
}

?

4

Either I'm doing it completely wrong or it still doesn't like it =(
Error: Expected one of #, input, filter, output at line 83, column 9 (byte 1766) after
if [type] == "syslog-relay" {
output {
->83 elasticsearch {
hosts => "localhost:9200"
index => "logstash-%{+YYYY.MM.dd}"
}
} else if [type] == "netflow" {
output {
elasticsearch {
hosts => "localhost:9200"
index => "flowstash-%{+YYYY.MM.dd}"
}
}

5

Pretty sure I just got it.
Had the conditionals in the wrong place If this fixed it thanks for the pointers!

6

Excellent. It should look something like this:

output {
  if [type] == "syslog-relay" {
    elasticsearch {
      hosts => "localhost:9200"
      index => "logstash-%{+YYYY.MM.dd}"
    }
  } else if [type] == "netflow" {
    elasticsearch {
      hosts => "localhost:9200"
      index => "flowstash-%{+YYYY.MM.dd}"
    }
  }
}
7

The below is what I had to do to get it to work -- the way you just suggested was the first thing I tried and it was no love -- this finally worked but I think yours makes more sense. Maybe I had a typo and got frustrated too quickly.

output {
        if [type] == "syslog-relay" {
                        elasticsearch {
                                hosts => "localhost:9200"
                                index => "logstash-%{+YYYY.MM.dd}"
                        }
                }
        }
output {
                if [type] == "netflow" {
                                elasticsearch {
                                        hosts => "localhost:9200"
                                        index => "flowstash-%{+YYYY.MM.dd}"
                                }
                        }