Wrong indexes with syslog

Elastic Stack Logstash
1

I have configured Logstash (see below configuration) to get logs from Cisco devices with syslog. Despite I have configured in "output" section index (index => "network-%{+YYYY.MM.dd}") on Kibana side I see wrong index. It shows %{[@metadata][beat]}-%{[@metadata][version]} instead of index (screenshot attached).

P.S with Winlogbeat and Filebeat everything ok.

Can anybody help to fix this issue?

# INPUT - Logstash listens on port 8514 for these logs.
#

input {
  syslog {
    port => "8514"
    type => "syslog"
  }
}


filter {
if [type] == "syslog" {
grok {
patterns_dir => [ "/opt/logstash/patterns" ]
match => [ 
"message", "%{SYSLOG5424PRI}%{NUMBER:log_sequence#}: %{CISCOTIMESTAMP:log_date}: %%{CISCO_REASON:facility}-%{INT:severity_level}-%{CISCO_REASON:facility_mnemonic}: %{GREEDYDATA:message}", 
"message", "%{SYSLOG5424PRI}%{NUMBER:log_sequence#}: %{CISCOTIMESTAMP:log_date}: %%{CISCO_REASON:facility}-%{CISCO_REASON:facility_sub}-%{INT:severity_level}-%{CISCO_REASON:facility_mnemonic}: %{GREEDYDATA:message}" 
]
overwrite => [ "message" ]
remove_field => [ "syslog5424_pri", "@version" ]
}
mutate {
gsub => [
"severity_level", "0", "0 - Emergency",
"severity_level", "1", "1 - Alert",
"severity_level", "2", "2 - Critical",
"severity_level", "3", "3 - Error",
"severity_level", "4", "4 - Warning",
"severity_level", "5", "5 - Notification",
"severity_level", "6", "6 - Informational"
]
}
}
}
}


output {
  # Something went wrong with the grok parsing, don't discard the messages though
  if "_grokparsefailure" in [tags] {
    file {
      path => "/tmp/fail-%{type}-%{+YYYY.MM.dd}.log"
    }
  }

  # The message was parsed correctly, and should be sent to elasicsearch.
  if "cisco" in [tags] {
    #file {
    #  path => "/tmp/%{type}-%{+YYYY.MM.dd}.log"
    #}

    elasticsearch {
      hosts           => "localhost:9200"
      manage_template => false
      index           => "network-%{+YYYY.MM.dd}"
#      document_id     => "%{fingerprint}"
    }
  }
}
2

When I delete "filter" section I can see "network-*" index on Kibana. But it shows logs from winlogbeat. Any ideas?

3

Do you have multiple configuration files in path.config?

4

Yes I have 3 files there (/etc/logstash/conf.d).

5

OK. Those three files are combined into one configuration. Events are read from all of the inputs, sent through all of the filters, and, unless there are conditionals, sent to all of the outputs.

If you want each configuration to be standalone you would have to configure them as pipelines.

6

I did it like this but it also does not help. Any other ideas?

# This file is where you define your pipelines. You can define multiple.
# For more information on multiple pipelines, see the documentation:
#   https://www.elastic.co/guide/en/logstash/current/multiple-pipelines.html

- pipeline.id: main
  path.config: "/etc/logstash/conf.d/*.conf"

- pipeline.id: cisco
  path.config: "/etc/logstash/conf.d/cisco.conf"

- pipeline.id: vmware
  path.config: "/etc/logstash/conf.d/vmware.conf"

- pipeline.id: windows
  path.config: "/etc/logstash/conf.d/windows.conf"
7

That is telling it to combine all three files and run them as a single pipeline. Remove those two lines.

1 Like
8

Thanks a lot. It works now.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.