Logstash is Unable to Emit Events after Setting Up X-Pack Security

Elastic Stack Logstash
1

Running 6.3.2 and everything seems to be working with security, except that I am no longer receiving events from Logstash. I verified from my machines running Filebeat that they are able to send logs out to Logstash, but it appears that Logstash is holding on to them. I feel like there is a configuration option somewhere that i'm missing out on.. I have added the kibana user information to kibana.yml, but is there something in particular that I need to add to logstash.yml that relates to the new users I created within X-Pack Security?

I haven't seen anything specified in general within the guides that i've been looking through; maybe i'm blind.

Here is what i'm seeing within Monitoring:
logstashnotemitting

2

https://www.elastic.co/guide/en/logstash/6.3/ls-security.html

3

Issue still persists.
I followed the steps under 'Configuring Logstash to use Basic Authentication' within that guide and still none of the logs are emitting. I created the user and added the information to my logstash.conf file.

Here is my current conf:

filter {
  elasticsearch {
  if [type] == "syslog" {
  mutate {
 gsub => ["message",".auth_user_code.=>.\d+.", "auth_user_code=XXXX"]
   }
grok {
  match => { "message" => "%{SYSLOG5424SD:Time}%{SYSLOG5424SD:Application}%{SYSLOG5424SD:Process}%{SYSLOG5424SD:RemoveMe1}%{SYSLOG5424SD:RemoveMe2}%{DATA:RemoveMe3}%{SYSLOG5424S$
  remove_field => [ "RemoveMe1" ]
  remove_field => [ "RemoveMe2" ]
  remove_field => [ "RemoveMe3" ]
  add_field => [ "received_at", "%{@timestamp}" ]
  add_field => [ "received_from", "%{host}" ]
}
syslog_pri { }
date {
  match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
  }else if [type] == "nginx" {
 grok {
   match => { "message" => "%{COMBINEDAPACHELOG}%{GREEDYDATA}" }
 }
 geoip {
   source => "clientip"
 }
  }else if [type] == "eventlog" {
 grok {
   match => { "message" => "%{SYSLOG5424SD:Time}%{SYSLOG5424SD:RemoveMe1}%{SYSLOG5424SD:RemoveMe2}%{SYSLOG5424SD:Event_Type}%{SYSLOG5424SD:Event_UUID}%{GREEDYDATA:Event_Message}$
   remove_field => [ "RemoveMe1" ]
   remove_field => [ "RemoveMe2" ]
   add_field => [ "received_at", "%{@timestamp}" ]
   add_field => [ "received_from", "%{host}" ]
 }
   }else if [type] == "dealdirlog" {
  grok {
    match => { "message" => "%{SYSLOG5424SD:Time}%{GREEDYDATA:DealDir_Message}" }
  }
}
user => logstash_internal
password => logstash
  }
}
4

I'm not a Logstash expert, but you've provided a configuration with a filter but no output - where is the section where you actually send these log messages to ES?

5

I have another file, called 30-elasticsearch-output.conf, within the same directory (/etc/logstash/conf.d/).
It contains this:

output {
  elasticsearch {
hosts => ["localhost:9200"]
user => logstash
password => logstash
sniffing => true
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
  }
}

In monitoring, I can see my number of documents go up, but i'm not seeing anything inside of the Discover menu..

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.