Logstash is Unable to Emit Events after Setting Up X-Pack Security

Running 6.3.2 and everything seems to be working with security, except that I am no longer receiving events from Logstash. I verified from my machines running Filebeat that they are able to send logs out to Logstash, but it appears that Logstash is holding on to them. I feel like there is a configuration option somewhere that i'm missing out on.. I have added the kibana user information to kibana.yml, but is there something in particular that I need to add to logstash.yml that relates to the new users I created within X-Pack Security?

I haven't seen anything specified in general within the guides that i've been looking through; maybe i'm blind.

Here is what i'm seeing within Monitoring:
logstashnotemitting

https://www.elastic.co/guide/en/logstash/6.3/ls-security.html

Issue still persists.
I followed the steps under 'Configuring Logstash to use Basic Authentication' within that guide and still none of the logs are emitting. I created the user and added the information to my logstash.conf file.

Here is my current conf:

filter {
  elasticsearch {
  if [type] == "syslog" {
  mutate {
 gsub => ["message",".auth_user_code.=>.\d+.", "auth_user_code=XXXX"]
   }
grok {
  match => { "message" => "%{SYSLOG5424SD:Time}%{SYSLOG5424SD:Application}%{SYSLOG5424SD:Process}%{SYSLOG5424SD:RemoveMe1}%{SYSLOG5424SD:RemoveMe2}%{DATA:RemoveMe3}%{SYSLOG5424S$
  remove_field => [ "RemoveMe1" ]
  remove_field => [ "RemoveMe2" ]
  remove_field => [ "RemoveMe3" ]
  add_field => [ "received_at", "%{@timestamp}" ]
  add_field => [ "received_from", "%{host}" ]
}
syslog_pri { }
date {
  match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
  }else if [type] == "nginx" {
 grok {
   match => { "message" => "%{COMBINEDAPACHELOG}%{GREEDYDATA}" }
 }
 geoip {
   source => "clientip"
 }
  }else if [type] == "eventlog" {
 grok {
   match => { "message" => "%{SYSLOG5424SD:Time}%{SYSLOG5424SD:RemoveMe1}%{SYSLOG5424SD:RemoveMe2}%{SYSLOG5424SD:Event_Type}%{SYSLOG5424SD:Event_UUID}%{GREEDYDATA:Event_Message}$
   remove_field => [ "RemoveMe1" ]
   remove_field => [ "RemoveMe2" ]
   add_field => [ "received_at", "%{@timestamp}" ]
   add_field => [ "received_from", "%{host}" ]
 }
   }else if [type] == "dealdirlog" {
  grok {
    match => { "message" => "%{SYSLOG5424SD:Time}%{GREEDYDATA:DealDir_Message}" }
  }
}
user => logstash_internal
password => logstash
  }
}

I'm not a Logstash expert, but you've provided a configuration with a filter but no output - where is the section where you actually send these log messages to ES?

I have another file, called 30-elasticsearch-output.conf, within the same directory (/etc/logstash/conf.d/).
It contains this:

output {
  elasticsearch {
hosts => ["localhost:9200"]
user => logstash
password => logstash
sniffing => true
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
  }
}

In monitoring, I can see my number of documents go up, but i'm not seeing anything inside of the Discover menu..

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.