Logstash data appears but filebeat data not appearing in kibana after x-pack is enabled

Elastic Stack Beats
1

I have configured filebeat to send logs to logstash followed by logstash sending logs to elasticsearch.
All worked fine .. i could see logstash-* and filebeat-* data getting updated in kibana .. until i enabled x-pack security.
After giving user creds in logtash.conf, i can see the logstash data appear but none appears in filebeat.

My problem looks similar to Kibana not updating indices after X-Pack installed - #2 by TimV

so i followed the steps mentioned in the article link provided in the topic - Beats and Security | X-Pack for the Elastic Stack [6.2] | Elastic

But that did not help.

Also, following Logstash Reference [8.11] | Elastic , placing username and password in input and filter fields results in config errors for me.

Any suggestions?

filebeat.yml (cannot see filebeat-* data in kibana)

----------------------------- Logstash output --------------------------------
output.logstash:

The Logstash hosts

hosts: ["localhost:5044"]
index: "filebeat"
username: "filebeat_internal"
password: "changeme"

logstash.conf (can see logstash-* data in kibana but in syslog format instead of json :pensive:

input {
beats {
port => 5044
codec => json
}
}
filter {
grok {
match => { "message" => "%{COMBINEDAPACHELOG}"}
}
}
output {
elasticsearch {
hosts => "localhost:9200"
user => "elastic"
password => "changeme"
}
}

  • though elastic being a superuser does not make filebeat work either.
2

Those instructions are for having Beats send data to Elasticsearch.

Since you are sending data from Beats to Logstash, you shouldn't be using those instructions.

There is no need for a username/password for passing data from beats to logstash.

3

@TimV : okay.. so in that case why is my filebeat-* data (without user/password also) not appearing while the logstash data appears in kibana ?

4

With your filebeat and logstash configuration, I'd expect filebeat send data to logstash (unencrypted, as no TLS/SSL is configured between beats and LS here). As you have not configured the index in the elasticsearch output, all events received from filebeat will be put into the logstash index. See filebeat getting started guide + follow link to Logstash Setup.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.