I think I solved my problem.
This post saved me: my logstash configuration had the same issue, where I was using if [type] == "cowrie" {
instead of the correct if [fields][document_type] == "cowrie"
.
Once I modified this and restarted logstash, I started seeing useful data in the logstash data.
Then, I want back to Kibana. I don't fully understand indexes to be honest, but from log messages, I know I needed to create one. So, I created an index pattern for logstash-*
(which is the name of my logstash logs). It found all the interesting fields of my logs (e.g timestamp, arch, data, geoip_cityname...). Then, in Discover, I can see my data
Note I still have MapperParsingExceptions in my /var/log/elasticsearch/elasticsearch.log
, and in logstash logs, I still have many warnings Could not index event to Elasticsearch.
. I'll look into that, but probably a different issue.