Example Dashboards - Via Logstash

Elastic Stack Beats
1

Hello, I have been reading through various threads in regards to the example dashboards showing "no data found" when logs are being pushed via logstash instead of directly to elasticsearch which begs the question if all beats push to elasticsearch what is the point of logstash in the stack?

Anyway, is there a way to get these fields working if I change some configuration?

Here is my filebeats config:

- input_type: log
  paths:
    - /var/log/syslog
  document_type: syslog

- input_type: log
  paths:
    - /var/log/*.log
  document_type: generic_log

- input_type: log
  paths:
    - /var/log/kafka/*.log

And below logstash:

output {
  elasticsearch {
    hosts => ["xxx:9200"]
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
    document_type => "%{[@metadata][type]}"
    user => xxx
    password => xxx
  }
}

I guess it may be a lack of understanding on my part, how the example dashboards has all these "fields" to it's disposable, as all I seem to have is "timestamp".

2

Which sample dashboards exactly do you mean? The filebeat ones are to be used with filebeat modules, as some parsing is required. The parsing can be either implemented in Logstash or Elasticsearch Ingest Node. As of now, filebeat modules indeed only work with Elasticsearch.

For filebeat modules via Logstash see this discussion. Especially This response and this one.

3

My apologies, I should have been more clear.

Yes my ultimate goal is to get the fields from the filebeat dashboards into elasticsearch so I can create similar visuals without having to rely on "timestamp"

I will take a look at the links you have provided.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.