No logstash data showing up in kibana

I have just installed a new ELK stack instance but I cant figure out what I am missing. By default filebeats were being sent and that worked. I dont intend to use filebeats as since Im mainly collecting syslogs from network devices such as routers that wont have a filebeat client. I have built up logstash to accept UDP from the source with an output to Elasticsearch as well as to a file to verify the logstash piece is working. Data seems be getting indexed for logstash on elasticsearch but nothing shows up in the discover tab.

here is mylogstash config

Blockquote

Sample Logstash configuration for creating a simple

Beats -> Logstash -> Elasticsearch pipeline.

input {
tcp {
port => 5002
type => "cradlepoint"
}
udp {
port => 5002
type => "cradlepoint"
}

beats {

port => 5044

}

file {
path => "/var/log/syslog-ng"
start_position => "beginning"
type => "syslog"
}

}

filter {

if [type] == "syslog-ng"{
grok {
match => { "message" => "%{SYSLOGLINE}"}
}
date {
match => [ "timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}

else if [type] == "syslog" {

grok {

match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:[%{POSINT:syslog_pid}])?: %{GREEDYDATA:syslog_message}" }

add_field => [ "received_at", "%{@timestamp}" ]

add_field => [ "received_from", "%{host}" ]

}

date {

match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]

}

}

#}

output {

#stdout { codec => rubydebug}

elasticsearch {
hosts => "localhost:9200"
manage_template => false

index => "syslog-ng%{+YYY.MM.dd}"

index => "logstash-%{+YYYY.MM.dd}"

index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"

index => "%{[@metadata][index]}"

document_type => "system_logs"

}

file { path => "/tmp/logstash.log"}

}

Blockquote

Hi,

Are you able to see the data with a GET logstash-*/_search (you can use the Dev Tools for this).

Did you create an index pattern in Kibana for the logstash-* indexes?

Best regards
Wolfram

When running the GET logstash-/_search I do see a few logs there but only like 4. And yes I have created the logstash- index and have attached a screenshot.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.