Trouble getting topbeat data into elasticsearch via logstash

Elastic Stack Beats
1

Newbie here so bash lightly :slight_smile:

I have a host server with ELK installed and able to view filebeat data sent from a client server to logstash on the host.

I want to also view topbeat data from that same client in kibana on the host and am not able to.

From the client I enter this:

# curl -XPUT 'http://10.24.7.157:9200/_template/topbeat' -d@/etc/topbeat/topbeat.template.json

And get the response of:
{"acknowledged":true}

10.24.7.157 is the ELK host

My /etc/topbeat/topbeat.yml file has:

output:
  #elasticsearch:
    #hosts: ["localhost:9200"]
  logstash:
    hosts: ["10.24.7.157:5044"]

 curl -XGET 'http://10.24.7.157:9200/topbeat-*/_search?pretty'
{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 0,
    "successful" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 0,
    "max_score" : 0.0,
    "hits" : [ ]
  }
}

On the client I started topbeat and confirmed it's running:

# service topbeat status
 * topbeat is running

I then test to see if there is topbeat data in elasticsearch on the host and nothing:

# curl -XGET 'http://10.24.7.157:9200/topbeat-*/_search?pretty'
{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 0,
    "successful" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 0,
    "max_score" : 0.0,
    "hits" : [ ]
  }
}

Both host and client are Ubuntu 14.04

Host:
elasticsearch 1.7.5
logstash 1:1.5.6-1
Kibana 4.1.1

Client:
topbeat 1.1.2

2

Is Logstash listening for events and receiving them correctly? Is it able to send events to Elasticsearch? Check the logs of both Topbeat and Logstash.

3

Thanks Magnus. Yes I am able to see filebeat data from that same client in Kibana so filebeat on the client is able to send to logstash on the host so I know that part is working correctly.

I have logs now set to debug for topbeat on the client and I see:

2016-03-22T16:39:52Z DBG output worker: publish 96 events
2016-03-22T16:39:52Z DBG connect
2016-03-22T16:39:52Z DBG Try to publish 96 events to logstash with window size 1
2016-03-22T16:39:52Z DBG close connection
2016-03-22T16:39:52Z DBG 0 events out of 96 events sent to logstash. Continue sending ...
2016-03-22T16:39:52Z INFO Error publishing events (retrying): read tcp 10.24.7.155:54954->10.24.7.157:5044: read: connection reset by peer
2016-03-22T16:39:52Z INFO send fail
2016-03-22T16:39:52Z INFO backoff retry: 1s
2016-03-22T16:39:53Z DBG connect
2016-03-22T16:39:53Z DBG Try to publish 96 events to logstash with window size 1
2016-03-22T16:39:53Z DBG close connection
2016-03-22T16:39:53Z DBG 0 events out of 96 events sent to logstash. Continue sending ...
2016-03-22T16:39:53Z INFO Error publishing events (retrying): EOF
2016-03-22T16:39:53Z INFO send fail
2016-03-22T16:39:53Z INFO backoff retry: 2s
2016-03-22T16:39:55Z DBG connect
2016-03-22T16:39:55Z DBG Try to publish 96 events to logstash with window size 1
2016-03-22T16:39:55Z DBG close connection
2016-03-22T16:39:55Z DBG 0 events out of 96 events sent to logstash. Continue sending ...
2016-03-22T16:39:55Z INFO Error publishing events (retrying): EOF
2016-03-22T16:39:55Z INFO send fail
2016-03-22T16:39:55Z INFO backoff retry: 4s

I see this connection established:
root@u14:~# netstat -nat |grep 10.24.7.155
tcp 0 0 10.24.7.155:34823 10.24.7.157:5044 ESTABLISHED

4

And what does the beats input in your Logstash configuration look like?

5

On the host:

cat /etc/logstash/conf.d/02-beats-input.conf

input {
beats {
port => 5044
ssl => true
ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
}
}

6

You have configured SSL on the Logstash side but not on the Topbeat side.

1 Like
7

That was it! Thank you Magnus :slight_smile: