# Trouble getting topbeat data into elasticsearch via logstash

**URL:** <https://discuss.elastic.co/t/trouble-getting-topbeat-data-into-elasticsearch-via-logstash/45062>\
**Category:** Beats\
**Created:** [March 22, 2016, 3:23am UTC](https://discuss.elastic.co/t/trouble-getting-topbeat-data-into-elasticsearch-via-logstash/45062 "2016-03-22T03:23:08Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![velvetpixel](https://avatars.discourse-cdn.com/v4/letter/v/e68b1a/32.png) [@velvetpixel](https://discuss.elastic.co/u/velvetpixel)\
**Post date:** [March 22, 2016, 3:23am UTC](https://discuss.elastic.co/t/trouble-getting-topbeat-data-into-elasticsearch-via-logstash/45062/1 "2016-03-22T03:23:08Z")

</div>

Newbie here so bash lightly 🙂

I have a host server with ELK installed and able to view filebeat data sent from a client server to logstash on the host.

I want to also view topbeat data from that same client in kibana on the host and am not able to.

From the client I enter this:

`# curl -XPUT 'http://10.24.7.157:9200/_template/topbeat' -d@/etc/topbeat/topbeat.template.json`

And get the response of:  
`{"acknowledged":true}`

10.24.7.157 is the ELK host

My /etc/topbeat/topbeat.yml file has:

```auto
output:
  #elasticsearch:
    #hosts: ["localhost:9200"]
  logstash:
    hosts: ["10.24.7.157:5044"]

```

```auto
 curl -XGET 'http://10.24.7.157:9200/topbeat-*/_search?pretty'
{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 0,
    "successful" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 0,
    "max_score" : 0.0,
    "hits" : []
  }
}

```

On the client I started topbeat and confirmed it's running:

```auto
# service topbeat status
 * topbeat is running

```

I then test to see if there is topbeat data in elasticsearch on the host and nothing:

```auto
# curl -XGET 'http://10.24.7.157:9200/topbeat-*/_search?pretty'
{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 0,
    "successful" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 0,
    "max_score" : 0.0,
    "hits" : []
  }
}

```

Both host and client are Ubuntu 14.04

Host:  
elasticsearch 1.7.5  
logstash 1:1.5.6-1  
Kibana 4.1.1

Client:  
topbeat 1.1.2

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [March 22, 2016, 6:45am UTC](https://discuss.elastic.co/t/trouble-getting-topbeat-data-into-elasticsearch-via-logstash/45062/2 "2016-03-22T06:45:11Z")

</div>

Is Logstash listening for events and receiving them correctly? Is it able to send events to Elasticsearch? Check the logs of both Topbeat and Logstash.

---

<div class="post-metadata">

**Author:** ![velvetpixel](https://avatars.discourse-cdn.com/v4/letter/v/e68b1a/32.png) [@velvetpixel](https://discuss.elastic.co/u/velvetpixel)\
**Post date:** [March 22, 2016, 4:50pm UTC](https://discuss.elastic.co/t/trouble-getting-topbeat-data-into-elasticsearch-via-logstash/45062/3 "2016-03-22T16:50:23Z")

</div>

Thanks Magnus. Yes I am able to see filebeat data from that same client in Kibana so filebeat on the client is able to send to logstash on the host so I know that part is working correctly.

I have logs now set to debug for topbeat on the client and I see:

2016-03-22T16:39:52Z DBG output worker: publish 96 events  
2016-03-22T16:39:52Z DBG connect  
2016-03-22T16:39:52Z DBG Try to publish 96 events to logstash with window size 1  
2016-03-22T16:39:52Z DBG close connection  
2016-03-22T16:39:52Z DBG 0 events out of 96 events sent to logstash. Continue sending ...  
2016-03-22T16:39:52Z INFO Error publishing events (retrying): read tcp 10.24.7.155:54954-\>10.24.7.157:5044: read: connection reset by peer  
2016-03-22T16:39:52Z INFO send fail  
2016-03-22T16:39:52Z INFO backoff retry: 1s  
2016-03-22T16:39:53Z DBG connect  
2016-03-22T16:39:53Z DBG Try to publish 96 events to logstash with window size 1  
2016-03-22T16:39:53Z DBG close connection  
2016-03-22T16:39:53Z DBG 0 events out of 96 events sent to logstash. Continue sending ...  
2016-03-22T16:39:53Z INFO Error publishing events (retrying): EOF  
2016-03-22T16:39:53Z INFO send fail  
2016-03-22T16:39:53Z INFO backoff retry: 2s  
2016-03-22T16:39:55Z DBG connect  
2016-03-22T16:39:55Z DBG Try to publish 96 events to logstash with window size 1  
2016-03-22T16:39:55Z DBG close connection  
2016-03-22T16:39:55Z DBG 0 events out of 96 events sent to logstash. Continue sending ...  
2016-03-22T16:39:55Z INFO Error publishing events (retrying): EOF  
2016-03-22T16:39:55Z INFO send fail  
2016-03-22T16:39:55Z INFO backoff retry: 4s

I see this connection established:  
root@u14:~# netstat -nat |grep 10.24.7.155  
tcp 0 0 10.24.7.155:34823 10.24.7.157:5044 ESTABLISHED

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [March 22, 2016, 6:00pm UTC](https://discuss.elastic.co/t/trouble-getting-topbeat-data-into-elasticsearch-via-logstash/45062/4 "2016-03-22T18:00:51Z")

</div>

And what does the beats input in your Logstash configuration look like?

---

<div class="post-metadata">

**Author:** ![velvetpixel](https://avatars.discourse-cdn.com/v4/letter/v/e68b1a/32.png) [@velvetpixel](https://discuss.elastic.co/u/velvetpixel)\
**Post date:** [March 22, 2016, 6:32pm UTC](https://discuss.elastic.co/t/trouble-getting-topbeat-data-into-elasticsearch-via-logstash/45062/5 "2016-03-22T18:32:08Z")

</div>

On the host:

# cat /etc/logstash/conf.d/02-beats-input.conf

input {  
beats {  
port =\> 5044  
ssl =\> true  
ssl\_certificate =\> "/etc/pki/tls/certs/logstash-forwarder.crt"  
ssl\_key =\> "/etc/pki/tls/private/logstash-forwarder.key"  
}  
}

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [March 22, 2016, 8:40pm UTC](https://discuss.elastic.co/t/trouble-getting-topbeat-data-into-elasticsearch-via-logstash/45062/6 "2016-03-22T20:40:16Z")

</div>

You have configured SSL on the Logstash side but not on the Topbeat side.

---

<div class="post-metadata">

**Author:** ![velvetpixel](https://avatars.discourse-cdn.com/v4/letter/v/e68b1a/32.png) [@velvetpixel](https://discuss.elastic.co/u/velvetpixel)\
**Post date:** [March 24, 2016, 4:03pm UTC](https://discuss.elastic.co/t/trouble-getting-topbeat-data-into-elasticsearch-via-logstash/45062/7 "2016-03-24T16:03:48Z")

</div>

That was it! Thank you Magnus 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 9:54pm UTC](https://discuss.elastic.co/t/trouble-getting-topbeat-data-into-elasticsearch-via-logstash/45062/8 "2017-07-05T21:54:12Z")

</div>


