# Logstash to Elastic search not working for heartbeat

**URL:** <https://discuss.elastic.co/t/logstash-to-elastic-search-not-working-for-heartbeat/86694>\
**Category:** Logstash\
**Created:** [May 22, 2017, 3:29pm UTC](https://discuss.elastic.co/t/logstash-to-elastic-search-not-working-for-heartbeat/86694 "2017-05-22T15:29:21Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![arun\_prasath1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/arun_prasath1/32/49539_2.png) [@arun\_prasath1](https://discuss.elastic.co/u/arun_prasath1)\
**Post date:** [May 22, 2017, 3:29pm UTC](https://discuss.elastic.co/t/logstash-to-elastic-search-not-working-for-heartbeat/86694/1 "2017-05-22T15:29:21Z")

</div>

In the ELK pipeline, I have Heartbeat pushing logs to Logstash and then to Elasticsearch. The heartbeat to logstash works fine and from logstash to elasticsearh have issues. I have done my best using the debug options in logstash but could not figure the issue. From the logs I am sure its a logstash get the URL from heartbeat and it is not pushing to ES.  
I see the below lines in logstash logs, so LS has logs from Heartbeat.

```
 _RUBY.start_input(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:325)_
_java.lang.Thread.run(java/lang/Thread.java:745)_
_[2017-05-22T14:25:17,518][DEBUG][logstash.pipeline] filter received {"event"=>{"tls_handshake_rtt"=>{"us"=>451771}, "scheme"=>"https", "ip"=>"162.24.10.21", "tcp_connect_rtt"=>{"us"=>220262}, "monitor"=>"http@https://myapplication-url.mydomain.com", "type"=>"http", "http_rtt"=>{"us"=>255383}, "url"=>"https://myapplication-url.mydomain.com", "tags"=>["beats_input_raw_event"], "duration"=>{"us"=>993180}, "rtt"=>{"us"=>927568}, "@timestamp"=>2017-05-22T12:23:20.834Z, "port"=>443, "resolve_rtt"=>{"us"=>65529}, "response"=>{"status"=>200}, "host"=>"myapplication-url.mydomain.com", "beat"=>{"hostname"=>"build-server.domain.mydomain.com", "name"=> "build-server.domain.mydomain.com", "version"=>"5.4.0"}, "@version"=>"1", "up"=>true}}_
_[2017-05-22T14:25:22,520][DEBUG][logstash.pipeline] output received {"event"=>{"tls_handshake_rtt"=>{"us"=>415247}, "scheme"=>"https", "ip"=>"162.24.10.21", "tcp_connect_rtt"=>{"us"=>211131}, "monitor"=>"http@https://myapplication-url.mydomain.com", "type"=>"http", "http_rtt"=>{"us"=>205383}, "url"=>"https://myapplication-url.mydomain.com", "tags"=>["beats_input_raw_event"], "duration"=>{"us"=>888011}, "rtt"=>{"us"=>831959}, "@timestamp"=>2017-05-22T12:25:20.834Z, "port"=>443, "resolve_rtt"=>{"us"=>55934}, "response"=>{"status"=>200}, "beat"=>{"hostname"=>"build-server.domain.mydomain.com", "name"=>"build-server.domain.mydomain.com", "version"=>"5.4.0"}, "host"=>"myapplication-url.mydomain.com", "@version"=>"1", "up"=>true}}_

```

It is not creating the heartbeat-logstash- ndex (as mentioned in output filter) and even this hearbeat index name is not logged anywhere in the log file. I have the below .yml file in the elk server.

```
input {
  beats {
    port => 5044
  }
}
input {
  heartbeat {
    add_field => {
      "role" => "logstash_shipper"
    }
    interval => "10"
    type => "http"
  }
}

## some filter statements to process other filebeat from other servers

output {
  if [type] == "heartbeat" {
   elasticsearch {
   hosts => ["myelk-server.com:9200"]
    manage_template => false
    index => "heartbeat-%{+YYYY.MM.dd}"
  }
  } else
  if [type] == "filebeat" {
  elasticsearch {
    hosts => ["myelk-server.com:9200"]
    manage_template => false
    index => "filebeat-logstash-%{+YYYY.MM.dd}"
  }
 }
}

```

Please help figure out what I am missing. However, other beat indexes like filebeat, metricbeat are created in the ES without any issues.

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [May 22, 2017, 4:57pm UTC](https://discuss.elastic.co/t/logstash-to-elastic-search-not-working-for-heartbeat/86694/2 "2017-05-22T16:57:54Z")

</div>

I'm confused. Where does the Logstash heartbeat plugin come into this? It does nothing more nor less than inject a message into the Logstash pipeline. It does _not_ listen to nor receive any information from the heartbeat beat. Everything that the heartbeat beat sends comes through the regular beats input.

It seems that your line:

```auto
if [type] == "heartbeat" {

```

conflicts with

```auto
"type"=>"http"

```

from the two sample log lines. This might be why nothing is going into the heartbeat-YYYY.MM.dd index.

---

<div class="post-metadata">

**Author:** ![arun\_prasath1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/arun_prasath1/32/49539_2.png) [@arun\_prasath1](https://discuss.elastic.co/u/arun_prasath1)\
**Post date:** [May 22, 2017, 7:58pm UTC](https://discuss.elastic.co/t/logstash-to-elastic-search-not-working-for-heartbeat/86694/3 "2017-05-22T19:58:37Z")

</div>

it worked now like a charm. you saved me Aaron. 🙂 This is problem for beginner like me.

---

<div class="post-metadata">

**Author:** ![arun\_prasath1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/arun_prasath1/32/49539_2.png) [@arun\_prasath1](https://discuss.elastic.co/u/arun_prasath1)\
**Post date:** [May 25, 2017, 2:42pm UTC](https://discuss.elastic.co/t/logstash-to-elastic-search-not-working-for-heartbeat/86694/4 "2017-05-25T14:42:04Z")

</div>

I have the data now in Kibana but there is no return status (response.status) of the URL. I have created a separate post, could you help me with the new issue if i have some wrong code.  
the new issue is at [Heartbeat - how to send respone.status field to kibana to get URL status](https://discuss.elastic.co/t/heartbeat-how-to-send-respone-status-field-to-kibana-to-get-url-status/87140)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 22, 2017, 2:42pm UTC](https://discuss.elastic.co/t/logstash-to-elastic-search-not-working-for-heartbeat/86694/5 "2017-06-22T14:42:30Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.