# Logstash not sending to elasticsearch **URL:** \ **Category:** Logstash\ **Created:** [July 12, 2016, 10:46pm UTC](https://discuss.elastic.co/t/logstash-not-sending-to-elasticsearch/55357 "2016-07-12T22:46:39Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![Sergio\_Pavez](https://avatars.discourse-cdn.com/v4/letter/s/c68b51/32.png) [@Sergio\_Pavez](https://discuss.elastic.co/u/Sergio_Pavez)\ **Post date:** [July 12, 2016, 10:46pm UTC](https://discuss.elastic.co/t/logstash-not-sending-to-elasticsearch/55357/1 "2016-07-12T22:46:39Z")
Hello. I have a logstash configuration with a custom template. The template appears when I ask for it: "GET /\_template/\*". I followed this example: [Logstash mapping](http://www.pipebug.com/elasticsearch-logstash-kibana-4-mapping-4.html) If I send the data to a file with json format this is what I get: > { > "@version": "1", > "@timestamp": "2016-07-12T22:35:03.423Z", > "date": "20160704", > "time": "105522.383935", > "thread": "13340", > "uuid": "18d1ccf3-9d2d-4602-bdb7-52a84351a57d", > "resource\_id\_module": "S2", > "method": "tryDeliver", > "file": "XMLdispatcher.cpp.532", > "severity": "info", > "interpreter": "InterpreterTimestamp", > "execute\_time": "0", > "id\_log": "sigas-control.log", > "received\_from": "zero" > } My template looks like this: ``` { "template" : "sigas-control.log", "settings" : { "index.refresh_interval" : "60s" }, "mappings" : { "_default_" : { "_all" : { "enabled" : false }, "dynamic_templates" : [{ "message_field" : { "match" : "message", "match_mapping_type" : "string", "mapping" : { "type" : "string", "index" : "not_analyzed" } } }, { "string_fields" : { "match" : "*", "match_mapping_type" : "string", "mapping" : { "type" : "string", "index" : "not_analyzed" } } }], "properties" : { "@timestamp" : { "type" : "date", "format" : "dateOptionalTime" }, "@version" : { "type" : "integer", "index" : "not_analyzed" }, "thread": { "type" : "integer", "index" : "not_analyzed" }, "uuid": { "type" : "string", "index" : "not_analyzed" }, "resource_id_module": { "type" : "string", "index" : "not_analyzed" }, "file": { "type" : "string", "index" : "not_analyzed" }, "severity": { "type" : "string", "index" : "not_analyzed" }, "log_message": { "type" : "string", "index" : "not_analyzed" }, "interpreter": { "type" : "string", "index" : "not_analyzed" }, "execute_time": { "type" : "integer", "index" : "not_analyzed" }, "date": { "type" : "date", "format" : "yyyyMMdd"}, "time": { "type" : "date", "format" : "HHmmss" }, "id_log" : { "type" : "string", "index" : "not_analyzed" }, "received_from" : { "type" : "string", "index" : "not_analyzed" } } } } } ``` What is that I'm doing wrong? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [July 13, 2016, 5:42am UTC](https://discuss.elastic.co/t/logstash-not-sending-to-elasticsearch/55357/2 "2016-07-13T05:42:35Z")
What's the problem? Is Logstash not sending anything to ES? Or do the events end up with the wrong mappings? What does your Logstash configuration look like? ---
**Author:** ![Sergio\_Pavez](https://avatars.discourse-cdn.com/v4/letter/s/c68b51/32.png) [@Sergio\_Pavez](https://discuss.elastic.co/u/Sergio_Pavez)\ **Post date:** [July 13, 2016, 1:25pm UTC](https://discuss.elastic.co/t/logstash-not-sending-to-elasticsearch/55357/3 "2016-07-13T13:25:13Z")
The mapping is created, but Logstash is not sending data to ES. This is my Logstash conf: ``` input { file { path => "/home/spavez/Desktop/sigas.log" start_position => beginning ignore_older => 0 } } filter { grok { patterns_dir => ["/etc/logstash/conf.d/patterns"] match => { "message" => "%{DATE2:date}\|%{TIME2:time}\|%{NUMBER:thread}\|%{UUID2:uuid}-%{MODULERES:resource_id_module}\|%{WORD:method}\|%{JAVACLASS:file}\|%{LOGLEVEL:severity}\|%{WORD:interpreter} %{WORD:method2} %{NUMBER:execute_time}" } add_field => ["id_log", "sigas-control.log"] add_field => ["received_from", "%{host}"] remove_field => ["method2", "host", "message", "path"] } } output { if [id_log] == "sigas-control.log" { elasticsearch { index => "sigas-control.log" template => "/etc/logstash/sigas-control.log.json" template_name => "sigas-control.log" template_overwrite => true } stdout { codec => "json" } file { codec => "json" path => "/tmp/debug-filters.json" } } } ``` ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [July 13, 2016, 1:28pm UTC](https://discuss.elastic.co/t/logstash-not-sending-to-elasticsearch/55357/4 "2016-07-13T13:28:52Z")
Is Logstash even reading anything from the file or is it waiting for more data to be appended, i.e. is its sincedb position pointing to the end of the file. ---
**Author:** ![Sergio\_Pavez](https://avatars.discourse-cdn.com/v4/letter/s/c68b51/32.png) [@Sergio\_Pavez](https://discuss.elastic.co/u/Sergio_Pavez)\ **Post date:** [July 13, 2016, 1:31pm UTC](https://discuss.elastic.co/t/logstash-not-sending-to-elasticsearch/55357/5 "2016-07-13T13:31:38Z")
It's waiting for more data to be appended. I need to add new lines to the file in order for logstash to do anything. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [July 13, 2016, 1:38pm UTC](https://discuss.elastic.co/t/logstash-not-sending-to-elasticsearch/55357/6 "2016-07-13T13:38:33Z")
So.... problem solved then? ---
**Author:** ![Sergio\_Pavez](https://avatars.discourse-cdn.com/v4/letter/s/c68b51/32.png) [@Sergio\_Pavez](https://discuss.elastic.co/u/Sergio_Pavez)\ **Post date:** [July 13, 2016, 1:43pm UTC](https://discuss.elastic.co/t/logstash-not-sending-to-elasticsearch/55357/7 "2016-07-13T13:43:49Z")
No, because I can see in my temp file that something is being parsed, but not going to elastic. I can read the line and send it to a file, but not to ES. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [July 13, 2016, 1:45pm UTC](https://discuss.elastic.co/t/logstash-not-sending-to-elasticsearch/55357/8 "2016-07-13T13:45:24Z")
What do the events logged to debug-filter.json look like? ---
**Author:** ![Sergio\_Pavez](https://avatars.discourse-cdn.com/v4/letter/s/c68b51/32.png) [@Sergio\_Pavez](https://discuss.elastic.co/u/Sergio_Pavez)\ **Post date:** [July 13, 2016, 1:51pm UTC](https://discuss.elastic.co/t/logstash-not-sending-to-elasticsearch/55357/9 "2016-07-13T13:51:14Z")
Like this: > [@Sergio\_Pavez](#): > > { > "@version": "1", > "@timestamp": "2016-07-12T22:35:03.423Z", > "date": "20160704", > "time": "105522.383935", > "thread": "13340", > "uuid": "18d1ccf3-9d2d-4602-bdb7-52a84351a57d", > "resource\_id\_module": "S2", > "method": "tryDeliver", > "file": "XMLdispatcher.cpp.532", > "severity": "info", > "interpreter": "InterpreterTimestamp", > "execute\_time": "0", > "id\_log": "sigas-control.log", > "received\_from": "zero" > } ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [July 13, 2016, 1:58pm UTC](https://discuss.elastic.co/t/logstash-not-sending-to-elasticsearch/55357/10 "2016-07-13T13:58:52Z")
Okay, that's odd. Things I'd do: - Crank up Logstash's logging and look for clues. - Make sure that it's not buffering in the elasticsearch output that's making you think that Logstash isn't sending anything. Appending a number of messages to fill up the buffer should accomplish that. - Maybe Logstash _is_ sending to ES and you're just looking in the wrong place? Capturing the network traffic will tell for sure. ---
**Author:** ![Sergio\_Pavez](https://avatars.discourse-cdn.com/v4/letter/s/c68b51/32.png) [@Sergio\_Pavez](https://discuss.elastic.co/u/Sergio_Pavez)\ **Post date:** [July 13, 2016, 2:03pm UTC](https://discuss.elastic.co/t/logstash-not-sending-to-elasticsearch/55357/11 "2016-07-13T14:03:21Z")
The thing is, when I don't use my template, the data appears in ES as new docs. `health status index pri rep docs.count docs.deleted store.size pri.store.size yellow open sigas-control.log 5 1 2 0 6.2kb 6.2kb` I know that the problem is in my template, but I don't know why. I'm new to ELK so there is a lot I don't know. ---
**Author:** ![Sergio\_Pavez](https://avatars.discourse-cdn.com/v4/letter/s/c68b51/32.png) [@Sergio\_Pavez](https://discuss.elastic.co/u/Sergio_Pavez)\ **Post date:** [July 13, 2016, 2:20pm UTC](https://discuss.elastic.co/t/logstash-not-sending-to-elasticsearch/55357/12 "2016-07-13T14:20:17Z")
This is what I get when in Logstash log: ``` "reason"=>"failed to parse [time]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid format: \"105522.383935\" is malformed at \".383935\""}}}}, :level=>:warn} ``` Even though grok recognizes that as time format HHmmss, ES does not accept it. I'm going to set that as a plain string and see what happens. Edit: That was the problem, ES didn't accept 105522.383935 as HHmmss time format. Now the data is being sended to ES and I can access it via Kibana with my template format. Sorry for the issue, should of just started by debugging Logstash. Thanks for your help @magnusbaeck . ---
**Author:** ![honey](https://avatars.discourse-cdn.com/v4/letter/h/a4c791/32.png) [@honey](https://discuss.elastic.co/u/honey)\ **Post date:** [May 5, 2017, 5:16am UTC](https://discuss.elastic.co/t/logstash-not-sending-to-elasticsearch/55357/13 "2017-05-05T05:16:23Z")
here logstash is creating logs but not sending it to ES. this is my log\_config file input { tcp { port =\> 5022 type =\> "syslog" } udp { port =\> 5022 type =\> "syslog" } } output { elasticsearch { hosts =\> ["192.168.1.103:9200"] user =\> "elastic" password =\> "changeme" index =\> "logstash-%{+YYYY.MM.dd}" } stdout { codec =\> "rubydebug" } } can anyone help me with this? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [May 5, 2017, 5:23am UTC](https://discuss.elastic.co/t/logstash-not-sending-to-elasticsearch/55357/14 "2017-05-05T05:23:40Z")
@honey, please start a new thread for your question. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:26am UTC](https://discuss.elastic.co/t/logstash-not-sending-to-elasticsearch/55357/15 "2017-07-06T04:26:47Z")