Logstash send all messages to their log file instead of elasticsearch

Elastic Stack Logstash
1

Yesterday I upgraded logstash and elasticsearch to the latest version (from 1.x). I have had some problem with it but after fixing all issues it started to work. Today at 2am (at this time server load increase for 5-7min to maximum, it happens everyday) logstash stop sending data to elasticsearch it send all messages to /var/log/logstash/logstash.log. I restarted all ELK stack but it can't halp. I check indices and not found one for today.

curl http://localhost:9200/_cat/indices | grep "2015.12.11"
yellow open .marvel-es-2015.12.11 1 1   322537 6544 126.6mb 126.6mb

I can't understand what is wrong. In kibana i see all messages before 2am. My timezone GMT+2. Last received message at *December 11th 2015, 01:59:59.000 * seems that ES create indices in GMT time.

I found in elasticsearch log repeated line (this is why it can't create index)

[2015-12-11 01:00:48,559][DEBUG][action.admin.indices.create] [MondomonELK] [logstash-2015.12.11] failed to create
MapperParsingException[Failed to parse mapping [_default_]: Mapping definition for [geoip] has unsupported parameters:  [path : full]]; nested: MapperParsingException[Mapping definition for [geoip] has unsupported parameters:  [path : full]];

CentOS release 6.7 (Final)
logstash 2.1.1
elasticsearch Version: 2.1.0, Build: 72cd1f1/2015-11-18T22:40:03Z, JVM: 1.7.0_91
kibana 4.3.0 and 3.x

elasticsearch.yml

cluster.name: ELK_cluster_xxxxx
node.name: "xxxxxx"
network.host: 0.0.0.0
# kibana 3 compatibility
http.cors.allow-origin: "/.*/"
http.cors.enabled: true

logstash.conf (only output)

output {
  elasticsearch {
    hosts => [ "127.0.0.1:9200" ]
  }
}
2

The answer is simple)
I found that path is deprecated link
You need to add this parameter template_overwrite => true to your logstash config.

logstash.conf (only output)

output {
  elasticsearch {
    hosts => [ "127.0.0.1:9200" ]
    template_overwrite => true
  }
}

And restart logstash.

source link

1 Like