[SOLVED]Logstash strange indexing error

Elastic Stack Logstash
1

Hello,
During reindexing through logstash I get this error (lot of times...):
Should not be that message part also qouted?

Failed action. {:status=>400, :action=>["index", {:_id=>"+lmAQBUf3Mrk9Yymy08bABnbE8mg8EFRHqrx6gqNvfkzBbZ5IvXyi8ZkiXMNxIJMgY3g46sA0cofWwOSDdKeQQ==", :_index=>"logs-torque-2013-12", :_type=>"torque_accounting", :_routing=>nil}, 2013-12-31T23:46:40.000Z {"geoip":{"timezone":"Europe/Prague","ip":"147.231.25.5","latitude":50.08330000000001,"country_code2":"CZ","country_name":"Czech Republic","country_code3":"CZE","continent_code":"EU","location":[14.416699999999992,50.08330000000001],"longitude":14.416699999999992},"ip":"147.231.25.5","name":"torque.farm.particle.cz"} 01/01/2014 00:46:40;S;26399646.torque.farm.particle.cz;user=atlasprd013 group=atlasprd jobname=cream_521474028 queue=gridatlas ctime=1388533344 qtime=1388533344 etime=1388533344 start=1388533600 owner=atlasprd013@cream1.farm.particle.cz exec_host=ibis28/1 Resource_List.neednodes=1 Resource_List.nodect=1 Resource_List.nodes=1 Resource_List.vmem=8000mb Resource_List.walltime=200:00:00], :response=>{"index"=>{"_index"=>"logs-torque-2013-12", "_type"=>"torque_accounting", "_id"=>"+lmAQBUf3Mrk9Yymy08bABnbE8mg8EFRHqrx6gqNvfkzBbZ5IvXyi8ZkiXMNxIJMgY3g46sA0cofWwOSDdKeQQ==", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [torque.resource_list]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"unknown property [neednodes]"}}}}, :level=>:warn}

AM

2

It also happens with fresh logs which was not before version 2.3.0

3

I solved that, but it is starnge taht I needed to define new mapping in place where I didin't need any before...

4

Hi,
how did you solve it ? can you share your config file?
I am getting the same error and wondering for a solution.

5

Hi,
my config won't help you.
You need to pay attention to variable mapping. It is nearly certain that you are trying to index document into anything flat (string, or whatever).

Small example:
you first indexed (using recursive kv filter) this

var1="xxxx" var2="1233" var3="euzybei"

which resulted into object with these properties:

var1(string)=xxx
var2(int)=1233
var3(string)=euzybei

This also initialized mapping in ES in this way (you can't index string into var2, because it is int)

And now you process:

var1="rrrr" var2="var2_1=1254 var2_2=6523" var3="euzi"

which leads to

var1(string)=rrrr
var2(document)=
    var2_1(int)=1254
    var2_2(int)=6523
var3(string)=euzi

Which fails with precisely this error
Your previous mapping for var2 is int and you can't index document into it.
You need to take care of this in your ES template.

6

If this won't help you, please share your config (ES and LS) and a sample (big one) of input data.