Hanneu
July 18, 2018, 10:28pm
1
Hello experts,
I have been getting very many below error and warning messages.
1.) [ERROR][logstash.filters.ruby ] Ruby exception occurred: undefined method `[]' for nil:NilClass
2.) ][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"ind_ecs_airs_counter-2018.07.18", :_type=>"logs", :_routing=>nil}, 2018-07-18T04:01:01.541Z POAIR001 DATE/TIME,EMAP Request V1,EMAP Response V1,EMAP Request V2,EMAP Response V2,ETSI MAP Request V2,ETSI MAP Response V2,ETSI MAP Request V3,ETSI MAP Response V3], :response=>{"index"=>{"_index"=>"ind_ecs_airs_counter-2018.07.18", "_type"=>"logs", "_id"=>"AWSrjIbB6iFc8xSmLrcS", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [ETSI_MAP_Response_V3]", "caused_by"=>{"type"=>"number_format_exception", "reason"=>"For input string: "ETSI MAP Response V3""}}}}}
Can someone please suggest?
Badger
July 18, 2018, 10:52pm
2
For the first one, you need to show us at least the ruby filter from your logstash configuration.
For the second, you have a field that contains "ETSI MAP Response V3" and elasticsearch is expecting that field to be a number.
Hanneu
July 19, 2018, 4:56pm
3
I am looking for ruby filter in files under logstash config (/etc/logstash/conf.d). However, I am not 100% sure how to check for ruby filter. Can you suggest?
Second issue, I understand elastic search is expecting the field "EPSI Map Response V3" to be in numeric format. However, is there a way we can change to string or any other non-numeric type?
1 Like
Hanneu
July 19, 2018, 10:37pm
4
For first issue, would the ruby filter be under /usr/share/logstash/logstash-core/lib/logstash/filters?
Badger
July 19, 2018, 10:49pm
5
No, I would expect it to be in your logstash config (/etc/logstash/conf.d or somewhere like that).
Hanneu
July 20, 2018, 3:30pm
6
ruby
{
code => " event.set('TotalMap', event.get('EMAP_Request_V1').to_i + event.get('EMAP_Request_V2').to_i + event.get('ETSI_MAP_Request_V2').to_i + event.get('ETSI_MAP_Request_V3').to_i )
event.set('SuccessMap', event.get('EMAP_Response_V1').to_i + event.get('EMAP_Response_V2').to_i + event.get('ETSI_MAP_Response_V2').to_i + event.get('ETSI_MAP_Response_V3').to_i ) "
}
ruby
{
code => " if (event.get('BalanceEnquiryOut').to_i + event.get('AccumulatorEnquiryOut').to_i != 0)
event.set('balance', ( event.get('BalanceEnquiryIn').to_i + event.get('AccumulatorEnquiryIn').to_i ) / ( event.get('BalanceEnquiryOut').to_i + event.get('AccumulatorEnquiryOut').to_i ))
else
event.set('balance', 0 )
end "
}
Badger
July 20, 2018, 3:43pm
7
That's the kind of thing we are looking for. Although I would not expect those to produce that error. The NilClass could be produced by doing event.get on a field that does not exist. ruby seems to be OK doing to_i on nil, so those filters should be OK. I'd be looking for something that indexes into an array without checking for nil first. A reference to something like
event.get("noSuchField")[0]
Hanneu
July 20, 2018, 3:51pm
8
ruby
{
code => "
event.to_hash.keys.each { |k|
if k.start_with? '/Air:GetFaFList'
event.set('GetFaFList',event.get('GetFaFList').to_i + event.get(k).to_i)
elsif k.start_with? '/Air:UpdateFaFList'
event.set('UpdateFaFList',event.get('UpdateFaFList').to_i + event.get(k).to_i)
Badger
July 20, 2018, 3:58pm
9
k is certain to exist, so the start_with? should be OK, and the only other thing that one does it to_i, which we know is OK.
system
(system)
Closed
August 17, 2018, 3:58pm
10
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.