Hey folks,
I am trying to integrate Bro ids with the elastic stack and keep running into this error message:
[2019-04-05T14:12:05,234][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"bro-2019.04.05", :_type=>"doc", :routing=>nil}, #LogStash::Event:0x4fadd86f], :response=>{"index"=>{"_index"=>"bro-2019.04.05", "_type"=>"doc", "_id"=>"976z7mkBHuey8d3JU1qX", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"Could not dynamically add mapping for field [id.resp_h]. Existing mapping for [id] must be of type object but found [text]."}}}}
I understand this is likely a simple question and apologize in advance.
I know you change the mapping on elasticsearch directly, however due to external circumstances this solution is not available to me.
Is there a way to specify that the "id" field is to be processed as an object within a logstash pipeline config file? Is there a way to do this using mutate?
For extra information, the id field is considered to be a record, which works like an array. Here is the explanation in the bro documentation.
A “record” is a collection of values. Each value has a field name and a type. Values do not need to have the same type and the types have no restrictions. Field names must follow the same syntax as regular variable names (except that field names are allowed to be the same as local or global variables). An example record type definition:
Am I approaching this incorrectly altogether? I am having a difficult time understanding the issue here.
Thank you for your time!