Hi Guyboertje,
I have adjust my ruby code, then this issue is resolved, the cause is i have referred the wrong data format that can't adapt the source log. Thank you.
But when I load my logstash script to parse my source log, i have met another performance issue.
`
[2019-09-03T14:21:49,106][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"logstash-2019.09.03", :_type=>"_doc", :_routing=>nil}, #LogStash::Event:0x1b6c342a], :response=>{"index"=>{"_index"=>"logstash-2019.09.03", "_type"=>"_doc", "_id"=>"4_PJ9WwB6iwMcN-ZT20T", "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"Limit of total fields [1000] in index [logstash-2019.09.03] has been exceeded"}}}}
`
Would you please help me check it? It seems a parameter that limit the max index count. Of course I have found a solution, but I don't know how to use it.
The detailed information you can access this ticket:
Regarding to logstash performance limit Logstash
Hi experts, I have met a logstash performance limit issue, when I parse my log, logstash report this error log: ` [2019-09-03T14:21:49,106][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"logstash-2019.09.03", :_type=>"_doc", :_routing=>nil}, #LogStash::Event:0x1b6c342a], :response=>{"index"=>{"_index"=>"logstash-2019.09.03", "_type"=>"_doc", "_id"=>"4_PJ9WwB6iwMcN-ZT20T", "status"=>400, "error"=>{"type"=>"…
Thank you again.
[/quote]