Created ILM successfully couple of weeks ago for multi topics but one of topics I am getting error messages on logstash log - limit of total field 1000 for footy-000013.
I added below:
PUT micky_aaa_footy*/_settings
{
"index.lifecycle.name":
"micky_aaa_footy_policy",
"index.mapping.total.fields.limit": 2000
}
Above is working ok from 000001 to 0000012 but failed on 0000013.
[2020-10-27T08:32:17,831][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"micky_aaa_footy-alias", :_type=>"_doc", :routing=>nil}, #LogStash::Event:0x24d85d3a], :response=>{"index"=>{"_index"=>"micky_aaa_footy-000013", "_type"=>"_doc", "_id"=>"oC8vaXUBlXxylKoIulX0", "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"Limit of total fields [1000] in index [micky_aaa_footy-000013] has been exceeded"}}}}
[2020-10-27T08:32:17,831][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"micky_aaa_footy-alias", :_type=>"_doc", :routing=>nil}, #LogStash::Event:0x4276faa6], :response=>{"index"=>{"_index"=>"micky_aaa_footy-000013", "_type"=>"_doc", "_id"=>"oy8vaXUBlXxylKoIulX0", "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"Limit of total fields [1000] in index [micky_aaa_footy-000013] has been exceeded"}}}}
[2020-10-27T08:32:17,831][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"micky_aaa_footy-alias", :_type=>"_doc", :routing=>nil}, #LogStash::Event:0x65a26454], :response=>{"index"=>{"_index"=>"micky_aaa_footy-000013", "_type"=>"_doc", "_id"=>"pi8vaXUBlXxylKoIulX0", "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"Limit of total fields [1000] in index [micky_aaa_footy-000013] has been exceeded"}}}}
[2020-10-27T08:32:17,831][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"micky_aaa_footy-alias", :_type=>"_doc", :routing=>nil}, #LogStash::Event:0x5ff5b935], :response=>{"index"=>{"_index"=>"micky_aaa_footy-000013", "_type"=>"_doc", "_id"=>"qS8vaXUBlXxylKoIulX0", "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"Limit of total fields [1000] in index [micky_aaa_footy-000013] has been exceeded"}}}}
[2020-10-27T08:32:17,831][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"micky_aaa_footy-alias", :_type=>"_doc", :routing=>nil}, #LogStash::Event:0x67d1e0ed], :response=>{"index"=>{"_index"=>"micky_aaa_footy-000013", "_type"=>"_doc", "_id"=>"rC8vaXUBlXxylKoIulX0", "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"Limit of total fields [1000] in index [micky_aaa_footy-000013] has been exceeded"}}}}
[2020-10-27T08:32:17,832][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"micky_aaa_footy-alias", :_type=>"_doc", :routing=>nil}, #LogStash::Event:0x7dec653b], :response=>{"index"=>{"_index"=>"micky_aaa_footy-000013", "_type"=>"_doc", "_id"=>"ry8vaXUBlXxylKoIulX0", "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"Limit of total fields [1000] in index [micky_aaa_footy-000013] has been exceeded"}}}}
[2020-10-27T08:32:17,832][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"micky_aaa_footy-alias", :_type=>"_doc", :routing=>nil}, #LogStash::Event:0x3c81c07c], :response=>{"index"=>{"_index"=>"micky_aaa_footy-000013", "_type"=>"_doc", "_id"=>"si8vaXUBlXxylKoIulX0", "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"Limit of total fields [1000] in index [micky_aaa_footy-000013] has been exceeded"}}}}
[2020-10-27T08:32:17,832][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"micky_aaa_footy-alias", :_type=>"_doc", :routing=>nil}, #LogStash::Event:0x353042e2], :response=>{"index"=>{"_index"=>"micky_aaa_footy-000013", "_type"=>"_doc", "_id"=>"tS8vaXUBlXxylKoIulX0", "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"Limit of total fields [1000] in index [micky_aaa_footy-000013] has been exceeded"}}}}
[2020-10-27T08:32:17,832][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"micky_aaa_footy-alias", :_type=>"_doc", :routing=>nil}, #LogStash::Event:0x38a57dd9], :response=>{"index"=>{"_index"=>"micky_aaa_footy-000013", "_type"=>"_doc", "_id"=>"uC8vaXUBlXxylKoIulX0", "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"Limit of total fields [1000] in index [micky_aaa_footy-000013] has been exceeded"}}}}
[2020-10-27T09:35:29,157][INFO ][org.apache.kafka.clients.FetchSessionHandler] [Consumer clientId=logstash-0, groupId=logstash] Node 0 was unable to process the fetch footy with (sessionId=1268431334, epoch=31638): INVALID_FETCH_SESSION_EPOCH.
[root@aaa-mmtt-elk02 ~]#