Hi
Followed this link to update the template Increase total fields limit via creation of index in logstash
Below is the part of my updated template.json file.
I tried to add "index.mapping.total_fields.limit: 10000" under setting. But still it is not working
{
"template": "event-%{+YYYY.MM.dd}",
"settings": {
"index.refresh_interval" : "5s",
"index.mapping.total_fields.limit": 10000
},
"mappings": {
"logs": {
"dynamic_templates": [
{
"boolean_value": {
"match": "*_boolean",
"mapping": {
"type": "boolean"
}
}
},
Updated logstash.conf
output {
elasticsearch {
index => "event-%{+YYYY.MM.dd}"
hosts => ["elasticsearch-machine"]
template => "C:\apps\logstash-6.3.0\config\templates\template.json"
template_name => 'event' # The default is logstash
template_overwrite => true
}
Run GET event-/_settings command from kibana UI, There is no "index.mapping.total_fields.limit": 10000 in the setting of "event-"
Error on Elastic Search cmd console
[2018-08-28T02:44:44,268][DEBUG][o.e.a.a.i.m.p.TransportPutMappingAction] [TIsOAqH] failed to put mappings on indices [[[event-2018.08.28/lYtCv2TZSvikfNhXHklUuQ]]], type [doc]
java.lang.IllegalArgumentException: Limit of total fields [1000] in index [event-2018.08.28] has been exceeded
at org.elasticsearch.index.mapper.MapperService.checkTotalFieldsLimit(MapperService.java:630) ~[elasticsearch-6.3.0.jar:6.3.0]
at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:463) ~[elasticsearch-6.3.0.jar:6.3.0]
at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:356) ~[elasticsearch-6.3.0.jar:6.3.0]
at org.elasticsearch.index.mapper.MapperService.merge(MapperService.java:288) ~[elasticsearch-6.3.0.jar:6.3.0]
at org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor.applyRequest(MetaDataMappingService.java:313) ~[elasticsearch-6.3.0.jar:6.3.0]
at org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor.execute(MetaDataMappingService.java:230) ~[elasticsearch-6.3.0.jar:6.3.0]
at org.elasticsearch.cluster.service.MasterService.executeTasks(MasterService.java:630) ~[elasticsearch-6.3.0.jar:6.3.0]
at org.elasticsearch.cluster.service.MasterService.calculateTaskOutputs(MasterService.java:267) ~[elasticsearch-6.3.0.jar:6.3.0]
at org.elasticsearch.cluster.service.MasterService.runTasks(MasterService.java:197) [elasticsearch-6.3.0.jar:6.3.0]
at org.elasticsearch.cluster.service.MasterService$Batcher.run(MasterService.java:132) [elasticsearch-6.3.0.jar:6.3.0]
at org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:150) [elasticsearch-6.3.0.jar:6.3.0]
at org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:188) [elasticsearch-6.3.0.jar:6.3.0]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:625) [elasticsearch-6.3.0.jar:6.3.0]
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:244) [elasticsearch-6.3.0.jar:6.3.0]
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:207) [elasticsearch-6.3.0.jar:6.3.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_181]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_181]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]
[2018-08-28T02:44:44,417][DEBUG][o.e.a.b.TransportShardBulkAction] [event-2018.08.28][1] failed to execute bulk item (index) BulkShardRequest [[event-2018.08.28][1]] containing [10] requests
java.lang.IllegalArgumentException: Limit of total fields [1000] in index [event-2018.08.28] has been exceeded
at org.elasticsearch.index.mapper.MapperService.checkTotalFieldsLimit(MapperService.java:630) ~[elasticsearch-6.3.0.jar:6.3.0]
at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:463) ~[elasticsearch-6.3.0.jar:6.3.0]