# Continuously getting - java.lang.IllegalArgumentException: Limit of total fields \[1000\] in index has been exceeded error
**URL:** https://discuss.elastic.co/t/continuously-getting-java-lang-illegalargumentexception-limit-of-total-fields-1000-in-index-has-been-exceeded-error/187982
**Category:** Elasticsearch
**Created:** [June 28, 2019, 7:24am UTC](https://discuss.elastic.co/t/continuously-getting-java-lang-illegalargumentexception-limit-of-total-fields-1000-in-index-has-been-exceeded-error/187982 "2019-06-28T07:24:36Z")
**Posts on this page:** 11
**Page:** 1
### Author:  [@Nishant\_Hurnekar](https://discuss.elastic.co/u/Nishant_Hurnekar)
#### Post date: [June 28, 2019, 7:24am UTC](https://discuss.elastic.co/t/continuously-getting-java-lang-illegalargumentexception-limit-of-total-fields-1000-in-index-has-been-exceeded-error/187982/1 "2019-06-28T07:24:36Z")
I am running with ELK cluster 5.6.16.
I am getting continuously below error into elasticsearch logs since quite some time for only one index. Is there any method reduce no of fields Or I will have to increase no of fields ?
I am looking for solution to configure settings into configuration file.
```
[2019-06-28T05:14:26,727][DEBUG][o.e.a.b.TransportShardBulkAction] [29-121-IDC.justdial.com] [www-2019.06.27][0] failed to execute bulk item (index) BulkShardRequest [[www-2019.06.27][0]] containing [27] requests
java.lang.IllegalArgumentException: Limit of total fields [1000] in index [www-2019.06.27] has been exceeded
at org.elasticsearch.index.mapper.MapperService.checkTotalFieldsLimit(MapperService.java:604) ~[elasticsearch-5.6.16.jar:5.6.16]
at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:420) ~[elasticsearch-5.6.16.jar:5.6.16]
at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:336) ~[elasticsearch-5.6.16.jar:5.6.16]
at org.elasticsearch.index.mapper.MapperService.merge(MapperService.java:268) ~[elasticsearch-5.6.16.jar:5.6.16]
at org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor.applyRequest(MetaDataMappingService.java:311) ~[elasticsearch-5.6.16.jar:5.6.16]
at org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor.execute(MetaDataMappingService.java:230) ~[elasticsearch-5.6.16.jar:5.6.16]
at org.elasticsearch.cluster.service.ClusterService.executeTasks(ClusterService.java:634) ~[elasticsearch-5.6.16.jar:5.6.16]
at org.elasticsearch.cluster.service.ClusterService.calculateTaskOutputs(ClusterService.java:612) ~[elasticsearch-5.6.16.jar:5.6.16]
at org.elasticsearch.cluster.service.ClusterService.runTasks(ClusterService.java:571) ~[elasticsearch-5.6.16.jar:5.6.16]
at org.elasticsearch.cluster.service.ClusterService$ClusterServiceTaskBatcher.run(ClusterService.java:263) ~[elasticsearch-5.6.16.jar:5.6.16]
at org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:150) ~[elasticsearch-5.6.16.jar:5.6.16]
at org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:188) ~[elasticsearch-5.6.16.jar:5.6.16]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:576) ~[elasticsearch-5.6.16.jar:5.6.16]
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:247) ~[elasticsearch-5.6.16.jar:5.6.16]
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:210) ~[elasticsearch-5.6.16.jar:5.6.16]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_131]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_131]
```
---
### Author:  [@Nishant\_Hurnekar](https://discuss.elastic.co/u/Nishant_Hurnekar)
#### Post date: [July 3, 2019, 9:49am UTC](https://discuss.elastic.co/t/continuously-getting-java-lang-illegalargumentexception-limit-of-total-fields-1000-in-index-has-been-exceeded-error/187982/2 "2019-07-03T09:49:05Z")
Can anyone help me here ?
---
### Author:  [@wangqinghuan](https://discuss.elastic.co/u/wangqinghuan)
#### Post date: [July 3, 2019, 10:48am UTC](https://discuss.elastic.co/t/continuously-getting-java-lang-illegalargumentexception-limit-of-total-fields-1000-in-index-has-been-exceeded-error/187982/3 "2019-07-03T10:48:29Z")
You can update limit of total fields after index has been created:
> PUT my\_index/\_settings
> {
> "index.mapping.total\_fields.limit": 2000
> }
Note too many fields will lead to mapping explosion, which is a not good practice.
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [July 3, 2019, 11:20am UTC](https://discuss.elastic.co/t/continuously-getting-java-lang-illegalargumentexception-limit-of-total-fields-1000-in-index-has-been-exceeded-error/187982/4 "2019-07-03T11:20:03Z")
Agreed. So may be reducing the number of fields is the first thing you need to look at.
Why do you have so many?
Does some of them have the same meaning?
---
### Author:  [@Nishant\_Hurnekar](https://discuss.elastic.co/u/Nishant_Hurnekar)
#### Post date: [July 3, 2019, 11:29am UTC](https://discuss.elastic.co/t/continuously-getting-java-lang-illegalargumentexception-limit-of-total-fields-1000-in-index-has-been-exceeded-error/187982/5 "2019-07-03T11:29:48Z")
Yes. I am looking for solution reduce unwanted fields. I am not in favour to increase total fields.
Please see the given screenshot. There were 2.2k fields into index. I am talking about these wanted fields which are detected as fields.
These names are part of request url & i don't want to be in index.
Can I define selective fields for index into configuration ? Let me know if more info require. plz help.
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [July 3, 2019, 11:30am UTC](https://discuss.elastic.co/t/continuously-getting-java-lang-illegalargumentexception-limit-of-total-fields-1000-in-index-has-been-exceeded-error/187982/6 "2019-07-03T11:30:41Z")
Why are you sending them to elasticsearch in the first place? If you don't need them, don't send them.
---
### Author:  [@Nishant\_Hurnekar](https://discuss.elastic.co/u/Nishant_Hurnekar)
#### Post date: [July 3, 2019, 11:58am UTC](https://discuss.elastic.co/t/continuously-getting-java-lang-illegalargumentexception-limit-of-total-fields-1000-in-index-has-been-exceeded-error/187982/7 "2019-07-03T11:58:32Z")
I am sending complete access log to ELK stack & storing below url pattrn under request field.
`/webmain/autosuggest.php?cases=what&search=chem&city=Samastipur&area=&s=1&pg=index`
Please see this access log :
`14.143.7.170, 96.17.12.143, 96.17.16.38 - - [03/Jul/2019:17:03:14 +0530] "GET /webmain/autosuggest.php?cases=what&search=chem&city=Samastipur&area=&s=1&pg=index HTTP/1.0" 200 3598 "https://www.jamuna.com/" "Mozilla/5.0 (X11; Linux i686; rv:34.0) Gecko/20100101 Firefox/34.0" "REMOTE_ADDR : 96.17.16.38" "TRUE_CLIENT : 14.143.7.170" "AKAXFF : 14.143.7.170" www.jamuna.com 1.218 0.154 IN .`
Logstash pattern :
> (?\%{IP}, .\*|%{IP:xforwardedfor}|-) (%{NGUSER:ident}|-) (%{NGUSER:auth}|%{USERNAME:user}|-) [%{HTTPDATE:timestamp}] "(?:%{WORD:method} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:request})" %{NOTSPACE:response} (%{NOTSPACE:bytes\_read}|-) (%{DATA:request\_header\_referer}|-) "(%{DATA:request\_header\_agent}|-)" "(REMOTE\_ADDR : %{DATA:clientip}|-)" "(TRUE\_CLIENT : %{DATA:http\_true\_client\_ip}|-)" "(AKAXFF : %{DATA:http\_akaxff}|-)" (?:%{HOSTNAME:http\_host}|%{IP}|%{HOSTNAME:http\_host}:%{POSINT}|-) (?:%{BASE10NUM:request\_duration}|-) (?:%{BASE10NUM:upstream\_request\_duration}|-) (%{WORD:Country}|-)
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [July 3, 2019, 12:18pm UTC](https://discuss.elastic.co/t/continuously-getting-java-lang-illegalargumentexception-limit-of-total-fields-1000-in-index-has-been-exceeded-error/187982/8 "2019-07-03T12:18:35Z")
If I'm not mistaken then `request` field just contains `/webmain/autosuggest.php?cases=what&search=chem&city=Samastipur&area=&s=1&pg=index`, right?
If so, and based only on the information you gave so far, which might be incomplete, you end up with a document like:
```auto
{
"request": "/webmain/autosuggest.php?cases=what&search=chem&city=Samastipur&area=&s=1&pg=index"
}
```
This is not generating as many fields as you shown.
---
### Author:  [@Nishant\_Hurnekar](https://discuss.elastic.co/u/Nishant_Hurnekar)
#### Post date: [July 3, 2019, 1:09pm UTC](https://discuss.elastic.co/t/continuously-getting-java-lang-illegalargumentexception-limit-of-total-fields-1000-in-index-has-been-exceeded-error/187982/9 "2019-07-03T13:09:11Z")
As you said, whatever data filled-up into request fields is correctly into ELK. My concern is only, I want to prevent to create below unwanted fields. That's the reason, I have raised this case.
&search=chem
&city=Samastipur
Hope you understand my concern.
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [July 3, 2019, 1:23pm UTC](https://discuss.elastic.co/t/continuously-getting-java-lang-illegalargumentexception-limit-of-total-fields-1000-in-index-has-been-exceeded-error/187982/10 "2019-07-03T13:23:49Z")
You can always have a strict mapping with `"dynamic": "strict"` or `"dynamic": false` . See [https://www.elastic.co/guide/en/elasticsearch/reference/current/dynamic.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/dynamic.html)
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 31, 2019, 1:23pm UTC](https://discuss.elastic.co/t/continuously-getting-java-lang-illegalargumentexception-limit-of-total-fields-1000-in-index-has-been-exceeded-error/187982/11 "2019-07-31T13:23:50Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.