# Field limit exceeded, Docs count too high

**URL:** <https://discuss.elastic.co/t/field-limit-exceeded-docs-count-too-high/140785>\
**Category:** Elasticsearch\
**Created:** [July 19, 2018, 5:58pm UTC](https://discuss.elastic.co/t/field-limit-exceeded-docs-count-too-high/140785 "2018-07-19T17:58:05Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jacob\_Steele](https://avatars.discourse-cdn.com/v4/letter/j/e19adc/32.png) [@Jacob\_Steele](https://discuss.elastic.co/u/Jacob_Steele)\
**Post date:** [July 19, 2018, 5:58pm UTC](https://discuss.elastic.co/t/field-limit-exceeded-docs-count-too-high/140785/1 "2018-07-19T17:58:06Z")

</div>

Hello Elastic-co,

We are currently using a full ride of your system, LogStash to Kibana to Elastic Search. However I have some questions,

**1. Field Limit Exceeded:**

I am randomly getting in my logs an error about the:

`[2018-07-19T12:10:00,717][WARN][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"myindex", :_type=>"doc", :_routing=>nil}, #<LogStash::Event:0x724d17fc>], :response=>{"index"=>{"_index"=>"myindex", "_type"=>"doc", "_id"=>"UmOFs2QBIfe17xrKFSEI", "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"Limit of total fields [1000] in index [myindex] has been exceeded"}}}}`

However when I visit the elastic search index and copy the entire mapping object, paste it into console and do a Object.Keys(myobject.doc.properties).length the return is only 723.

Where does this 1000 error come from? I am using managed templates and using :sql\_last\_value with a stored procedure.

**2. Docs count no longer close to matching**

When I left last night after running a single index, my docs count was within 20 records of the total number of results in my stored procedure. It was running for a few hours never exceeded that. When I turned on all 5 of my indexes and let it run over night the docs count has gone up by _943,000_ when I only have a count in the stored procedure of 28,855. It is properly showing the last id in the logs of logstash, what would cause the docs count to sky rocket so much?

**3. use\_lowecase\_columns**

I havent been able to find this setting anywhere in the documentation, however I noticed during a config error yesterday in my log file it had this setting set to true. I didn't set this value and it might explain why my original tracking column of VisitorID wasn't working. I have since changed it to "id" and its working properly, but is this value changeable? I am using the jdbc connector to sql server.

**Relevant Documents**  
mytestindex.conf (before compiled)

```
input {  
jdbc {
    jdbc_connection_string => "@connstring"
    jdbc_user => nil
    jdbc_driver_library => "@driverpath"
    jdbc_driver_class => "com.microsoft.sqlserver.jdbc.SQLServerDriver"
		schedule => "*/5 * * * *"
    statement => "EXEC GetMyTestIndex :sql_last_value"
		use_column_value => true
		tracking_column => "id"
		tracking_column_type => "numeric"
		id => "test_index_runner"
		record_last_run => true
}
}
filter{
	useragent {
		source => "browser"
		target => "user_agent"
		remove_field => "browser"
	}
}
output {  
elasticsearch { 
		hosts => "@awiip"
		user => "@awslogstashuser"
		password => "@awslogstashpw"
		index => "mytestindex"
		template => "@templatepath"
	}
}

```

Template:

```
{
  "template" : "logstash-*",
  "version" : 60001,
  "settings" : {
    "index.refresh_interval" : "5s",
	"index": {
      "mapping": {
        "total_fields": {
          "limit": "2000"
        }
      }
	}
  },
  "mappings" : {
    "_default_" : {
    }
  }
}
```

---

<div class="post-metadata">

**Author:** ![mayya](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mayya/32/83147_2.png) [@mayya](https://discuss.elastic.co/u/mayya)\
**Post date:** [July 27, 2018, 5:30pm UTC](https://discuss.elastic.co/t/field-limit-exceeded-docs-count-too-high/140785/2 "2018-07-27T17:30:30Z")

</div>

Answering only your first question about field limit exceeded.

> Where does this 1000 error come from?

In this total number of fields, we also include meta fields. We have a [github issue](https://github.com/elastic/elasticsearch/issues/24096) to clarify the documentation on total number of field. In this issue, you can also find some examples of these metafields.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 24, 2018, 5:30pm UTC](https://discuss.elastic.co/t/field-limit-exceeded-docs-count-too-high/140785/3 "2018-08-24T17:30:41Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.