# Impact of ignore\_above property in ES mapping

**URL:** <https://discuss.elastic.co/t/impact-of-ignore-above-property-in-es-mapping/20745>\
**Category:** Elasticsearch\
**Created:** [November 14, 2014, 8:24am UTC](https://discuss.elastic.co/t/impact-of-ignore-above-property-in-es-mapping/20745 "2014-11-14T08:24:25Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![thomasvdb](https://avatars.discourse-cdn.com/v4/letter/t/22d042/32.png) [@thomasvdb](https://discuss.elastic.co/u/thomasvdb)\
**Post date:** [November 14, 2014, 8:24am UTC](https://discuss.elastic.co/t/impact-of-ignore-above-property-in-es-mapping/20745/1 "2014-11-14T08:24:25Z")

</div>

We've configured logstash in combination with elasticsearch and Kibana to  
centralize our server logs.  
In Kibana I want to setup a table which groups all error messages so we can  
create a top 10 of the most occuring errors.

We tried to setup a terms table grouped on a specific field (which contains  
the error message).  
However there is a record "missing fields" in the table with a count of  
16997.

The configuration of the terms table results in the following elasticsearch  
query (posting a part of the filter):

"facets": {  
"terms": {  
"terms": {  
"field": "information.information.raw",  
"size": 100000000,  
"order": "count",  
"exclude": []  
}  
[...]

In the result a missing property indicates quite a lot of missing results  
as we've noticed in the terms table itself:

"facets": {  
"terms": {  
"\_type": "terms",  
"missing": 16997,  
"total": 584,  
"other": 0,  
[...]

I've noticed that in the mapping defined by logstash the following is added:

"information.raw": {  
"type": "string",  
"index": "not\_analyzed",  
"ignore\_above": 256  
}

I assume that ignore\_above means that any message longer then 256  
characters will not be taken into account when doing queries?  
I tried to find some documentation about it but the only thing I can find  
is at the Core Types documentation  
[http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-core-types.html](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-core-types.html)  
of elasticsearch and I'm not entirely sure if I understand it correctly.

If I remove this property in the mapping, what could be the impact of this?  
Slower queries because it needs to take larger strings into account? Is if  
safe to remove this property?

Thanks in advance!

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/3a5f188f-e77e-458a-ab34-eaf913f8b291%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/3a5f188f-e77e-458a-ab34-eaf913f8b291%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![ssharma](https://avatars.discourse-cdn.com/v4/letter/s/e19b73/32.png) [@ssharma](https://discuss.elastic.co/u/ssharma)\
**Post date:** [March 11, 2015, 9:39pm UTC](https://discuss.elastic.co/t/impact-of-ignore-above-property-in-es-mapping/20745/2 "2015-03-11T21:39:03Z")

</div>

Bump? I actually removed the "ignore\_above" option from the template and  
disabled logstash from creating a template. So my mapping now looks like:

```
      "error-message" : {
        "type" : "string",
        "norms" : {
          "enabled" : false
        },
        "fields" : {
          "raw" : {
            "type" : "string",
            "index" : "not_analyzed"
          }
        }
      },

```

But I'm still not seeing any results in kibana. Anyone have any ideas on  
this?

--Ajay

On Friday, November 14, 2014 at 12:24:25 AM UTC-8, thomasvdb wrote:

> We've configured logstash in combination with elasticsearch and Kibana to  
> centralize our server logs.  
> In Kibana I want to setup a table which groups all error messages so we  
> can create a top 10 of the most occuring errors.
> 
> We tried to setup a terms table grouped on a specific field (which  
> contains the error message).  
> However there is a record "missing fields" in the table with a count of  
> 16997.
> 
> The configuration of the terms table results in the following  
> elasticsearch query (posting a part of the filter):
> 
> "facets": {  
> "terms": {  
> "terms": {  
> "field": "information.information.raw",  
> "size": 100000000,  
> "order": "count",  
> "exclude":   
> }  
> [...]
> 
> In the result a missing property indicates quite a lot of missing results  
> as we've noticed in the terms table itself:
> 
> "facets": {  
> "terms": {  
> "\_type": "terms",  
> "missing": 16997,  
> "total": 584,  
> "other": 0,  
> [...]
> 
> I've noticed that in the mapping defined by logstash the following is  
> added:
> 
> "information.raw": {  
> "type": "string",  
> "index": "not\_analyzed",  
> "ignore\_above": 256  
> }
> 
> I assume that ignore\_above means that any message longer then 256  
> characters will not be taken into account when doing queries?  
> I tried to find some documentation about it but the only thing I can find  
> is at the Core Types documentation  
> [http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-core-types.html](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-core-types.html)  
> of elasticsearch and I'm not entirely sure if I understand it correctly.
> 
> If I remove this property in the mapping, what could be the impact of this?  
> Slower queries because it needs to take larger strings into account? Is if  
> safe to remove this property?
> 
> Thanks in advance!

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/ffa0a7d3-2b95-4d0c-a86d-58f77ec01011%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/ffa0a7d3-2b95-4d0c-a86d-58f77ec01011%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 12:27am UTC](https://discuss.elastic.co/t/impact-of-ignore-above-property-in-es-mapping/20745/3 "2017-07-06T00:27:22Z")

</div>