# Shingles and terms aggregation not working as expected

**URL:** <https://discuss.elastic.co/t/shingles-and-terms-aggregation-not-working-as-expected/240705>\
**Category:** Elasticsearch\
**Created:** [July 10, 2020, 1:33pm UTC](https://discuss.elastic.co/t/shingles-and-terms-aggregation-not-working-as-expected/240705 "2020-07-10T13:33:44Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![zp230](https://avatars.discourse-cdn.com/v4/letter/z/4da419/32.png) [@zp230](https://discuss.elastic.co/u/zp230)\
**Post date:** [July 10, 2020, 1:33pm UTC](https://discuss.elastic.co/t/shingles-and-terms-aggregation-not-working-as-expected/240705/1 "2020-07-10T13:33:44Z")

</div>

Hi,

I am having issues with getting even a simple terms aggregation on a shingled field. For context, I want to be able to get a list of words/phrases found in the sentence/display\_text field (using shingles), and their frequency of occurrence.

My index mapping is as follows:

```auto
{
    "settings": {
        "max_shingle_diff": 4,
        "analysis": {
            "analyzer": {
                "custom": {
                    "tokenizer": "standard",
                    "filter": ["lowercase", "shingle_filter"]
                }
            },
            "filter": {
                "shingle_filter": {
                    "type": "shingle",
                    "min_shingle_size": 2,
                    "max_shingle_size": 5,
                    "output_unigrams": "true"
                }
            }
        }
    },
    "mappings": {
        "properties": {
             ...
            "sentences": {
                "type": "nested",
                "include_in_root": "true",
                "properties": {
                    "display_text": {
                        "type": "text",
                        "fields": {
                            "shingles": {
                                "type": "text",
                                "analyzer": "custom"
                            }
                        }
                        },
                         ...
}

```

And here is the query I run:

```auto
GET /books/_search
{
    "size": 0,
    "query": {
        "terms": {
            "title": ["Book1"]
        }
    },
    "aggs": {

        "sentences_count": {
            "nested": {
                "path": "sentences"
            },
            "aggs": {
                "phrases": {
                    "terms": {
                        "field": "sentences.display_text.shingles",
                        "size": 100
                    }
                }
            }
        }
    }
}

```

However, I am getting an error that seems to still view the sentences.display\_text.shingles as text, not shingle tokens?

> "type" : "illegal\_argument\_exception",  
> "reason" : "Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [sentences.display\_text.shingles] in order to load field data by uninverting the inverted index. Note that this can use significant memory."

Does anyone know what I am doing wrong? From previous discussions on here it seems that this would be the correct query structure and index mapping.

---

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [July 14, 2020, 10:01am UTC](https://discuss.elastic.co/t/shingles-and-terms-aggregation-not-working-as-expected/240705/2 "2020-07-14T10:01:42Z")

</div>

See [https://www.elastic.co/guide/en/elasticsearch/reference/current/fielddata.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/fielddata.html)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 11, 2020, 10:15am UTC](https://discuss.elastic.co/t/shingles-and-terms-aggregation-not-working-as-expected/240705/3 "2020-08-11T10:15:19Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.