# Term Aggregations and StopWords **URL:** \ **Category:** Elasticsearch\ **Created:** [July 24, 2014, 3:57pm UTC](https://discuss.elastic.co/t/term-aggregations-and-stopwords/18872 "2014-07-24T15:57:59Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![Andre\_Morais](https://avatars.discourse-cdn.com/v4/letter/a/57b2e6/32.png) [@Andre\_Morais](https://discuss.elastic.co/u/Andre_Morais)\ **Post date:** [July 24, 2014, 3:57pm UTC](https://discuss.elastic.co/t/term-aggregations-and-stopwords/18872/1 "2014-07-24T15:57:59Z")
Hi! I'm really enjoying all the possibilities brought about by the move from facets to aggregations. However, I still can't figure out the relationship between facets or buckets and analyzers. Is it not possible at all to get the buckets out of an analyzed field? Specifically, I need to get list of most common words, but I want to use my stopword list to exclude those that do not matter to me. I am using a stop word filter: ``` index.analysis.filter.fnstop: type: stop stopwords: ["my", "it", "the", "likes"] ``` And a custom analyzer: ``` index.analysis.analyzer.test: type: custom tokenizer: whitespace filter: lowercase, asciifolding, fnstop ``` I then map my field with the custom analyzer: ... "Clean\_Message" : {{"type" : "string", "analyzer" : "test"} And request list of top 100 most common terms, using the search API: { "query": { "bool": { "must": [{ "match\_all": {} }] } }, "aggs": { "Message": { "terms": { "field": "Clean\_Message", "size": 100, "order": { "\_count": "desc" } } } } } However, some words in my stop filter appear in that list. Is it by design? Are we not supposed to run facets or aggregations agains an analyzed field? Is it possible to get the list of most common terms against an analyzed field? Thank you very much for your attention and for your work! ``` André Morais ``` -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/0b9f0874-6f79-46d6-8e9b-5393b0b3cd10%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0b9f0874-6f79-46d6-8e9b-5393b0b3cd10%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Andre\_Morais](https://avatars.discourse-cdn.com/v4/letter/a/57b2e6/32.png) [@Andre\_Morais](https://discuss.elastic.co/u/Andre_Morais)\ **Post date:** [September 17, 2014, 3:58pm UTC](https://discuss.elastic.co/t/term-aggregations-and-stopwords/18872/2 "2014-09-17T15:58:59Z")
Hello! Still can't get the result I want: stop words not appearing in buckets. Further testing showed that: - if I filter aggregation with a query for one of the stop words, I get an empty result for aggregations; - the same analyzer is changing all đŸ™‚ and ☹ and replacing them with SMILE and FROWN, these appear as such in the aggregation results; - if I include all the stop words using the "exclude" option, it works; So it appears that my analyzer is doing everything it should, except filtering the stop words when getting the aggregations (it works for search). And I am beginning to wonder if this could be, in fact, a bug... Any thoughts? Thanks, ``` AndrĂ© Morais ``` Quinta-feira, 24 de Julho de 2014 16:57:59 UTC+1, AndrĂ© Morais escreveu: > Hi! > > I'm really enjoying all the possibilities brought about by the move from > facets to aggregations. However, I still can't figure out the relationship > between facets or buckets and analyzers. Is it not possible at all to get > the buckets out of an analyzed field? > > Specifically, I need to get list of most common words, but I want to use > my stopword list to exclude those that do not matter to me. > > I am using a stop word filter: > > ``` > index.analysis.filter.fnstop: > type: stop > stopwords: ["my", "it", "the", "likes"] > > ``` > > And a custom analyzer: > > ``` > index.analysis.analyzer.test: > type: custom > tokenizer: whitespace > filter: lowercase, asciifolding, fnstop > > ``` > > I then map my field with the custom analyzer: > ... > "Clean\_Message" : {{"type" : "string", "analyzer" : "test"} > > And request list of top 100 most common terms, using the search API: > { > "query": { "bool": { "must": [{ "match\_all": {} }] } }, > "aggs": { > "Message": { > "terms": { > "field": "Clean\_Message", > "size": 100, > "order": { "\_count": "desc" } > } > } > } > } > > However, some words in my stop filter appear in that list. > > Is it by design? Are we not supposed to run facets or aggregations agains > an analyzed field? > > Is it possible to get the list of most common terms against an analyzed > field? > > Thank you very much for your attention and for your work! > > ``` > AndrĂ© Morais > > ``` -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/79bd7b05-ba26-4f26-a817-e3c34061a325%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/79bd7b05-ba26-4f26-a817-e3c34061a325%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:01am UTC](https://discuss.elastic.co/t/term-aggregations-and-stopwords/18872/3 "2017-07-06T01:01:30Z")