# Stopword functionality suppression **URL:** \ **Category:** Elasticsearch\ **Created:** [August 6, 2012, 1:52pm UTC](https://discuss.elastic.co/t/stopword-functionality-suppression/8646 "2012-08-06T13:52:58Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![John\_Ohno](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/john_ohno/32/1746_2.png) [@John\_Ohno](https://discuss.elastic.co/u/John_Ohno)\ **Post date:** [August 6, 2012, 1:52pm UTC](https://discuss.elastic.co/t/stopword-functionality-suppression/8646/1 "2012-08-06T13:52:58Z")
We are trying to make Elastic work under constraints inherited from an older system that didn't support stopwords. We tried setting the stopword list in JSON to a null string both at index creation time and at search time, but we have discovered that this doesn't allow one to search the index for english stopwords. Is there a straightforward way to turn off stopwords globally, aside from diking out the code that handles them? Alternately, were we to use the method of setting the stopword list to a null string, would there be yet a third place to set the stopword list in order to suppress this behavior? ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [August 6, 2012, 2:26pm UTC](https://discuss.elastic.co/t/stopword-functionality-suppression/8646/2 "2012-08-06T14:26:45Z")
You can define your own mapping and use a standard analyzer with an empty stopwords collection. > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. [http://www.elasticsearch.org/guide/reference/index-modules/analysis/standard-analyzer.html](http://www.elasticsearch.org/guide/reference/index-modules/analysis/standard-analyzer.html) I suppose it could work. I never tested an empty stopword list. HTH David Le 6 août 2012 à 15:52, John Ohno [john.ohno@gmail.com](mailto:john.ohno@gmail.com) a écrit : > We are trying to make Elastic work under constraints inherited from an older > system that didn't support stopwords. We tried setting the stopword list in > JSON to a null string both at index creation time and at search time, but we > have discovered that this doesn't allow one to search the index for english > stopwords. Is there a straightforward way to turn off stopwords globally, > aside from diking out the code that handles them? Alternately, were we to use > the method of setting the stopword list to a null string, would there be yet a > third place to set the stopword list in order to suppress this behavior? -- David Pilato [http://www.scrutmydocs.org/](http://www.scrutmydocs.org/) [http://dev.david.pilato.fr/](http://dev.david.pilato.fr/) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs ---
**Author:** ![John\_Ohno](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/john_ohno/32/1746_2.png) [@John\_Ohno](https://discuss.elastic.co/u/John_Ohno)\ **Post date:** [August 6, 2012, 4:35pm UTC](https://discuss.elastic.co/t/stopword-functionality-suppression/8646/3 "2012-08-06T16:35:02Z")
I finally got that to work, but it turned out in order to do it, you have to specify the custom analyzer for the index and for every field in the mapping individually, then add a blank stopwords list to the query at search-time. While this is feasible for me (I generate the searches, the mappings, and the index requests), it seems a bit strange that specifying it top-level doesn't trickle-down. For instance, the analyzer attribute for the field mapping does not appear to cascade to the index\_analyzer attribute for the same field's mapping, and even if you specify a stopword-less search\_analyzer for every field, stopwords are not excluded unless the query passes a blank stopword list. Surely there must be other people who want to remove stopwords or change them globally? On Monday, August 6, 2012 10:26:45 AM UTC-4, David Pilato wrote: > You can define your own mapping and use a standard analyzer with an > empty stopwords collection. > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/analysis/standard-analyzer.html) > > I suppose it could work. I never tested an empty stopword list. > > HTH > > David > > Le 6 août 2012 à 15:52, John Ohno [john.ohno@gmail.com](mailto:john.ohno@gmail.com) a écrit : > > We are trying to make Elastic work under constraints inherited from an > older system that didn't support stopwords. We tried setting the stopword > list in JSON to a null string both at index creation time and at search > time, but we have discovered that this doesn't allow one to search the > index for english stopwords. Is there a straightforward way to turn off > stopwords globally, aside from diking out the code that handles them? > Alternately, were we to use the method of setting the stopword list to a > null string, would there be yet a third place to set the stopword list in > order to suppress this behavior? > > -- > David Pilato > [http://www.scrutmydocs.org/](http://www.scrutmydocs.org/) > [http://dev.david.pilato.fr/](http://dev.david.pilato.fr/) > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs ---
**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\ **Post date:** [August 6, 2012, 4:59pm UTC](https://discuss.elastic.co/t/stopword-functionality-suppression/8646/4 "2012-08-06T16:59:37Z")
You can override default analyzer globally by adding these lines to elasticsearch.yml: index.analysis.analyzer.default: type: custom tokenizer: standard filter: standard, lowercase This will create a custom analyzer with Standard Tokenized and two filters: Standard and Lowercase. This way your custom analyzer will be identical to the standard analyzer but it will not use the stopword filter. Because it's named "default", elasticsearch will use it everywhere where analyzer is not explicitly set. Igor On Monday, August 6, 2012 12:35:02 PM UTC-4, John Ohno wrote: > I finally got that to work, but it turned out in order to do it, you have > to specify the custom analyzer for the index and for every field in the > mapping individually, then add a blank stopwords list to the query at > search-time. > > While this is feasible for me (I generate the searches, the mappings, and > the index requests), it seems a bit strange that specifying it top-level > doesn't trickle-down. For instance, the analyzer attribute for the field > mapping does not appear to cascade to the index\_analyzer attribute for the > same field's mapping, and even if you specify a stopword-less > search\_analyzer for every field, stopwords are not excluded unless the > query passes a blank stopword list. > > Surely there must be other people who want to remove stopwords or change > them globally? > > On Monday, August 6, 2012 10:26:45 AM UTC-4, David Pilato wrote: > > > You can define your own mapping and use a standard analyzer with an > > empty stopwords collection. > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/analysis/standard-analyzer.html) > > > > I suppose it could work. I never tested an empty stopword list. > > > > HTH > > > > David > > > > Le 6 août 2012 à 15:52, John Ohno [john.ohno@gmail.com](mailto:john.ohno@gmail.com) a écrit : > > > > We are trying to make Elastic work under constraints inherited from an > > older system that didn't support stopwords. We tried setting the stopword > > list in JSON to a null string both at index creation time and at search > > time, but we have discovered that this doesn't allow one to search the > > index for english stopwords. Is there a straightforward way to turn off > > stopwords globally, aside from diking out the code that handles them? > > Alternately, were we to use the method of setting the stopword list to a > > null string, would there be yet a third place to set the stopword list in > > order to suppress this behavior? > > > > -- > > David Pilato > > [http://www.scrutmydocs.org/](http://www.scrutmydocs.org/) > > [http://dev.david.pilato.fr/](http://dev.david.pilato.fr/) > > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs ---
**Author:** ![John\_Ohno](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/john_ohno/32/1746_2.png) [@John\_Ohno](https://discuss.elastic.co/u/John_Ohno)\ **Post date:** [August 6, 2012, 5:37pm UTC](https://discuss.elastic.co/t/stopword-functionality-suppression/8646/5 "2012-08-06T17:37:02Z")
This looks like exactly what I'm looking for. Thanks! On Monday, August 6, 2012 12:59:37 PM UTC-4, Igor Motov wrote: > You can override default analyzer globally by adding these lines to > elasticsearch.yml: > > index.analysis.analyzer.default: > type: custom > tokenizer: standard > filter: standard, lowercase > > This will create a custom analyzer with Standard Tokenized and two > filters: Standard and Lowercase. This way your custom analyzer will be > identical to the standard analyzer but it will not use the stopword filter. > Because it's named "default", elasticsearch will use it everywhere where > analyzer is not explicitly set. > > Igor > > On Monday, August 6, 2012 12:35:02 PM UTC-4, John Ohno wrote: > > > I finally got that to work, but it turned out in order to do it, you have > > to specify the custom analyzer for the index and for every field in the > > mapping individually, then add a blank stopwords list to the query at > > search-time. > > > > While this is feasible for me (I generate the searches, the mappings, and > > the index requests), it seems a bit strange that specifying it top-level > > doesn't trickle-down. For instance, the analyzer attribute for the field > > mapping does not appear to cascade to the index\_analyzer attribute for the > > same field's mapping, and even if you specify a stopword-less > > search\_analyzer for every field, stopwords are not excluded unless the > > query passes a blank stopword list. > > > > Surely there must be other people who want to remove stopwords or change > > them globally? > > > > On Monday, August 6, 2012 10:26:45 AM UTC-4, David Pilato wrote: > > > > > You can define your own mapping and use a standard analyzer with an > > > empty stopwords collection. > > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/analysis/standard-analyzer.html) > > > > > > I suppose it could work. I never tested an empty stopword list. > > > > > > HTH > > > > > > David > > > > > > Le 6 août 2012 à 15:52, John Ohno [john.ohno@gmail.com](mailto:john.ohno@gmail.com) a écrit : > > > > > > We are trying to make Elastic work under constraints inherited from an > > > older system that didn't support stopwords. We tried setting the stopword > > > list in JSON to a null string both at index creation time and at search > > > time, but we have discovered that this doesn't allow one to search the > > > index for english stopwords. Is there a straightforward way to turn off > > > stopwords globally, aside from diking out the code that handles them? > > > Alternately, were we to use the method of setting the stopword list to a > > > null string, would there be yet a third place to set the stopword list in > > > order to suppress this behavior? > > > > > > -- > > > David Pilato > > > [http://www.scrutmydocs.org/](http://www.scrutmydocs.org/) > > > [http://dev.david.pilato.fr/](http://dev.david.pilato.fr/) > > > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs ---
**Author:** ![Schmurfy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/schmurfy/32/2763_2.png) [@Schmurfy](https://discuss.elastic.co/u/Schmurfy)\ **Post date:** [August 14, 2012, 2:27pm UTC](https://discuss.elastic.co/t/stopword-functionality-suppression/8646/6 "2012-08-14T14:27:18Z")
Hi, I was looking for exactly the same thing but I have no idea what to put in the config file to keep the default behavior but with an empty stopwords list. Could you post what you ended up adding in your config file please ? Thanks. On Monday, 6 August 2012 19:37:02 UTC+2, John Ohno wrote: > This looks like exactly what I'm looking for. Thanks! > > On Monday, August 6, 2012 12:59:37 PM UTC-4, Igor Motov wrote: > > > You can override default analyzer globally by adding these lines to > > elasticsearch.yml: > > > > index.analysis.analyzer.default: > > type: custom > > tokenizer: standard > > filter: standard, lowercase > > > > This will create a custom analyzer with Standard Tokenized and two > > filters: Standard and Lowercase. This way your custom analyzer will be > > identical to the standard analyzer but it will not use the stopword filter. > > Because it's named "default", elasticsearch will use it everywhere where > > analyzer is not explicitly set. > > > > Igor > > > > On Monday, August 6, 2012 12:35:02 PM UTC-4, John Ohno wrote: > > > > > I finally got that to work, but it turned out in order to do it, you > > > have to specify the custom analyzer for the index and for every field in > > > the mapping individually, then add a blank stopwords list to the query at > > > search-time. > > > > > > While this is feasible for me (I generate the searches, the mappings, > > > and the index requests), it seems a bit strange that specifying it > > > top-level doesn't trickle-down. For instance, the analyzer attribute for > > > the field mapping does not appear to cascade to the index\_analyzer > > > attribute for the same field's mapping, and even if you specify a > > > stopword-less search\_analyzer for every field, stopwords are not excluded > > > unless the query passes a blank stopword list. > > > > > > Surely there must be other people who want to remove stopwords or change > > > them globally? > > > > > > On Monday, August 6, 2012 10:26:45 AM UTC-4, David Pilato wrote: > > > > > > > You can define your own mapping and use a standard analyzer with an > > > > empty stopwords collection. > > > > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/analysis/standard-analyzer.html) > > > > > > > > I suppose it could work. I never tested an empty stopword list. > > > > > > > > HTH > > > > > > > > David > > > > > > > > Le 6 août 2012 à 15:52, John Ohno \<[john...@gmail.com](mailto:john...@gmail.com) \\> a > > > > écrit : > > > > > > > > We are trying to make Elastic work under constraints inherited from > > > > an older system that didn't support stopwords. We tried setting the > > > > stopword list in JSON to a null string both at index creation time and at > > > > search time, but we have discovered that this doesn't allow one to search > > > > the index for english stopwords. Is there a straightforward way to turn off > > > > stopwords globally, aside from diking out the code that handles them? > > > > Alternately, were we to use the method of setting the stopword list to a > > > > null string, would there be yet a third place to set the stopword list in > > > > order to suppress this behavior? > > > > > > > > -- > > > > David Pilato > > > > [http://www.scrutmydocs.org/](http://www.scrutmydocs.org/) > > > > [http://dev.david.pilato.fr/](http://dev.david.pilato.fr/) > > > > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs -- ---
**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\ **Post date:** [August 14, 2012, 9:46pm UTC](https://discuss.elastic.co/t/stopword-functionality-suppression/8646/7 "2012-08-14T21:46:39Z")
Hi Julien, The fragment in my post is exactly what you need to add to the elasticsearch.yml file: index.analysis.analyzer.default: type: custom tokenizer: standard filter: standard, lowercase Igor On Tuesday, August 14, 2012 10:27:18 AM UTC-4, Julien Ammous wrote: > Hi, I was looking for exactly the same thing but I have no idea what to > put in the config file to keep the default behavior but with an empty > stopwords list. > Could you post what you ended up adding in your config file please ? > > Thanks. > > On Monday, 6 August 2012 19:37:02 UTC+2, John Ohno wrote: > > > This looks like exactly what I'm looking for. Thanks! > > > > On Monday, August 6, 2012 12:59:37 PM UTC-4, Igor Motov wrote: > > > > > You can override default analyzer globally by adding these lines to > > > elasticsearch.yml: > > > > > > index.analysis.analyzer.default: > > > type: custom > > > tokenizer: standard > > > filter: standard, lowercase > > > > > > This will create a custom analyzer with Standard Tokenized and two > > > filters: Standard and Lowercase. This way your custom analyzer will be > > > identical to the standard analyzer but it will not use the stopword filter. > > > Because it's named "default", elasticsearch will use it everywhere where > > > analyzer is not explicitly set. > > > > > > Igor > > > > > > On Monday, August 6, 2012 12:35:02 PM UTC-4, John Ohno wrote: > > > > > > > I finally got that to work, but it turned out in order to do it, you > > > > have to specify the custom analyzer for the index and for every field in > > > > the mapping individually, then add a blank stopwords list to the query at > > > > search-time. > > > > > > > > While this is feasible for me (I generate the searches, the mappings, > > > > and the index requests), it seems a bit strange that specifying it > > > > top-level doesn't trickle-down. For instance, the analyzer attribute for > > > > the field mapping does not appear to cascade to the index\_analyzer > > > > attribute for the same field's mapping, and even if you specify a > > > > stopword-less search\_analyzer for every field, stopwords are not excluded > > > > unless the query passes a blank stopword list. > > > > > > > > Surely there must be other people who want to remove stopwords or > > > > change them globally? > > > > > > > > On Monday, August 6, 2012 10:26:45 AM UTC-4, David Pilato wrote: > > > > > > > > > You can define your own mapping and use a standard analyzer with an > > > > > empty stopwords collection. > > > > > > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/analysis/standard-analyzer.html) > > > > > > > > > > I suppose it could work. I never tested an empty stopword list. > > > > > > > > > > HTH > > > > > > > > > > David > > > > > > > > > > Le 6 août 2012 à 15:52, John Ohno [john...@gmail.com](mailto:john...@gmail.com) a écrit : > > > > > > > > > > We are trying to make Elastic work under constraints inherited from > > > > > an older system that didn't support stopwords. We tried setting the > > > > > stopword list in JSON to a null string both at index creation time and at > > > > > search time, but we have discovered that this doesn't allow one to search > > > > > the index for english stopwords. Is there a straightforward way to turn off > > > > > stopwords globally, aside from diking out the code that handles them? > > > > > Alternately, were we to use the method of setting the stopword list to a > > > > > null string, would there be yet a third place to set the stopword list in > > > > > order to suppress this behavior? > > > > > > > > > > -- > > > > > David Pilato > > > > > [http://www.scrutmydocs.org/](http://www.scrutmydocs.org/) > > > > > [http://dev.david.pilato.fr/](http://dev.david.pilato.fr/) > > > > > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs -- ---
**Author:** ![Schmurfy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/schmurfy/32/2763_2.png) [@Schmurfy](https://discuss.elastic.co/u/Schmurfy)\ **Post date:** [August 16, 2012, 11:53am UTC](https://discuss.elastic.co/t/stopword-functionality-suppression/8646/8 "2012-08-16T11:53:57Z")
Thanks ! On 14 August 2012 23:46, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com) wrote: > Hi Julien, > > The fragment in my post is exactly what you need to add to the > elasticsearch.yml file: > > index.analysis.analyzer.\*\*default: > type: custom > tokenizer: standard > filter: standard, lowercase > > Igor > > On Tuesday, August 14, 2012 10:27:18 AM UTC-4, Julien Ammous wrote: > > > Hi, I was looking for exactly the same thing but I have no idea what to > > put in the config file to keep the default behavior but with an empty > > stopwords list. > > Could you post what you ended up adding in your config file please ? > > > > Thanks. > > > > On Monday, 6 August 2012 19:37:02 UTC+2, John Ohno wrote: > > > > > This looks like exactly what I'm looking for. Thanks! > > > > > > On Monday, August 6, 2012 12:59:37 PM UTC-4, Igor Motov wrote: > > > > > > > You can override default analyzer globally by adding these lines to > > > > elasticsearch.yml: > > > > > > > > index.analysis.analyzer.\*\*default: > > > > type: custom > > > > tokenizer: standard > > > > filter: standard, lowercase > > > > > > > > This will create a custom analyzer with Standard Tokenized and two > > > > filters: Standard and Lowercase. This way your custom analyzer will be > > > > identical to the standard analyzer but it will not use the stopword filter. > > > > Because it's named "default", elasticsearch will use it everywhere where > > > > analyzer is not explicitly set. > > > > > > > > Igor > > > > > > > > On Monday, August 6, 2012 12:35:02 PM UTC-4, John Ohno wrote: > > > > > > > > > I finally got that to work, but it turned out in order to do it, you > > > > > have to specify the custom analyzer for the index and for every field in > > > > > the mapping individually, then add a blank stopwords list to the query at > > > > > search-time. > > > > > > > > > > While this is feasible for me (I generate the searches, the mappings, > > > > > and the index requests), it seems a bit strange that specifying it > > > > > top-level doesn't trickle-down. For instance, the analyzer attribute for > > > > > the field mapping does not appear to cascade to the index\_analyzer > > > > > attribute for the same field's mapping, and even if you specify a > > > > > stopword-less search\_analyzer for every field, stopwords are not excluded > > > > > unless the query passes a blank stopword list. > > > > > > > > > > Surely there must be other people who want to remove stopwords or > > > > > change them globally? > > > > > > > > > > On Monday, August 6, 2012 10:26:45 AM UTC-4, David Pilato wrote: > > > > > > > > > > > You can define your own mapping and use a standard analyzer with an > > > > > > empty stopwords collection. > > > > > > > > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/**guide/reference/index-modules/)\*\* > > > > > > analysis/standard-analyzer.\*\*html[http://www.elasticsearch.org/guide/reference/index-modules/analysis/standard-analyzer.html](http://www.elasticsearch.org/guide/reference/index-modules/analysis/standard-analyzer.html) > > > > > > > > > > > > I suppose it could work. I never tested an empty stopword list. > > > > > > > > > > > > HTH > > > > > > > > > > > > David > > > > > > > > > > > > Le 6 août 2012 à 15:52, John Ohno [john...@gmail.com](mailto:john...@gmail.com) a écrit : > > > > > > > > > > > > We are trying to make Elastic work under constraints inherited from > > > > > > an older system that didn't support stopwords. We tried setting the > > > > > > stopword list in JSON to a null string both at index creation time and at > > > > > > search time, but we have discovered that this doesn't allow one to search > > > > > > the index for english stopwords. Is there a straightforward way to turn off > > > > > > stopwords globally, aside from diking out the code that handles them? > > > > > > Alternately, were we to use the method of setting the stopword list to a > > > > > > null string, would there be yet a third place to set the stopword list in > > > > > > order to suppress this behavior? > > > > > > > > > > > > -- > > > > > > David Pilato > > > > > > [http://www.scrutmydocs.org/](http://www.scrutmydocs.org/) > > > > > > [http://dev.david.pilato.fr/](http://dev.david.pilato.fr/) > > > > > > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > > > > > > > > > -- -- ---
**Author:** ![brian\_yoder](https://avatars.discourse-cdn.com/v4/letter/b/f1d935/32.png) [@brian\_yoder](https://discuss.elastic.co/u/brian_yoder)\ **Post date:** [November 1, 2012, 9:29pm UTC](https://discuss.elastic.co/t/stopword-functionality-suppression/8646/9 "2012-11-01T21:29:47Z")
I added these lines to my elasticsearch.yml configuration file, and restarted Elasticsearch. But still no joy. When I try to use the default (or standard) analyzer, it still throws away the built-in English stop words. For example: curl -XGET 'localhost:9200/\_analyze?analyzer=default&pretty=true' -d 'This is a Test' && echo { "tokens" : [ { "token" : "test", "start\_offset" : 10, "end\_offset" : 14, "type" : "", "position" : 4 } ] } So close, and yet so far. The database I'm creating consists entirely of names (people, businesses, places) and stop words just get in the way. A J FOYT's first initial should not be considered a stop word: curl -XGET 'localhost:9200/\_analyze?analyzer=default&pretty=true' -d 'A J Foyt' && echo { "tokens" : [ { "token" : "j", "start\_offset" : 2, "end\_offset" : 3, "type" : "", "position" : 2 }, { "token" : "foyt", "start\_offset" : 4, "end\_offset" : 8, "type" : "", "position" : 3 } ] } On Monday, August 6, 2012 12:59:37 PM UTC-4, Igor Motov wrote: > You can override default analyzer globally by adding these lines to > elasticsearch.yml: > > index.analysis.analyzer.default: > type: custom > tokenizer: standard > filter: standard, lowercase > > This will create a custom analyzer with Standard Tokenized and two > filters: Standard and Lowercase. This way your custom analyzer will be > identical to the standard analyzer but it will not use the stopword filter. > Because it's named "default", elasticsearch will use it everywhere where > analyzer is not explicitly set. > > Igor > > On Monday, August 6, 2012 12:35:02 PM UTC-4, John Ohno wrote: > > > I finally got that to work, but it turned out in order to do it, you have > > to specify the custom analyzer for the index and for every field in the > > mapping individually, then add a blank stopwords list to the query at > > search-time. > > > > While this is feasible for me (I generate the searches, the mappings, and > > the index requests), it seems a bit strange that specifying it top-level > > doesn't trickle-down. For instance, the analyzer attribute for the field > > mapping does not appear to cascade to the index\_analyzer attribute for the > > same field's mapping, and even if you specify a stopword-less > > search\_analyzer for every field, stopwords are not excluded unless the > > query passes a blank stopword list. > > > > Surely there must be other people who want to remove stopwords or change > > them globally? > > > > On Monday, August 6, 2012 10:26:45 AM UTC-4, David Pilato wrote: > > > > > You can define your own mapping and use a standard analyzer with an > > > empty stopwords collection. > > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/analysis/standard-analyzer.html) > > > > > > I suppose it could work. I never tested an empty stopword list. > > > > > > HTH > > > > > > David > > > > > > Le 6 août 2012 à 15:52, John Ohno \<[john...@gmail.com](mailto:john...@gmail.com) \\> a > > > écrit : > > > > > > We are trying to make Elastic work under constraints inherited from an > > > older system that didn't support stopwords. We tried setting the stopword > > > list in JSON to a null string both at index creation time and at search > > > time, but we have discovered that this doesn't allow one to search the > > > index for english stopwords. Is there a straightforward way to turn off > > > stopwords globally, aside from diking out the code that handles them? > > > Alternately, were we to use the method of setting the stopword list to a > > > null string, would there be yet a third place to set the stopword list in > > > order to suppress this behavior? > > > > > > -- > > > David Pilato > > > [http://www.scrutmydocs.org/](http://www.scrutmydocs.org/) > > > [http://dev.david.pilato.fr/](http://dev.david.pilato.fr/) > > > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs -- ---
**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\ **Post date:** [November 2, 2012, 1:14pm UTC](https://discuss.elastic.co/t/stopword-functionality-suppression/8646/10 "2012-11-02T13:14:09Z")
If you don't specify an index name in the analyze API request, it can only work with built-in analyzers. So, when you are asking for "default" analyzer it returns you built-in default analyzer instead of one that you configured in the config file. Try using \_analyze with an index: curl -XGET 'localhost:9200/my\_index/\_analyze?analyzer=default&pretty=true' -d 'This is a Test' && echo On Thursday, November 1, 2012 5:29:48 PM UTC-4, InquiringMind wrote: > I added these lines to my elasticsearch.yml configuration file, and > restarted Elasticsearch. > > But still no joy. When I try to use the default (or standard) analyzer, it > still throws away the built-in English stop words. For example: > > curl -XGET 'localhost:9200/\_analyze?analyzer=default&pretty=true' -d 'This > is a Test' && echo > { > "tokens" : [ { > "token" : "test", > "start\_offset" : 10, > "end\_offset" : 14, > "type" : "", > "position" : 4 > } ] > } > > So close, and yet so far. The database I'm creating consists entirely of > names (people, businesses, places) and stop words just get in the way. A J > FOYT's first initial should not be considered a stop word: > > curl -XGET 'localhost:9200/\_analyze?analyzer=default&pretty=true' -d 'A J > Foyt' && echo > { > "tokens" : [ { > "token" : "j", > "start\_offset" : 2, > "end\_offset" : 3, > "type" : "", > "position" : 2 > }, { > "token" : "foyt", > "start\_offset" : 4, > "end\_offset" : 8, > "type" : "", > "position" : 3 > } ] > } > > On Monday, August 6, 2012 12:59:37 PM UTC-4, Igor Motov wrote: > > > You can override default analyzer globally by adding these lines to > > elasticsearch.yml: > > > > index.analysis.analyzer.default: > > type: custom > > tokenizer: standard > > filter: standard, lowercase > > > > This will create a custom analyzer with Standard Tokenized and two > > filters: Standard and Lowercase. This way your custom analyzer will be > > identical to the standard analyzer but it will not use the stopword filter. > > Because it's named "default", elasticsearch will use it everywhere where > > analyzer is not explicitly set. > > > > Igor > > > > On Monday, August 6, 2012 12:35:02 PM UTC-4, John Ohno wrote: > > > > > I finally got that to work, but it turned out in order to do it, you > > > have to specify the custom analyzer for the index and for every field in > > > the mapping individually, then add a blank stopwords list to the query at > > > search-time. > > > > > > While this is feasible for me (I generate the searches, the mappings, > > > and the index requests), it seems a bit strange that specifying it > > > top-level doesn't trickle-down. For instance, the analyzer attribute for > > > the field mapping does not appear to cascade to the index\_analyzer > > > attribute for the same field's mapping, and even if you specify a > > > stopword-less search\_analyzer for every field, stopwords are not excluded > > > unless the query passes a blank stopword list. > > > > > > Surely there must be other people who want to remove stopwords or change > > > them globally? > > > > > > On Monday, August 6, 2012 10:26:45 AM UTC-4, David Pilato wrote: > > > > > > > You can define your own mapping and use a standard analyzer with an > > > > empty stopwords collection. > > > > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/analysis/standard-analyzer.html) > > > > > > > > I suppose it could work. I never tested an empty stopword list. > > > > > > > > HTH > > > > > > > > David > > > > > > > > Le 6 août 2012 à 15:52, John Ohno [john...@gmail.com](mailto:john...@gmail.com) a écrit : > > > > > > > > We are trying to make Elastic work under constraints inherited from > > > > an older system that didn't support stopwords. We tried setting the > > > > stopword list in JSON to a null string both at index creation time and at > > > > search time, but we have discovered that this doesn't allow one to search > > > > the index for english stopwords. Is there a straightforward way to turn off > > > > stopwords globally, aside from diking out the code that handles them? > > > > Alternately, were we to use the method of setting the stopword list to a > > > > null string, would there be yet a third place to set the stopword list in > > > > order to suppress this behavior? > > > > > > > > -- > > > > David Pilato > > > > [http://www.scrutmydocs.org/](http://www.scrutmydocs.org/) > > > > [http://dev.david.pilato.fr/](http://dev.david.pilato.fr/) > > > > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs -- ---
**Author:** ![brian\_yoder](https://avatars.discourse-cdn.com/v4/letter/b/f1d935/32.png) [@brian\_yoder](https://discuss.elastic.co/u/brian_yoder)\ **Post date:** [November 2, 2012, 3:57pm UTC](https://discuss.elastic.co/t/stopword-functionality-suppression/8646/11 "2012-11-02T15:57:30Z")
Thanks, Igor! That was exactly what I needed to do! Adding the index name caused the configured default to be used and no more stop words! Then deleting and loading the data re-created the index (it fails, of course, if the specified index doesn't exist yet), and now the queries (which I've directed to analyzer("default") just in case it needs some help! also find those stop words as separate terms (and not just within phrases). Very cool! On Friday, November 2, 2012 9:14:09 AM UTC-4, Igor Motov wrote: > If you don't specify an index name in the analyze API request, it can only > work with built-in analyzers. So, when you are asking for "default" > analyzer it returns you built-in default analyzer instead of one that you > configured in the config file. Try using \_analyze with an index: > > curl -XGET 'localhost:9200/my\_index/\_analyze?analyzer=default&pretty=true' > -d 'This is a Test' && echo -- ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:06am UTC](https://discuss.elastic.co/t/stopword-functionality-suppression/8646/12 "2017-07-06T03:06:25Z")