# Prevent some words from being "stopped" **URL:** \ **Category:** Elasticsearch\ **Created:** [May 19, 2013, 8:55pm UTC](https://discuss.elastic.co/t/prevent-some-words-from-being-stopped/12030 "2013-05-19T20:55:06Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![anon44344346](https://avatars.discourse-cdn.com/v4/letter/a/e56c9b/32.png) [@anon44344346](https://discuss.elastic.co/u/anon44344346)\ **Post date:** [May 19, 2013, 8:55pm UTC](https://discuss.elastic.co/t/prevent-some-words-from-being-stopped/12030/1 "2013-05-19T20:55:06Z")
Is there any way of "marking" a word not to be filtered out by the stop filter, even if it's on the stop words?? I'm looking for something like "keywordmarker" filter, which "marks" a keyword not to be stemmed afterwards. Use case. Some customers have a special meaning from just one or two words belonging to the standard stop words list of their language. Say, for a spanish customer, he want the word "tenga" to be searchable even if it belongs to the "_es_" expanded stopwords. I know I could just override the whole stopwords set, and provide a custom stopwords list with "tenga" not in it, and I just did that, but it would be "cheaper" instead of reespecifying the whole set, to say "exception"s: ["tenga"] On a sidenote: where can I find the default list that elasticsearch uses when specifying ["_es_"] as stopwords??? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![simonw\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simonw_2/32/1130_2.png) [@simonw\_2](https://discuss.elastic.co/u/simonw_2)\ **Post date:** [May 19, 2013, 9:38pm UTC](https://discuss.elastic.co/t/prevent-some-words-from-being-stopped/12030/2 "2013-05-19T21:38:13Z")
you might want to look at the `cutoff_frequency` that was added to the match\_query ([Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/query-dsl/match-query/)) It might enable you to not filter stopwords at all. Another option is to create a stopword list per customer and add it to the mapping yourself, how are you creating your stopword filters right now? simon On Sunday, May 19, 2013 10:55:06 PM UTC+2, JoeZ99 wrote: > Is there any way of "marking" a word not to be filtered out by the stop > filter, even if it's on the stop words?? > > I'm looking for something like "keywordmarker" filter, which "marks" a > keyword not to be stemmed afterwards. > > Use case. Some customers have a special meaning from just one or two words > belonging to the standard stop words list of their language. Say, for a > spanish customer, he want the word "tenga" to be searchable even if it > belongs to the "_es_" expanded stopwords. > > I know I could just override the whole stopwords set, and provide a custom > stopwords list with "tenga" not in it, and I just did that, but it would be > "cheaper" instead of reespecifying the whole set, to say "exception"s: > ["tenga"] > > On a sidenote: where can I find the default list that elasticsearch uses > when specifying ["_es_"] as stopwords??? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![anon44344346](https://avatars.discourse-cdn.com/v4/letter/a/e56c9b/32.png) [@anon44344346](https://discuss.elastic.co/u/anon44344346)\ **Post date:** [May 20, 2013, 5:58am UTC](https://discuss.elastic.co/t/prevent-some-words-from-being-stopped/12030/3 "2013-05-20T05:58:53Z")
I didn't know about that feature. is it a new thing? anyway, I think other considerations regarding how we build the query before hitting the search server may prevent us for taking that road. but I gotta take a deeper look into it. as how we use stopwords, it's pretty standard: "index\_name": { "index": { ... ... "analysis": { "analyzer": { "default\_index": { "tokenizer": "standard", "filter": ["standard", "lowercase", "my\_stop", ...] }, "default\_search": { "tokenizer": "standard", "filter": ["standard", "lowercase", "my\_stop", ...] }, }, "filter": { ... "my\_stop":{ "type": "stop", "stopwords": ["_es_"] }, ... } } } }, do you happen to know where the default stopwords are defined? On Sunday, May 19, 2013 5:38:13 PM UTC-4, simonw wrote: > you might want to look at the `cutoff_frequency` that was added to the > match\_query ( > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/query-dsl/match-query/)) It > might enable you to not filter stopwords at all. Another option is to > create a stopword list per customer and add it to the mapping yourself, how > are you creating your stopword filters right now? > > simon > > On Sunday, May 19, 2013 10:55:06 PM UTC+2, JoeZ99 wrote: > > > Is there any way of "marking" a word not to be filtered out by the stop > > filter, even if it's on the stop words?? > > > > I'm looking for something like "keywordmarker" filter, which "marks" a > > keyword not to be stemmed afterwards. > > > > Use case. Some customers have a special meaning from just one or two > > words belonging to the standard stop words list of their language. Say, for > > a spanish customer, he want the word "tenga" to be searchable even if it > > belongs to the "_es_" expanded stopwords. > > > > I know I could just override the whole stopwords set, and provide a > > custom stopwords list with "tenga" not in it, and I just did that, but it > > would be "cheaper" instead of reespecifying the whole set, to say > > "exception"s: ["tenga"] > > > > On a sidenote: where can I find the default list that elasticsearch uses > > when specifying ["_es_"] as stopwords??? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![simonw\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simonw_2/32/1130_2.png) [@simonw\_2](https://discuss.elastic.co/u/simonw_2)\ **Post date:** [May 20, 2013, 6:53pm UTC](https://discuss.elastic.co/t/prevent-some-words-from-being-stopped/12030/4 "2013-05-20T18:53:02Z")
On Monday, May 20, 2013 7:58:53 AM UTC+2, JoeZ99 wrote: > I didn't know about that feature. is it a new thing? yes it's pretty new. 0.90 only. > anyway, I think other considerations regarding how we build the query > before hitting the search server may prevent us for taking that road. but I > gotta take a deeper look into it. this feature makes your live easier not harder so going don't that road should be downhill. It essentially means you don't filter stopwords anymore and gain the ability to run stopword only searches fast as well. stuff like "to be or not to be" etc. > as how we use stopwords, it's pretty standard: ok so why don't you just modify the stopword list if a customer asks for it? I assume you have an index per customer? simon > "index\_name": { > "index": { > ... > ... > "analysis": { > "analyzer": { > "default\_index": { > "tokenizer": "standard", > "filter": ["standard", "lowercase", "my\_stop", ...] > }, > "default\_search": { > "tokenizer": "standard", > "filter": ["standard", "lowercase", "my\_stop", ...] > }, > }, > "filter": { > ... > "my\_stop":{ > "type": "stop", > "stopwords": ["_es_"] > }, > ... > } > } > } > }, > > do you happen to know where the default stopwords are defined? > > On Sunday, May 19, 2013 5:38:13 PM UTC-4, simonw wrote: > > > you might want to look at the `cutoff_frequency` that was added to the > > match\_query ( > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/query-dsl/match-query/)) It > > might enable you to not filter stopwords at all. Another option is to > > create a stopword list per customer and add it to the mapping yourself, how > > are you creating your stopword filters right now? > > > > simon > > > > On Sunday, May 19, 2013 10:55:06 PM UTC+2, JoeZ99 wrote: > > > > > Is there any way of "marking" a word not to be filtered out by the stop > > > filter, even if it's on the stop words?? > > > > > > I'm looking for something like "keywordmarker" filter, which "marks" a > > > keyword not to be stemmed afterwards. > > > > > > Use case. Some customers have a special meaning from just one or two > > > words belonging to the standard stop words list of their language. Say, for > > > a spanish customer, he want the word "tenga" to be searchable even if it > > > belongs to the "_es_" expanded stopwords. > > > > > > I know I could just override the whole stopwords set, and provide a > > > custom stopwords list with "tenga" not in it, and I just did that, but it > > > would be "cheaper" instead of reespecifying the whole set, to say > > > "exception"s: ["tenga"] > > > > > > On a sidenote: where can I find the default list that elasticsearch uses > > > when specifying ["_es_"] as stopwords??? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![anon44344346](https://avatars.discourse-cdn.com/v4/letter/a/e56c9b/32.png) [@anon44344346](https://discuss.elastic.co/u/anon44344346)\ **Post date:** [May 21, 2013, 2:12pm UTC](https://discuss.elastic.co/t/prevent-some-words-from-being-stopped/12030/5 "2013-05-21T14:12:14Z")
On Monday, May 20, 2013 2:53:02 PM UTC-4, simonw wrote: > On Monday, May 20, 2013 7:58:53 AM UTC+2, JoeZ99 wrote: > > > I didn't know about that feature. is it a new thing? > > yes it's pretty new. 0.90 only. > > > anyway, I think other considerations regarding how we build the query > > before hitting the search server may prevent us for taking that road. but I > > gotta take a deeper look into it. > > this feature makes your live easier not harder so going don't that road > should be downhill. It essentially means you don't filter stopwords anymore > and gain the ability to run stopword only searches fast as well. stuff like > "to be or not to be" etc. Well, I'm definitely going to check that out . Can I do that "cutoff" thing in a query that is not a "match query". I understood I couldn't (that's why I'm not sure I can use it, because I'm not sure I can use match query altogether). > > as how we use stopwords, it's pretty standard: > > ok so why don't you just modify the stopword list if a customer asks for > it? I assume you have an index per customer? > > simon > > > Of course I've got an index per customer, and of course that's what I'm doing now (that's why I want to know where the "standard" stopwords are defined, so I can copy/paste them without a few ones the customer doesn't want), replacing the entire stopwords collection for a custom-made one. I just thought it would be easiest to define the two stopwords that "shouldn't be there" than redeclare the whole stopwords bunch. > > "index\_name": { > > "index": { > > ... > > ... > > "analysis": { > > "analyzer": { > > "default\_index": { > > "tokenizer": "standard", > > "filter": ["standard", "lowercase", "my\_stop", > > ...] > > }, > > "default\_search": { > > "tokenizer": "standard", > > "filter": ["standard", "lowercase", "my\_stop", > > ...] > > }, > > }, > > "filter": { > > ... > > "my\_stop":{ > > "type": "stop", > > "stopwords": ["_es_"] > > }, > > ... > > } > > } > > } > > }, > > > > do you happen to know where the default stopwords are defined? > > > > On Sunday, May 19, 2013 5:38:13 PM UTC-4, simonw wrote: > > > > > you might want to look at the `cutoff_frequency` that was added to the > > > match\_query ( > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/query-dsl/match-query/)) It > > > might enable you to not filter stopwords at all. Another option is to > > > create a stopword list per customer and add it to the mapping yourself, how > > > are you creating your stopword filters right now? > > > > > > simon > > > > > > On Sunday, May 19, 2013 10:55:06 PM UTC+2, JoeZ99 wrote: > > > > > > > Is there any way of "marking" a word not to be filtered out by the stop > > > > filter, even if it's on the stop words?? > > > > > > > > I'm looking for something like "keywordmarker" filter, which "marks" a > > > > keyword not to be stemmed afterwards. > > > > > > > > Use case. Some customers have a special meaning from just one or two > > > > words belonging to the standard stop words list of their language. Say, for > > > > a spanish customer, he want the word "tenga" to be searchable even if it > > > > belongs to the "_es_" expanded stopwords. > > > > > > > > I know I could just override the whole stopwords set, and provide a > > > > custom stopwords list with "tenga" not in it, and I just did that, but it > > > > would be "cheaper" instead of reespecifying the whole set, to say > > > > "exception"s: ["tenga"] > > > > > > > > On a sidenote: where can I find the default list that elasticsearch > > > > uses when specifying ["_es_"] as stopwords??? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![simonw\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simonw_2/32/1130_2.png) [@simonw\_2](https://discuss.elastic.co/u/simonw_2)\ **Post date:** [May 21, 2013, 7:40pm UTC](https://discuss.elastic.co/t/prevent-some-words-from-being-stopped/12030/6 "2013-05-21T19:40:20Z")
On Tuesday, May 21, 2013 4:12:14 PM UTC+2, JoeZ99 wrote: > On Monday, May 20, 2013 2:53:02 PM UTC-4, simonw wrote: > > > On Monday, May 20, 2013 7:58:53 AM UTC+2, JoeZ99 wrote: > > > > > I didn't know about that feature. is it a new thing? > > > > yes it's pretty new. 0.90 only. > > > > > anyway, I think other considerations regarding how we build the query > > > before hitting the search server may prevent us for taking that road. but I > > > gotta take a deeper look into it. > > > > this feature makes your live easier not harder so going don't that road > > should be downhill. It essentially means you don't filter stopwords anymore > > and gain the ability to run stopword only searches fast as well. stuff like > > "to be or not to be" etc. > > Well, I'm definitely going to check that out . Can I do that "cutoff" > thing in a query that is not a "match query". I understood I couldn't > (that's why I'm not sure I can use it, because I'm not sure I can use match > query altogether). yes there is a common\_terms query but apparently I never finished the documentation for this one. Hmm it's still sitting in my local clone. Lemme finish the documentation and update you once it's there. what query are you using right now? > > > as how we use stopwords, it's pretty standard: > > > > ok so why don't you just modify the stopword list if a customer asks for > > it? I assume you have an index per customer? > > > > simon > > > > > > > Of course I've got an index per customer, and of course that's what I'm > doing now (that's why I want to know where the "standard" stopwords are > defined, so I can copy/paste them without a few ones the customer doesn't > want), replacing the entire stopwords collection for a custom-made one. I > just thought it would be easiest to define the two stopwords that > "shouldn't be there" than redeclare the whole stopwords bunch. it seems like stop filter could support keyword attribute as well but I am not sure if that is really what we should do. seems like a unique requirement. usually you wanna specify a custom list not an exclusion set. simon > > > "index\_name": { > > > "index": { > > > ... > > > ... > > > "analysis": { > > > "analyzer": { > > > "default\_index": { > > > "tokenizer": "standard", > > > "filter": ["standard", "lowercase", "my\_stop", > > > ...] > > > }, > > > "default\_search": { > > > "tokenizer": "standard", > > > "filter": ["standard", "lowercase", "my\_stop", > > > ...] > > > }, > > > }, > > > "filter": { > > > ... > > > "my\_stop":{ > > > "type": "stop", > > > "stopwords": ["_es_"] > > > }, > > > ... > > > } > > > } > > > } > > > }, > > > > > > do you happen to know where the default stopwords are defined? > > > > > > On Sunday, May 19, 2013 5:38:13 PM UTC-4, simonw wrote: > > > > > > > you might want to look at the `cutoff_frequency` that was added to the > > > > match\_query ( > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/query-dsl/match-query/)) > > > > It might enable you to not filter stopwords at all. Another option is to > > > > create a stopword list per customer and add it to the mapping yourself, how > > > > are you creating your stopword filters right now? > > > > > > > > simon > > > > > > > > On Sunday, May 19, 2013 10:55:06 PM UTC+2, JoeZ99 wrote: > > > > > > > > > Is there any way of "marking" a word not to be filtered out by the > > > > > stop filter, even if it's on the stop words?? > > > > > > > > > > I'm looking for something like "keywordmarker" filter, which "marks" a > > > > > keyword not to be stemmed afterwards. > > > > > > > > > > Use case. Some customers have a special meaning from just one or two > > > > > words belonging to the standard stop words list of their language. Say, for > > > > > a spanish customer, he want the word "tenga" to be searchable even if it > > > > > belongs to the "_es_" expanded stopwords. > > > > > > > > > > I know I could just override the whole stopwords set, and provide a > > > > > custom stopwords list with "tenga" not in it, and I just did that, but it > > > > > would be "cheaper" instead of reespecifying the whole set, to say > > > > > "exception"s: ["tenga"] > > > > > > > > > > On a sidenote: where can I find the default list that elasticsearch > > > > > uses when specifying ["_es_"] as stopwords??? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![simonw\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simonw_2/32/1130_2.png) [@simonw\_2](https://discuss.elastic.co/u/simonw_2)\ **Post date:** [May 22, 2013, 3:43pm UTC](https://discuss.elastic.co/t/prevent-some-words-from-being-stopped/12030/7 "2013-05-22T15:43:34Z")
On Tuesday, May 21, 2013 9:40:20 PM UTC+2, simonw wrote: > On Tuesday, May 21, 2013 4:12:14 PM UTC+2, JoeZ99 wrote: > > > On Monday, May 20, 2013 2:53:02 PM UTC-4, simonw wrote: > > > > > On Monday, May 20, 2013 7:58:53 AM UTC+2, JoeZ99 wrote: > > > > > > > I didn't know about that feature. is it a new thing? > > > > > > yes it's pretty new. 0.90 only. > > > > > > > anyway, I think other considerations regarding how we build the query > > > > before hitting the search server may prevent us for taking that road. but I > > > > gotta take a deeper look into it. > > > > > > this feature makes your live easier not harder so going don't that road > > > should be downhill. It essentially means you don't filter stopwords anymore > > > and gain the ability to run stopword only searches fast as well. stuff like > > > "to be or not to be" etc. > > > > Well, I'm definitely going to check that out . Can I do that "cutoff" > > thing in a query that is not a "match query". I understood I couldn't > > (that's why I'm not sure I can use it, because I'm not sure I can use match > > query altogether). > > yes there is a common\_terms query but apparently I never finished the > documentation for this one. Hmm it's still sitting in my local clone. Lemme > finish the documentation and update you once it's there. what query are you > using right now? here you go: > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. simon > > > > as how we use stopwords, it's pretty standard: > > > > > > ok so why don't you just modify the stopword list if a customer asks for > > > it? I assume you have an index per customer? > > > > > > simon > > > > > > > > > > > Of course I've got an index per customer, and of course that's what I'm > > doing now (that's why I want to know where the "standard" stopwords are > > defined, so I can copy/paste them without a few ones the customer doesn't > > want), replacing the entire stopwords collection for a custom-made one. I > > just thought it would be easiest to define the two stopwords that > > "shouldn't be there" than redeclare the whole stopwords bunch. > > it seems like stop filter could support keyword attribute as well but I am > not sure if that is really what we should do. seems like a unique > requirement. usually you wanna specify a custom list not an exclusion set. > > simon > > > > > "index\_name": { > > > > "index": { > > > > ... > > > > ... > > > > "analysis": { > > > > "analyzer": { > > > > "default\_index": { > > > > "tokenizer": "standard", > > > > "filter": ["standard", "lowercase", "my\_stop", > > > > ...] > > > > }, > > > > "default\_search": { > > > > "tokenizer": "standard", > > > > "filter": ["standard", "lowercase", "my\_stop", > > > > ...] > > > > }, > > > > }, > > > > "filter": { > > > > ... > > > > "my\_stop":{ > > > > "type": "stop", > > > > "stopwords": ["_es_"] > > > > }, > > > > ... > > > > } > > > > } > > > > } > > > > }, > > > > > > > > do you happen to know where the default stopwords are defined? > > > > > > > > On Sunday, May 19, 2013 5:38:13 PM UTC-4, simonw wrote: > > > > > > > > > you might want to look at the `cutoff_frequency` that was added to the > > > > > match\_query ( > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/query-dsl/match-query/)) > > > > > It might enable you to not filter stopwords at all. Another option is to > > > > > create a stopword list per customer and add it to the mapping yourself, how > > > > > are you creating your stopword filters right now? > > > > > > > > > > simon > > > > > > > > > > On Sunday, May 19, 2013 10:55:06 PM UTC+2, JoeZ99 wrote: > > > > > > > > > > > Is there any way of "marking" a word not to be filtered out by the > > > > > > stop filter, even if it's on the stop words?? > > > > > > > > > > > > I'm looking for something like "keywordmarker" filter, which "marks" > > > > > > a keyword not to be stemmed afterwards. > > > > > > > > > > > > Use case. Some customers have a special meaning from just one or two > > > > > > words belonging to the standard stop words list of their language. Say, for > > > > > > a spanish customer, he want the word "tenga" to be searchable even if it > > > > > > belongs to the "_es_" expanded stopwords. > > > > > > > > > > > > I know I could just override the whole stopwords set, and provide a > > > > > > custom stopwords list with "tenga" not in it, and I just did that, but it > > > > > > would be "cheaper" instead of reespecifying the whole set, to say > > > > > > "exception"s: ["tenga"] > > > > > > > > > > > > On a sidenote: where can I find the default list that elasticsearch > > > > > > uses when specifying ["_es_"] as stopwords??? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![anon44344346](https://avatars.discourse-cdn.com/v4/letter/a/e56c9b/32.png) [@anon44344346](https://discuss.elastic.co/u/anon44344346)\ **Post date:** [May 22, 2013, 4:09pm UTC](https://discuss.elastic.co/t/prevent-some-words-from-being-stopped/12030/8 "2013-05-22T16:09:23Z")
On Tuesday, May 21, 2013 3:40:20 PM UTC-4, simonw wrote: > On Tuesday, May 21, 2013 4:12:14 PM UTC+2, JoeZ99 wrote: > > > On Monday, May 20, 2013 2:53:02 PM UTC-4, simonw wrote: > > > > > On Monday, May 20, 2013 7:58:53 AM UTC+2, JoeZ99 wrote: > > > > > > > I didn't know about that feature. is it a new thing? > > > > > > yes it's pretty new. 0.90 only. > > > > > > > anyway, I think other considerations regarding how we build the query > > > > before hitting the search server may prevent us for taking that road. but I > > > > gotta take a deeper look into it. > > > > > > this feature makes your live easier not harder so going don't that road > > > should be downhill. It essentially means you don't filter stopwords anymore > > > and gain the ability to run stopword only searches fast as well. stuff like > > > "to be or not to be" etc. > > > > Well, I'm definitely going to check that out . Can I do that "cutoff" > > thing in a query that is not a "match query". I understood I couldn't > > (that's why I'm not sure I can use it, because I'm not sure I can use match > > query altogether). > > yes there is a common\_terms query but apparently I never finished the > documentation for this one. Hmm it's still sitting in my local clone. Lemme > finish the documentation and update you once it's there. what query are you > using right now? I'm using either text, phrase\_prefix or, in some cases, fuzzy > > > > as how we use stopwords, it's pretty standard: > > > > > > ok so why don't you just modify the stopword list if a customer asks for > > > it? I assume you have an index per customer? > > > > > > simon > > > > > > > > > > > Of course I've got an index per customer, and of course that's what I'm > > doing now (that's why I want to know where the "standard" stopwords are > > defined, so I can copy/paste them without a few ones the customer doesn't > > want), replacing the entire stopwords collection for a custom-made one. I > > just thought it would be easiest to define the two stopwords that > > "shouldn't be there" than redeclare the whole stopwords bunch. > > it seems like stop filter could support keyword attribute as well but I am > not sure if that is really what we should do. seems like a unique > requirement. usually you wanna specify a custom list not an exclusion set. > > simon > > > > > "index\_name": { > > > > "index": { > > > > ... > > > > ... > > > > "analysis": { > > > > "analyzer": { > > > > "default\_index": { > > > > "tokenizer": "standard", > > > > "filter": ["standard", "lowercase", "my\_stop", > > > > ...] > > > > }, > > > > "default\_search": { > > > > "tokenizer": "standard", > > > > "filter": ["standard", "lowercase", "my\_stop", > > > > ...] > > > > }, > > > > }, > > > > "filter": { > > > > ... > > > > "my\_stop":{ > > > > "type": "stop", > > > > "stopwords": ["_es_"] > > > > }, > > > > ... > > > > } > > > > } > > > > } > > > > }, > > > > > > > > do you happen to know where the default stopwords are defined? > > > > > > > > On Sunday, May 19, 2013 5:38:13 PM UTC-4, simonw wrote: > > > > > > > > > you might want to look at the `cutoff_frequency` that was added to the > > > > > match\_query ( > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/query-dsl/match-query/)) > > > > > It might enable you to not filter stopwords at all. Another option is to > > > > > create a stopword list per customer and add it to the mapping yourself, how > > > > > are you creating your stopword filters right now? > > > > > > > > > > simon > > > > > > > > > > On Sunday, May 19, 2013 10:55:06 PM UTC+2, JoeZ99 wrote: > > > > > > > > > > > Is there any way of "marking" a word not to be filtered out by the > > > > > > stop filter, even if it's on the stop words?? > > > > > > > > > > > > I'm looking for something like "keywordmarker" filter, which "marks" > > > > > > a keyword not to be stemmed afterwards. > > > > > > > > > > > > Use case. Some customers have a special meaning from just one or two > > > > > > words belonging to the standard stop words list of their language. Say, for > > > > > > a spanish customer, he want the word "tenga" to be searchable even if it > > > > > > belongs to the "_es_" expanded stopwords. > > > > > > > > > > > > I know I could just override the whole stopwords set, and provide a > > > > > > custom stopwords list with "tenga" not in it, and I just did that, but it > > > > > > would be "cheaper" instead of reespecifying the whole set, to say > > > > > > "exception"s: ["tenga"] > > > > > > > > > > > > On a sidenote: where can I find the default list that elasticsearch > > > > > > uses when specifying ["_es_"] as stopwords??? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![anon44344346](https://avatars.discourse-cdn.com/v4/letter/a/e56c9b/32.png) [@anon44344346](https://discuss.elastic.co/u/anon44344346)\ **Post date:** [May 22, 2013, 4:13pm UTC](https://discuss.elastic.co/t/prevent-some-words-from-being-stopped/12030/9 "2013-05-22T16:13:05Z")
thanks!! On Wednesday, May 22, 2013 11:43:34 AM UTC-4, simonw wrote: > On Tuesday, May 21, 2013 9:40:20 PM UTC+2, simonw wrote: > > > On Tuesday, May 21, 2013 4:12:14 PM UTC+2, JoeZ99 wrote: > > > > > On Monday, May 20, 2013 2:53:02 PM UTC-4, simonw wrote: > > > > > > > On Monday, May 20, 2013 7:58:53 AM UTC+2, JoeZ99 wrote: > > > > > > > > > I didn't know about that feature. is it a new thing? > > > > > > > > yes it's pretty new. 0.90 only. > > > > > > > > > anyway, I think other considerations regarding how we build the query > > > > > before hitting the search server may prevent us for taking that road. but I > > > > > gotta take a deeper look into it. > > > > > > > > this feature makes your live easier not harder so going don't that road > > > > should be downhill. It essentially means you don't filter stopwords anymore > > > > and gain the ability to run stopword only searches fast as well. stuff like > > > > "to be or not to be" etc. > > > > > > Well, I'm definitely going to check that out . Can I do that "cutoff" > > > thing in a query that is not a "match query". I understood I couldn't > > > (that's why I'm not sure I can use it, because I'm not sure I can use match > > > query altogether). > > > > yes there is a common\_terms query but apparently I never finished the > > documentation for this one. Hmm it's still sitting in my local clone. Lemme > > finish the documentation and update you once it's there. what query are you > > using right now? > > here you go: > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/query-dsl/common-terms-query/) > > simon > > > > > > as how we use stopwords, it's pretty standard: > > > > > > > > ok so why don't you just modify the stopword list if a customer asks > > > > for it? I assume you have an index per customer? > > > > > > > > simon > > > > > > > > > > > > > > > Of course I've got an index per customer, and of course that's what I'm > > > doing now (that's why I want to know where the "standard" stopwords are > > > defined, so I can copy/paste them without a few ones the customer doesn't > > > want), replacing the entire stopwords collection for a custom-made one. I > > > just thought it would be easiest to define the two stopwords that > > > "shouldn't be there" than redeclare the whole stopwords bunch. > > > > it seems like stop filter could support keyword attribute as well but I > > am not sure if that is really what we should do. seems like a unique > > requirement. usually you wanna specify a custom list not an exclusion set. > > > > simon > > > > > > > "index\_name": { > > > > > "index": { > > > > > ... > > > > > ... > > > > > "analysis": { > > > > > "analyzer": { > > > > > "default\_index": { > > > > > "tokenizer": "standard", > > > > > "filter": ["standard", "lowercase", "my\_stop", > > > > > ...] > > > > > }, > > > > > "default\_search": { > > > > > "tokenizer": "standard", > > > > > "filter": ["standard", "lowercase", "my\_stop", > > > > > ...] > > > > > }, > > > > > }, > > > > > "filter": { > > > > > ... > > > > > "my\_stop":{ > > > > > "type": "stop", > > > > > "stopwords": ["_es_"] > > > > > }, > > > > > ... > > > > > } > > > > > } > > > > > } > > > > > }, > > > > > > > > > > do you happen to know where the default stopwords are defined? > > > > > > > > > > On Sunday, May 19, 2013 5:38:13 PM UTC-4, simonw wrote: > > > > > > > > > > > you might want to look at the `cutoff_frequency` that was added to > > > > > > the match\_query ( > > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/query-dsl/match-query/)) > > > > > > It might enable you to not filter stopwords at all. Another option is to > > > > > > create a stopword list per customer and add it to the mapping yourself, how > > > > > > are you creating your stopword filters right now? > > > > > > > > > > > > simon > > > > > > > > > > > > On Sunday, May 19, 2013 10:55:06 PM UTC+2, JoeZ99 wrote: > > > > > > > > > > > > > Is there any way of "marking" a word not to be filtered out by the > > > > > > > stop filter, even if it's on the stop words?? > > > > > > > > > > > > > > I'm looking for something like "keywordmarker" filter, which "marks" > > > > > > > a keyword not to be stemmed afterwards. > > > > > > > > > > > > > > Use case. Some customers have a special meaning from just one or two > > > > > > > words belonging to the standard stop words list of their language. Say, for > > > > > > > a spanish customer, he want the word "tenga" to be searchable even if it > > > > > > > belongs to the "_es_" expanded stopwords. > > > > > > > > > > > > > > I know I could just override the whole stopwords set, and provide a > > > > > > > custom stopwords list with "tenga" not in it, and I just did that, but it > > > > > > > would be "cheaper" instead of reespecifying the whole set, to say > > > > > > > "exception"s: ["tenga"] > > > > > > > > > > > > > > On a sidenote: where can I find the default list that elasticsearch > > > > > > > uses when specifying ["_es_"] as stopwords??? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:35am UTC](https://discuss.elastic.co/t/prevent-some-words-from-being-stopped/12030/10 "2017-07-06T02:35:22Z")