# Stop words filter not working

**URL:** <https://discuss.elastic.co/t/stop-words-filter-not-working/10530>\
**Category:** Elasticsearch\
**Created:** [January 28, 2013, 7:31pm UTC](https://discuss.elastic.co/t/stop-words-filter-not-working/10530 "2013-01-28T19:31:58Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Haensel](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/haensel/32/2595_2.png) [@Haensel](https://discuss.elastic.co/u/Haensel)\
**Post date:** [January 28, 2013, 7:31pm UTC](https://discuss.elastic.co/t/stop-words-filter-not-working/10530/1 "2013-01-28T19:31:58Z")

</div>

Hi,

I have the following problem: I have a list of company names but want to  
exclude the "form of the organization" (like Limited, LLC etc.) by using my  
own stopwords filter. So far so good, but I can't get it to work. It does  
the indexing, everything is searchable, but when searching for "LLC" etc I  
am still getting matches. Here is my config (I am using PHP syntax here,  
but I guess the values are obvious):

'analysis' =\> array(  
'analyzer' =\> array(  
'name\_analyzer' =\> array(  
'type' =\> 'custom',  
'tokenizer' =\> 'standard',  
'filter' =\> array('my\_stopwords','lowercase','icu\_normalizer','ngram')  
),  
'address\_analyzer' =\> array(  
'type' =\> 'custom',  
'tokenizer' =\> 'standard',  
'filter' =\> array('standard','ngram')  
),  
'country\_analyzer' =\> array(  
'type' =\> 'custom',  
'tokenizer' =\> 'lowercase',  
'filter' =\> array('country\_synonyms')  
),  
),  
'filter' =\> array(  
'ngram' =\> array(  
'type' =\> 'nGram',  
'min\_gram' =\> 1,  
'max\_gram' =\> 5,  
),  
'country\_synonyms' =\> array(  
'type' =\> 'synonym',  
'synonyms' =\> array('some synonyms that work perfectly')  
),  
'my\_stopwords' =\> array(  
'type' =\> 'stop',  
'stopwords' =\> array('llc','gmbh',etc.etc.),  
'ignore\_case' =\> true  
)  
)  
)

And here is my mapping:

'names' =\> array(  
'type' =\> 'string',  
'analyzer' =\> 'name\_analyzer',  
'index\_analyzer' =\> 'name\_analyzer',  
'search\_analyzer' =\> 'name\_analyzer',  
'include\_in\_all' =\> true  
),  
'addresses' =\> array(  
'dynamic' =\> false,  
'analyzer' =\> 'address\_analyzer',  
'index\_analyzer' =\> 'address\_analyzer',  
'search\_analyzer' =\> 'address\_analyzer',  
'properties' =\> array(  
'street' =\> array(  
'type' =\> 'string',  
'analyzer' =\> 'address\_analyzer',  
'index\_analyzer' =\> 'address\_analyzer',  
'search\_analyzer' =\> 'address\_analyzer',  
'include\_in\_all' =\> true  
),  
'city' =\> array(  
'type' =\> 'string',  
'analyzer' =\> 'address\_analyzer',  
'index\_analyzer' =\> 'address\_analyzer',  
'search\_analyzer' =\> 'address\_analyzer',  
'include\_in\_all' =\> true  
),  
'state' =\> array(  
'type' =\> 'string',  
'analyzer' =\> 'address\_analyzer',  
'index\_analyzer' =\> 'address\_analyzer',  
'search\_analyzer' =\> 'address\_analyzer',  
'include\_in\_all' =\> true  
),  
'country' =\> array(  
'type' =\> 'string',  
'analyzer' =\> 'country\_analyzer',  
'index\_analyzer' =\> 'country\_analyzer',  
'search\_analyzer' =\> 'country\_analyzer',  
'include\_in\_all' =\> true  
)  
)  
)

A GET request to myindex/\_settings shows that the values are set correctly.  
Example:

"index.analysis.filter.my\_stopwords.stopwords.90":"llc"

I feel pretty lost here. So any help would be really appreciated!

Thanks in advance,

Hannes

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group, send email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![mvg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mvg/32/98890_2.png) [@mvg](https://discuss.elastic.co/u/mvg)\
**Post date:** [January 29, 2013, 8:40am UTC](https://discuss.elastic.co/t/stop-words-filter-not-working/10530/2 "2013-01-29T08:40:57Z")

</div>

Hi Hannes,

How does the query look like?

Martijn

On 28 January 2013 20:31, Haensel [thehaensel@gmail.com](mailto:thehaensel@gmail.com) wrote:

> Hi,
> 
> I have the following problem: I have a list of company names but want to  
> exclude the "form of the organization" (like Limited, LLC etc.) by using my  
> own stopwords filter. So far so good, but I can't get it to work. It does  
> the indexing, everything is searchable, but when searching for "LLC" etc I  
> am still getting matches. Here is my config (I am using PHP syntax here, but  
> I guess the values are obvious):
> 
> 'analysis' =\> array(  
> 'analyzer' =\> array(  
> 'name\_analyzer' =\> array(  
> 'type' =\> 'custom',  
> 'tokenizer' =\> 'standard',  
> 'filter' =\> array('my\_stopwords','lowercase','icu\_normalizer','ngram')  
> ),  
> 'address\_analyzer' =\> array(  
> 'type' =\> 'custom',  
> 'tokenizer' =\> 'standard',  
> 'filter' =\> array('standard','ngram')  
> ),  
> 'country\_analyzer' =\> array(  
> 'type' =\> 'custom',  
> 'tokenizer' =\> 'lowercase',  
> 'filter' =\> array('country\_synonyms')  
> ),  
> ),  
> 'filter' =\> array(  
> 'ngram' =\> array(  
> 'type' =\> 'nGram',  
> 'min\_gram' =\> 1,  
> 'max\_gram' =\> 5,  
> ),  
> 'country\_synonyms' =\> array(  
> 'type' =\> 'synonym',  
> 'synonyms' =\> array('some synonyms that work perfectly')  
> ),  
> 'my\_stopwords' =\> array(  
> 'type' =\> 'stop',  
> 'stopwords' =\> array('llc','gmbh',etc.etc.),  
> 'ignore\_case' =\> true  
> )  
> )  
> )
> 
> And here is my mapping:
> 
> 'names' =\> array(  
> 'type' =\> 'string',  
> 'analyzer' =\> 'name\_analyzer',  
> 'index\_analyzer' =\> 'name\_analyzer',  
> 'search\_analyzer' =\> 'name\_analyzer',  
> 'include\_in\_all' =\> true  
> ),  
> 'addresses' =\> array(  
> 'dynamic' =\> false,  
> 'analyzer' =\> 'address\_analyzer',  
> 'index\_analyzer' =\> 'address\_analyzer',  
> 'search\_analyzer' =\> 'address\_analyzer',  
> 'properties' =\> array(  
> 'street' =\> array(  
> 'type' =\> 'string',  
> 'analyzer' =\> 'address\_analyzer',  
> 'index\_analyzer' =\> 'address\_analyzer',  
> 'search\_analyzer' =\> 'address\_analyzer',  
> 'include\_in\_all' =\> true  
> ),  
> 'city' =\> array(  
> 'type' =\> 'string',  
> 'analyzer' =\> 'address\_analyzer',  
> 'index\_analyzer' =\> 'address\_analyzer',  
> 'search\_analyzer' =\> 'address\_analyzer',  
> 'include\_in\_all' =\> true  
> ),  
> 'state' =\> array(  
> 'type' =\> 'string',  
> 'analyzer' =\> 'address\_analyzer',  
> 'index\_analyzer' =\> 'address\_analyzer',  
> 'search\_analyzer' =\> 'address\_analyzer',  
> 'include\_in\_all' =\> true  
> ),  
> 'country' =\> array(  
> 'type' =\> 'string',  
> 'analyzer' =\> 'country\_analyzer',  
> 'index\_analyzer' =\> 'country\_analyzer',  
> 'search\_analyzer' =\> 'country\_analyzer',  
> 'include\_in\_all' =\> true  
> )  
> )  
> )
> 
> A GET request to myindex/\_settings shows that the values are set correctly.  
> Example:
> 
> "index.analysis.filter.my\_stopwords.stopwords.90":"llc"
> 
> I feel pretty lost here. So any help would be really appreciated!
> 
> Thanks in advance,
> 
> Hannes
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group, send email to  
> [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
Met vriendelijke groet,

Martijn van Groningen

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Haensel](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/haensel/32/2595_2.png) [@Haensel](https://discuss.elastic.co/u/Haensel)\
**Post date:** [January 29, 2013, 10:37am UTC](https://discuss.elastic.co/t/stop-words-filter-not-working/10530/3 "2013-01-29T10:37:33Z")

</div>

Hi,

I am using a combination of a query string (user searches via a "Google  
Search" like textbox) and a fuzzy query to be able to find misspelled names  
etc. Maybe the fuzzy search makes the stopwords useless? And if so, would  
there be a way around that?

{"bool":  
{"should":[  
{"query\_string":{"query":"gmbh","default\_operator":"AND"}},

{"fuzzy\_like\_this":{"boost":1,"like\_text":"gmbh","min\_similarity":0.5,"prefix\_length":0,"max\_query\_terms":25}}  
]}  
}

Thanks,

Hannes

On Tuesday, January 29, 2013 9:40:57 AM UTC+1, Martijn v Groningen wrote:

> Hi Hannes,
> 
> How does the query look like?
> 
> Martijn
> 
> On 28 January 2013 20:31, Haensel \<[theha...@gmail.com](mailto:theha...@gmail.com) \<javascript:\>\>  
> wrote:
> 
> > Hi,
> > 
> > I have the following problem: I have a list of company names but want to  
> > exclude the "form of the organization" (like Limited, LLC etc.) by using  
> > my  
> > own stopwords filter. So far so good, but I can't get it to work. It  
> > does  
> > the indexing, everything is searchable, but when searching for "LLC" etc  
> > I  
> > am still getting matches. Here is my config (I am using PHP syntax here,  
> > but  
> > I guess the values are obvious):
> > 
> > 'analysis' =\> array(  
> > 'analyzer' =\> array(  
> > 'name\_analyzer' =\> array(  
> > 'type' =\> 'custom',  
> > 'tokenizer' =\> 'standard',  
> > 'filter' =\> array('my\_stopwords','lowercase','icu\_normalizer','ngram')  
> > ),  
> > 'address\_analyzer' =\> array(  
> > 'type' =\> 'custom',  
> > 'tokenizer' =\> 'standard',  
> > 'filter' =\> array('standard','ngram')  
> > ),  
> > 'country\_analyzer' =\> array(  
> > 'type' =\> 'custom',  
> > 'tokenizer' =\> 'lowercase',  
> > 'filter' =\> array('country\_synonyms')  
> > ),  
> > ),  
> > 'filter' =\> array(  
> > 'ngram' =\> array(  
> > 'type' =\> 'nGram',  
> > 'min\_gram' =\> 1,  
> > 'max\_gram' =\> 5,  
> > ),  
> > 'country\_synonyms' =\> array(  
> > 'type' =\> 'synonym',  
> > 'synonyms' =\> array('some synonyms that work perfectly')  
> > ),  
> > 'my\_stopwords' =\> array(  
> > 'type' =\> 'stop',  
> > 'stopwords' =\> array('llc','gmbh',etc.etc.),  
> > 'ignore\_case' =\> true  
> > )  
> > )  
> > )
> > 
> > And here is my mapping:
> > 
> > 'names' =\> array(  
> > 'type' =\> 'string',  
> > 'analyzer' =\> 'name\_analyzer',  
> > 'index\_analyzer' =\> 'name\_analyzer',  
> > 'search\_analyzer' =\> 'name\_analyzer',  
> > 'include\_in\_all' =\> true  
> > ),  
> > 'addresses' =\> array(  
> > 'dynamic' =\> false,  
> > 'analyzer' =\> 'address\_analyzer',  
> > 'index\_analyzer' =\> 'address\_analyzer',  
> > 'search\_analyzer' =\> 'address\_analyzer',  
> > 'properties' =\> array(  
> > 'street' =\> array(  
> > 'type' =\> 'string',  
> > 'analyzer' =\> 'address\_analyzer',  
> > 'index\_analyzer' =\> 'address\_analyzer',  
> > 'search\_analyzer' =\> 'address\_analyzer',  
> > 'include\_in\_all' =\> true  
> > ),  
> > 'city' =\> array(  
> > 'type' =\> 'string',  
> > 'analyzer' =\> 'address\_analyzer',  
> > 'index\_analyzer' =\> 'address\_analyzer',  
> > 'search\_analyzer' =\> 'address\_analyzer',  
> > 'include\_in\_all' =\> true  
> > ),  
> > 'state' =\> array(  
> > 'type' =\> 'string',  
> > 'analyzer' =\> 'address\_analyzer',  
> > 'index\_analyzer' =\> 'address\_analyzer',  
> > 'search\_analyzer' =\> 'address\_analyzer',  
> > 'include\_in\_all' =\> true  
> > ),  
> > 'country' =\> array(  
> > 'type' =\> 'string',  
> > 'analyzer' =\> 'country\_analyzer',  
> > 'index\_analyzer' =\> 'country\_analyzer',  
> > 'search\_analyzer' =\> 'country\_analyzer',  
> > 'include\_in\_all' =\> true  
> > )  
> > )  
> > )
> > 
> > A GET request to myindex/\_settings shows that the values are set  
> > correctly.  
> > Example:
> > 
> > "index.analysis.filter.my\_stopwords.stopwords.90":"llc"
> > 
> > I feel pretty lost here. So any help would be really appreciated!
> > 
> > Thanks in advance,
> > 
> > Hannes
> > 
> > --  
> > You received this message because you are subscribed to the Google  
> > Groups  
> > "elasticsearch" group.  
> > To unsubscribe from this group, send email to  
> > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \<javascript:\>.  
> > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> 
> --  
> Met vriendelijke groet,
> 
> Martijn van Groningen

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![mvg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mvg/32/98890_2.png) [@mvg](https://discuss.elastic.co/u/mvg)\
**Post date:** [January 29, 2013, 12:12pm UTC](https://discuss.elastic.co/t/stop-words-filter-not-working/10530/4 "2013-01-29T12:12:59Z")

</div>

Hi Hannes,

You're not specifying a field for both the `query_string` and  
`fuzzy_like_this` queries. The behaviour is then to use the `_all`  
field.  
I think if you specify the `names` field then it should work. For the  
`query_string` query use the `default_field` option and for the  
`fuzzy_like_this` you need to use the `fields` option to use the your  
`names` field.

Martijn

On 29 January 2013 11:37, Haensel [thehaensel@gmail.com](mailto:thehaensel@gmail.com) wrote:

> Hi,
> 
> I am using a combination of a query string (user searches via a "Google  
> Search" like textbox) and a fuzzy query to be able to find misspelled names  
> etc. Maybe the fuzzy search makes the stopwords useless? And if so, would  
> there be a way around that?
> 
> {"bool":  
> {"should":[  
> {"query\_string":{"query":"gmbh","default\_operator":"AND"}},
> 
> {"fuzzy\_like\_this":{"boost":1,"like\_text":"gmbh","min\_similarity":0.5,"prefix\_length":0,"max\_query\_terms":25}}  
> ]}  
> }
> 
> Thanks,
> 
> Hannes
> 
> On Tuesday, January 29, 2013 9:40:57 AM UTC+1, Martijn v Groningen wrote:
> 
> > Hi Hannes,
> > 
> > How does the query look like?
> > 
> > Martijn
> > 
> > On 28 January 2013 20:31, Haensel [theha...@gmail.com](mailto:theha...@gmail.com) wrote:
> > 
> > > Hi,
> > > 
> > > I have the following problem: I have a list of company names but want to  
> > > exclude the "form of the organization" (like Limited, LLC etc.) by using  
> > > my  
> > > own stopwords filter. So far so good, but I can't get it to work. It  
> > > does  
> > > the indexing, everything is searchable, but when searching for "LLC" etc  
> > > I  
> > > am still getting matches. Here is my config (I am using PHP syntax here,  
> > > but  
> > > I guess the values are obvious):
> > > 
> > > 'analysis' =\> array(  
> > > 'analyzer' =\> array(  
> > > 'name\_analyzer' =\> array(  
> > > 'type' =\> 'custom',  
> > > 'tokenizer' =\> 'standard',  
> > > 'filter' =\> array('my\_stopwords','lowercase','icu\_normalizer','ngram')  
> > > ),  
> > > 'address\_analyzer' =\> array(  
> > > 'type' =\> 'custom',  
> > > 'tokenizer' =\> 'standard',  
> > > 'filter' =\> array('standard','ngram')  
> > > ),  
> > > 'country\_analyzer' =\> array(  
> > > 'type' =\> 'custom',  
> > > 'tokenizer' =\> 'lowercase',  
> > > 'filter' =\> array('country\_synonyms')  
> > > ),  
> > > ),  
> > > 'filter' =\> array(  
> > > 'ngram' =\> array(  
> > > 'type' =\> 'nGram',  
> > > 'min\_gram' =\> 1,  
> > > 'max\_gram' =\> 5,  
> > > ),  
> > > 'country\_synonyms' =\> array(  
> > > 'type' =\> 'synonym',  
> > > 'synonyms' =\> array('some synonyms that work perfectly')  
> > > ),  
> > > 'my\_stopwords' =\> array(  
> > > 'type' =\> 'stop',  
> > > 'stopwords' =\> array('llc','gmbh',etc.etc.),  
> > > 'ignore\_case' =\> true  
> > > )  
> > > )  
> > > )
> > > 
> > > And here is my mapping:
> > > 
> > > 'names' =\> array(  
> > > 'type' =\> 'string',  
> > > 'analyzer' =\> 'name\_analyzer',  
> > > 'index\_analyzer' =\> 'name\_analyzer',  
> > > 'search\_analyzer' =\> 'name\_analyzer',  
> > > 'include\_in\_all' =\> true  
> > > ),  
> > > 'addresses' =\> array(  
> > > 'dynamic' =\> false,  
> > > 'analyzer' =\> 'address\_analyzer',  
> > > 'index\_analyzer' =\> 'address\_analyzer',  
> > > 'search\_analyzer' =\> 'address\_analyzer',  
> > > 'properties' =\> array(  
> > > 'street' =\> array(  
> > > 'type' =\> 'string',  
> > > 'analyzer' =\> 'address\_analyzer',  
> > > 'index\_analyzer' =\> 'address\_analyzer',  
> > > 'search\_analyzer' =\> 'address\_analyzer',  
> > > 'include\_in\_all' =\> true  
> > > ),  
> > > 'city' =\> array(  
> > > 'type' =\> 'string',  
> > > 'analyzer' =\> 'address\_analyzer',  
> > > 'index\_analyzer' =\> 'address\_analyzer',  
> > > 'search\_analyzer' =\> 'address\_analyzer',  
> > > 'include\_in\_all' =\> true  
> > > ),  
> > > 'state' =\> array(  
> > > 'type' =\> 'string',  
> > > 'analyzer' =\> 'address\_analyzer',  
> > > 'index\_analyzer' =\> 'address\_analyzer',  
> > > 'search\_analyzer' =\> 'address\_analyzer',  
> > > 'include\_in\_all' =\> true  
> > > ),  
> > > 'country' =\> array(  
> > > 'type' =\> 'string',  
> > > 'analyzer' =\> 'country\_analyzer',  
> > > 'index\_analyzer' =\> 'country\_analyzer',  
> > > 'search\_analyzer' =\> 'country\_analyzer',  
> > > 'include\_in\_all' =\> true  
> > > )  
> > > )  
> > > )
> > > 
> > > A GET request to myindex/\_settings shows that the values are set  
> > > correctly.  
> > > Example:
> > > 
> > > "index.analysis.filter.my\_stopwords.stopwords.90":"llc"
> > > 
> > > I feel pretty lost here. So any help would be really appreciated!
> > > 
> > > Thanks in advance,
> > > 
> > > Hannes
> > > 
> > > --  
> > > You received this message because you are subscribed to the Google  
> > > Groups  
> > > "elasticsearch" group.  
> > > To unsubscribe from this group, send email to  
> > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).  
> > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > 
> > --  
> > Met vriendelijke groet,
> > 
> > Martijn van Groningen

--  
Met vriendelijke groet,

Martijn van Groningen

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Haensel](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/haensel/32/2595_2.png) [@Haensel](https://discuss.elastic.co/u/Haensel)\
**Post date:** [January 29, 2013, 12:41pm UTC](https://discuss.elastic.co/t/stop-words-filter-not-working/10530/5 "2013-01-29T12:41:01Z")

</div>

Thanks for your help but it still doesn't work ☹ Here is the new query:

"query":{"bool":  
{"should":[  
{"query\_string":{"query":"test  
gmbh","fuzzy\_prefix\_length":3,"default\_operator":"AND","default\_field":"names"}},  
{"fuzzy\_like\_this":{"fields":["names"],"boost":1,"like\_text":"test  
gmbh","min\_similarity":0.2,"prefix\_length":0,"max\_query\_terms":25}}]  
}}  
}

Here's an example: When searching for a company that was indexed as  
"TestCompany" a query like "TestComp Limited" (notice the "misspelled"  
company name) will find a lot of "Limited" companies while "TestCompany" is  
the string I am really interested in. I simply want "TestCompany" (or a  
similar name via fuzzy search) to be first in the list, effectively  
ignoring all legal forms defined in my stopwords list.

On Tuesday, January 29, 2013 1:12:59 PM UTC+1, Martijn v Groningen wrote:

> Hi Hannes,
> 
> You're not specifying a field for both the `query_string` and  
> `fuzzy_like_this` queries. The behaviour is then to use the `_all`  
> field.  
> I think if you specify the `names` field then it should work. For the  
> `query_string` query use the `default_field` option and for the  
> `fuzzy_like_this` you need to use the `fields` option to use the your  
> `names` field.
> 
> Martijn
> 
> On 29 January 2013 11:37, Haensel \<[theha...@gmail.com](mailto:theha...@gmail.com) \<javascript:\>\>  
> wrote:
> 
> > Hi,
> > 
> > I am using a combination of a query string (user searches via a "Google  
> > Search" like textbox) and a fuzzy query to be able to find misspelled  
> > names  
> > etc. Maybe the fuzzy search makes the stopwords useless? And if so,  
> > would  
> > there be a way around that?
> > 
> > {"bool":  
> > {"should":[  
> > {"query\_string":{"query":"gmbh","default\_operator":"AND"}},
> 
> {"fuzzy\_like\_this":{"boost":1,"like\_text":"gmbh","min\_similarity":0.5,"prefix\_length":0,"max\_query\_terms":25}}
> 
> > ]}  
> > }
> > 
> > Thanks,
> > 
> > Hannes
> > 
> > On Tuesday, January 29, 2013 9:40:57 AM UTC+1, Martijn v Groningen  
> > wrote:
> > 
> > > Hi Hannes,
> > > 
> > > How does the query look like?
> > > 
> > > Martijn
> > > 
> > > On 28 January 2013 20:31, Haensel [theha...@gmail.com](mailto:theha...@gmail.com) wrote:
> > > 
> > > > Hi,
> > > > 
> > > > I have the following problem: I have a list of company names but want  
> > > > to  
> > > > exclude the "form of the organization" (like Limited, LLC etc.) by  
> > > > using  
> > > > my  
> > > > own stopwords filter. So far so good, but I can't get it to work. It  
> > > > does  
> > > > the indexing, everything is searchable, but when searching for "LLC"  
> > > > etc  
> > > > I  
> > > > am still getting matches. Here is my config (I am using PHP syntax  
> > > > here,  
> > > > but  
> > > > I guess the values are obvious):
> > > > 
> > > > 'analysis' =\> array(  
> > > > 'analyzer' =\> array(  
> > > > 'name\_analyzer' =\> array(  
> > > > 'type' =\> 'custom',  
> > > > 'tokenizer' =\> 'standard',  
> > > > 'filter' =\>  
> > > > array('my\_stopwords','lowercase','icu\_normalizer','ngram')  
> > > > ),  
> > > > 'address\_analyzer' =\> array(  
> > > > 'type' =\> 'custom',  
> > > > 'tokenizer' =\> 'standard',  
> > > > 'filter' =\> array('standard','ngram')  
> > > > ),  
> > > > 'country\_analyzer' =\> array(  
> > > > 'type' =\> 'custom',  
> > > > 'tokenizer' =\> 'lowercase',  
> > > > 'filter' =\> array('country\_synonyms')  
> > > > ),  
> > > > ),  
> > > > 'filter' =\> array(  
> > > > 'ngram' =\> array(  
> > > > 'type' =\> 'nGram',  
> > > > 'min\_gram' =\> 1,  
> > > > 'max\_gram' =\> 5,  
> > > > ),  
> > > > 'country\_synonyms' =\> array(  
> > > > 'type' =\> 'synonym',  
> > > > 'synonyms' =\> array('some synonyms that work perfectly')  
> > > > ),  
> > > > 'my\_stopwords' =\> array(  
> > > > 'type' =\> 'stop',  
> > > > 'stopwords' =\> array('llc','gmbh',etc.etc.),  
> > > > 'ignore\_case' =\> true  
> > > > )  
> > > > )  
> > > > )
> > > > 
> > > > And here is my mapping:
> > > > 
> > > > 'names' =\> array(  
> > > > 'type' =\> 'string',  
> > > > 'analyzer' =\> 'name\_analyzer',  
> > > > 'index\_analyzer' =\> 'name\_analyzer',  
> > > > 'search\_analyzer' =\> 'name\_analyzer',  
> > > > 'include\_in\_all' =\> true  
> > > > ),  
> > > > 'addresses' =\> array(  
> > > > 'dynamic' =\> false,  
> > > > 'analyzer' =\> 'address\_analyzer',  
> > > > 'index\_analyzer' =\> 'address\_analyzer',  
> > > > 'search\_analyzer' =\> 'address\_analyzer',  
> > > > 'properties' =\> array(  
> > > > 'street' =\> array(  
> > > > 'type' =\> 'string',  
> > > > 'analyzer' =\> 'address\_analyzer',  
> > > > 'index\_analyzer' =\> 'address\_analyzer',  
> > > > 'search\_analyzer' =\> 'address\_analyzer',  
> > > > 'include\_in\_all' =\> true  
> > > > ),  
> > > > 'city' =\> array(  
> > > > 'type' =\> 'string',  
> > > > 'analyzer' =\> 'address\_analyzer',  
> > > > 'index\_analyzer' =\> 'address\_analyzer',  
> > > > 'search\_analyzer' =\> 'address\_analyzer',  
> > > > 'include\_in\_all' =\> true  
> > > > ),  
> > > > 'state' =\> array(  
> > > > 'type' =\> 'string',  
> > > > 'analyzer' =\> 'address\_analyzer',  
> > > > 'index\_analyzer' =\> 'address\_analyzer',  
> > > > 'search\_analyzer' =\> 'address\_analyzer',  
> > > > 'include\_in\_all' =\> true  
> > > > ),  
> > > > 'country' =\> array(  
> > > > 'type' =\> 'string',  
> > > > 'analyzer' =\> 'country\_analyzer',  
> > > > 'index\_analyzer' =\> 'country\_analyzer',  
> > > > 'search\_analyzer' =\> 'country\_analyzer',  
> > > > 'include\_in\_all' =\> true  
> > > > )  
> > > > )  
> > > > )
> > > > 
> > > > A GET request to myindex/\_settings shows that the values are set  
> > > > correctly.  
> > > > Example:
> > > > 
> > > > "index.analysis.filter.my\_stopwords.stopwords.90":"llc"
> > > > 
> > > > I feel pretty lost here. So any help would be really appreciated!
> > > > 
> > > > Thanks in advance,
> > > > 
> > > > Hannes
> > > > 
> > > > --  
> > > > You received this message because you are subscribed to the Google  
> > > > Groups  
> > > > "elasticsearch" group.  
> > > > To unsubscribe from this group, send email to  
> > > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).  
> > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > > 
> > > --  
> > > Met vriendelijke groet,
> > > 
> > > Martijn van Groningen
> 
> --  
> Met vriendelijke groet,
> 
> Martijn van Groningen

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![mvg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mvg/32/98890_2.png) [@mvg](https://discuss.elastic.co/u/mvg)\
**Post date:** [January 29, 2013, 3:01pm UTC](https://discuss.elastic.co/t/stop-words-filter-not-working/10530/6 "2013-01-29T15:01:00Z")

</div>

I think the reason you see other companies that have these stopwords  
is not because you had a match on the stop filter, but because the  
`fuzzy_like_this` query has a low `min_similarity`. I suggest that you  
increase it to something like `0.7` You have to test and play around  
with it a bit.

If you really think that you have a hit on a stopword (which I don't  
think happens) you can verify this the explain option:

> **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)**
>
> Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today.

Martijn

On 29 January 2013 13:41, Haensel [thehaensel@gmail.com](mailto:thehaensel@gmail.com) wrote:

> Thanks for your help but it still doesn't work ☹ Here is the new query:
> 
> "query":{"bool":  
> {"should":[  
> {"query\_string":{"query":"test  
> gmbh","fuzzy\_prefix\_length":3,"default\_operator":"AND","default\_field":"names"}},  
> {"fuzzy\_like\_this":{"fields":["names"],"boost":1,"like\_text":"test  
> gmbh","min\_similarity":0.2,"prefix\_length":0,"max\_query\_terms":25}}]  
> }}  
> }
> 
> Here's an example: When searching for a company that was indexed as  
> "TestCompany" a query like "TestComp Limited" (notice the "misspelled"  
> company name) will find a lot of "Limited" companies while "TestCompany" is  
> the string I am really interested in. I simply want "TestCompany" (or a  
> similar name via fuzzy search) to be first in the list, effectively ignoring  
> all legal forms defined in my stopwords list.
> 
> On Tuesday, January 29, 2013 1:12:59 PM UTC+1, Martijn v Groningen wrote:
> 
> > Hi Hannes,
> > 
> > You're not specifying a field for both the `query_string` and  
> > `fuzzy_like_this` queries. The behaviour is then to use the `_all`  
> > field.  
> > I think if you specify the `names` field then it should work. For the  
> > `query_string` query use the `default_field` option and for the  
> > `fuzzy_like_this` you need to use the `fields` option to use the your  
> > `names` field.
> > 
> > Martijn
> > 
> > On 29 January 2013 11:37, Haensel [theha...@gmail.com](mailto:theha...@gmail.com) wrote:
> > 
> > > Hi,
> > > 
> > > I am using a combination of a query string (user searches via a "Google  
> > > Search" like textbox) and a fuzzy query to be able to find misspelled  
> > > names  
> > > etc. Maybe the fuzzy search makes the stopwords useless? And if so,  
> > > would  
> > > there be a way around that?
> > > 
> > > {"bool":  
> > > {"should":[  
> > > {"query\_string":{"query":"gmbh","default\_operator":"AND"}},
> > > 
> > > {"fuzzy\_like\_this":{"boost":1,"like\_text":"gmbh","min\_similarity":0.5,"prefix\_length":0,"max\_query\_terms":25}}  
> > > ]}  
> > > }
> > > 
> > > Thanks,
> > > 
> > > Hannes
> > > 
> > > On Tuesday, January 29, 2013 9:40:57 AM UTC+1, Martijn v Groningen  
> > > wrote:
> > > 
> > > > Hi Hannes,
> > > > 
> > > > How does the query look like?
> > > > 
> > > > Martijn
> > > > 
> > > > On 28 January 2013 20:31, Haensel [theha...@gmail.com](mailto:theha...@gmail.com) wrote:
> > > > 
> > > > > Hi,
> > > > > 
> > > > > I have the following problem: I have a list of company names but want  
> > > > > to  
> > > > > exclude the "form of the organization" (like Limited, LLC etc.) by  
> > > > > using  
> > > > > my  
> > > > > own stopwords filter. So far so good, but I can't get it to work. It  
> > > > > does  
> > > > > the indexing, everything is searchable, but when searching for "LLC"  
> > > > > etc  
> > > > > I  
> > > > > am still getting matches. Here is my config (I am using PHP syntax  
> > > > > here,  
> > > > > but  
> > > > > I guess the values are obvious):
> > > > > 
> > > > > 'analysis' =\> array(  
> > > > > 'analyzer' =\> array(  
> > > > > 'name\_analyzer' =\> array(  
> > > > > 'type' =\> 'custom',  
> > > > > 'tokenizer' =\> 'standard',  
> > > > > 'filter' =\>  
> > > > > array('my\_stopwords','lowercase','icu\_normalizer','ngram')  
> > > > > ),  
> > > > > 'address\_analyzer' =\> array(  
> > > > > 'type' =\> 'custom',  
> > > > > 'tokenizer' =\> 'standard',  
> > > > > 'filter' =\> array('standard','ngram')  
> > > > > ),  
> > > > > 'country\_analyzer' =\> array(  
> > > > > 'type' =\> 'custom',  
> > > > > 'tokenizer' =\> 'lowercase',  
> > > > > 'filter' =\> array('country\_synonyms')  
> > > > > ),  
> > > > > ),  
> > > > > 'filter' =\> array(  
> > > > > 'ngram' =\> array(  
> > > > > 'type' =\> 'nGram',  
> > > > > 'min\_gram' =\> 1,  
> > > > > 'max\_gram' =\> 5,  
> > > > > ),  
> > > > > 'country\_synonyms' =\> array(  
> > > > > 'type' =\> 'synonym',  
> > > > > 'synonyms' =\> array('some synonyms that work perfectly')  
> > > > > ),  
> > > > > 'my\_stopwords' =\> array(  
> > > > > 'type' =\> 'stop',  
> > > > > 'stopwords' =\> array('llc','gmbh',etc.etc.),  
> > > > > 'ignore\_case' =\> true  
> > > > > )  
> > > > > )  
> > > > > )
> > > > > 
> > > > > And here is my mapping:
> > > > > 
> > > > > 'names' =\> array(  
> > > > > 'type' =\> 'string',  
> > > > > 'analyzer' =\> 'name\_analyzer',  
> > > > > 'index\_analyzer' =\> 'name\_analyzer',  
> > > > > 'search\_analyzer' =\> 'name\_analyzer',  
> > > > > 'include\_in\_all' =\> true  
> > > > > ),  
> > > > > 'addresses' =\> array(  
> > > > > 'dynamic' =\> false,  
> > > > > 'analyzer' =\> 'address\_analyzer',  
> > > > > 'index\_analyzer' =\> 'address\_analyzer',  
> > > > > 'search\_analyzer' =\> 'address\_analyzer',  
> > > > > 'properties' =\> array(  
> > > > > 'street' =\> array(  
> > > > > 'type' =\> 'string',  
> > > > > 'analyzer' =\> 'address\_analyzer',  
> > > > > 'index\_analyzer' =\> 'address\_analyzer',  
> > > > > 'search\_analyzer' =\> 'address\_analyzer',  
> > > > > 'include\_in\_all' =\> true  
> > > > > ),  
> > > > > 'city' =\> array(  
> > > > > 'type' =\> 'string',  
> > > > > 'analyzer' =\> 'address\_analyzer',  
> > > > > 'index\_analyzer' =\> 'address\_analyzer',  
> > > > > 'search\_analyzer' =\> 'address\_analyzer',  
> > > > > 'include\_in\_all' =\> true  
> > > > > ),  
> > > > > 'state' =\> array(  
> > > > > 'type' =\> 'string',  
> > > > > 'analyzer' =\> 'address\_analyzer',  
> > > > > 'index\_analyzer' =\> 'address\_analyzer',  
> > > > > 'search\_analyzer' =\> 'address\_analyzer',  
> > > > > 'include\_in\_all' =\> true  
> > > > > ),  
> > > > > 'country' =\> array(  
> > > > > 'type' =\> 'string',  
> > > > > 'analyzer' =\> 'country\_analyzer',  
> > > > > 'index\_analyzer' =\> 'country\_analyzer',  
> > > > > 'search\_analyzer' =\> 'country\_analyzer',  
> > > > > 'include\_in\_all' =\> true  
> > > > > )  
> > > > > )  
> > > > > )
> > > > > 
> > > > > A GET request to myindex/\_settings shows that the values are set  
> > > > > correctly.  
> > > > > Example:
> > > > > 
> > > > > "index.analysis.filter.my\_stopwords.stopwords.90":"llc"
> > > > > 
> > > > > I feel pretty lost here. So any help would be really appreciated!
> > > > > 
> > > > > Thanks in advance,
> > > > > 
> > > > > Hannes
> > > > > 
> > > > > --  
> > > > > You received this message because you are subscribed to the Google  
> > > > > Groups  
> > > > > "elasticsearch" group.  
> > > > > To unsubscribe from this group, send email to  
> > > > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).  
> > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > > > 
> > > > --  
> > > > Met vriendelijke groet,
> > > > 
> > > > Martijn van Groningen
> > 
> > --  
> > Met vriendelijke groet,
> > 
> > Martijn van Groningen

--  
Met vriendelijke groet,

Martijn van Groningen

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Haensel](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/haensel/32/2595_2.png) [@Haensel](https://discuss.elastic.co/u/Haensel)\
**Post date:** [January 29, 2013, 3:29pm UTC](https://discuss.elastic.co/t/stop-words-filter-not-working/10530/7 "2013-01-29T15:29:10Z")

</div>

I've played around with the min\_similarity quite a bit and it is currently  
set to 0.7 so this isn't the reason I suppose. But thanks for the hint  
regarding the explanation. For the query "gmbh" (which is a stopword) it  
returns the following

{"value":0.625,"description":"fieldWeight(names:gmbh in 4993), product of:",  
"details":[{"value":1,"description":"tf(termFreq(names:gmbh)=1)"},{"value":1  
,"description":"idf(docFreq=40, maxDocs=7428)"},{"value":0.625,"description"  
:"fieldNorm(field=names, doc=4993)"}]}

I don't really get the meaning of this but it seems that there ARE hits on  
gmbh as if it weren't a stopword although the \_settings show that gmbh is  
properly configured

"index.analysis.filter.my\_stopwords.stopwords.19" : "gmbh"

I am getting nuts here 🙂

On Tuesday, January 29, 2013 4:01:00 PM UTC+1, Martijn v Groningen wrote:

> I think the reason you see other companies that have these stopwords  
> is not because you had a match on the stop filter, but because the  
> `fuzzy_like_this` query has a low `min_similarity`. I suggest that you  
> increase it to something like `0.7` You have to test and play around  
> with it a bit.
> 
> If you really think that you have a hit on a stopword (which I don't  
> think happens) you can verify this the explain option:  
> [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/search/explain.html)
> 
> Martijn
> 
> On 29 January 2013 13:41, Haensel \<[theha...@gmail.com](mailto:theha...@gmail.com) \<javascript:\>\>  
> wrote:
> 
> > Thanks for your help but it still doesn't work ☹ Here is the new query:
> > 
> > "query":{"bool":  
> > {"should":[  
> > {"query\_string":{"query":"test
> 
> gmbh","fuzzy\_prefix\_length":3,"default\_operator":"AND","default\_field":"names"}},
> 
> > 
> 
> {"fuzzy\_like\_this":{"fields":["names"],"boost":1,"like\_text":"test
> 
> > gmbh","min\_similarity":0.2,"prefix\_length":0,"max\_query\_terms":25}}]  
> > }}  
> > }
> > 
> > Here's an example: When searching for a company that was indexed as  
> > "TestCompany" a query like "TestComp Limited" (notice the "misspelled"  
> > company name) will find a lot of "Limited" companies while "TestCompany"  
> > is  
> > the string I am really interested in. I simply want "TestCompany" (or a  
> > similar name via fuzzy search) to be first in the list, effectively  
> > ignoring  
> > all legal forms defined in my stopwords list.
> > 
> > On Tuesday, January 29, 2013 1:12:59 PM UTC+1, Martijn v Groningen  
> > wrote:
> > 
> > > Hi Hannes,
> > > 
> > > You're not specifying a field for both the `query_string` and  
> > > `fuzzy_like_this` queries. The behaviour is then to use the `_all`  
> > > field.  
> > > I think if you specify the `names` field then it should work. For the  
> > > `query_string` query use the `default_field` option and for the  
> > > `fuzzy_like_this` you need to use the `fields` option to use the your  
> > > `names` field.
> > > 
> > > Martijn
> > > 
> > > On 29 January 2013 11:37, Haensel [theha...@gmail.com](mailto:theha...@gmail.com) wrote:
> > > 
> > > > Hi,
> > > > 
> > > > I am using a combination of a query string (user searches via a  
> > > > "Google  
> > > > Search" like textbox) and a fuzzy query to be able to find misspelled  
> > > > names  
> > > > etc. Maybe the fuzzy search makes the stopwords useless? And if so,  
> > > > would  
> > > > there be a way around that?
> > > > 
> > > > {"bool":  
> > > > {"should":[  
> > > > {"query\_string":{"query":"gmbh","default\_operator":"AND"}},
> 
> {"fuzzy\_like\_this":{"boost":1,"like\_text":"gmbh","min\_similarity":0.5,"prefix\_length":0,"max\_query\_terms":25}}
> 
> > > > ]}  
> > > > }
> > > > 
> > > > Thanks,
> > > > 
> > > > Hannes
> > > > 
> > > > On Tuesday, January 29, 2013 9:40:57 AM UTC+1, Martijn v Groningen  
> > > > wrote:
> > > > 
> > > > > Hi Hannes,
> > > > > 
> > > > > How does the query look like?
> > > > > 
> > > > > Martijn
> > > > > 
> > > > > On 28 January 2013 20:31, Haensel [theha...@gmail.com](mailto:theha...@gmail.com) wrote:
> > > > > 
> > > > > > Hi,
> > > > > > 
> > > > > > I have the following problem: I have a list of company names but  
> > > > > > want  
> > > > > > to  
> > > > > > exclude the "form of the organization" (like Limited, LLC etc.) by  
> > > > > > using  
> > > > > > my  
> > > > > > own stopwords filter. So far so good, but I can't get it to work.  
> > > > > > It  
> > > > > > does  
> > > > > > the indexing, everything is searchable, but when searching for  
> > > > > > "LLC"  
> > > > > > etc  
> > > > > > I  
> > > > > > am still getting matches. Here is my config (I am using PHP syntax  
> > > > > > here,  
> > > > > > but  
> > > > > > I guess the values are obvious):
> > > > > > 
> > > > > > 'analysis' =\> array(  
> > > > > > 'analyzer' =\> array(  
> > > > > > 'name\_analyzer' =\> array(  
> > > > > > 'type' =\> 'custom',  
> > > > > > 'tokenizer' =\> 'standard',  
> > > > > > 'filter' =\>  
> > > > > > array('my\_stopwords','lowercase','icu\_normalizer','ngram')  
> > > > > > ),  
> > > > > > 'address\_analyzer' =\> array(  
> > > > > > 'type' =\> 'custom',  
> > > > > > 'tokenizer' =\> 'standard',  
> > > > > > 'filter' =\> array('standard','ngram')  
> > > > > > ),  
> > > > > > 'country\_analyzer' =\> array(  
> > > > > > 'type' =\> 'custom',  
> > > > > > 'tokenizer' =\> 'lowercase',  
> > > > > > 'filter' =\> array('country\_synonyms')  
> > > > > > ),  
> > > > > > ),  
> > > > > > 'filter' =\> array(  
> > > > > > 'ngram' =\> array(  
> > > > > > 'type' =\> 'nGram',  
> > > > > > 'min\_gram' =\> 1,  
> > > > > > 'max\_gram' =\> 5,  
> > > > > > ),  
> > > > > > 'country\_synonyms' =\> array(  
> > > > > > 'type' =\> 'synonym',  
> > > > > > 'synonyms' =\> array('some synonyms that work perfectly')  
> > > > > > ),  
> > > > > > 'my\_stopwords' =\> array(  
> > > > > > 'type' =\> 'stop',  
> > > > > > 'stopwords' =\> array('llc','gmbh',etc.etc.),  
> > > > > > 'ignore\_case' =\> true  
> > > > > > )  
> > > > > > )  
> > > > > > )
> > > > > > 
> > > > > > And here is my mapping:
> > > > > > 
> > > > > > 'names' =\> array(  
> > > > > > 'type' =\> 'string',  
> > > > > > 'analyzer' =\> 'name\_analyzer',  
> > > > > > 'index\_analyzer' =\> 'name\_analyzer',  
> > > > > > 'search\_analyzer' =\> 'name\_analyzer',  
> > > > > > 'include\_in\_all' =\> true  
> > > > > > ),  
> > > > > > 'addresses' =\> array(  
> > > > > > 'dynamic' =\> false,  
> > > > > > 'analyzer' =\> 'address\_analyzer',  
> > > > > > 'index\_analyzer' =\> 'address\_analyzer',  
> > > > > > 'search\_analyzer' =\> 'address\_analyzer',  
> > > > > > 'properties' =\> array(  
> > > > > > 'street' =\> array(  
> > > > > > 'type' =\> 'string',  
> > > > > > 'analyzer' =\> 'address\_analyzer',  
> > > > > > 'index\_analyzer' =\> 'address\_analyzer',  
> > > > > > 'search\_analyzer' =\> 'address\_analyzer',  
> > > > > > 'include\_in\_all' =\> true  
> > > > > > ),  
> > > > > > 'city' =\> array(  
> > > > > > 'type' =\> 'string',  
> > > > > > 'analyzer' =\> 'address\_analyzer',  
> > > > > > 'index\_analyzer' =\> 'address\_analyzer',  
> > > > > > 'search\_analyzer' =\> 'address\_analyzer',  
> > > > > > 'include\_in\_all' =\> true  
> > > > > > ),  
> > > > > > 'state' =\> array(  
> > > > > > 'type' =\> 'string',  
> > > > > > 'analyzer' =\> 'address\_analyzer',  
> > > > > > 'index\_analyzer' =\> 'address\_analyzer',  
> > > > > > 'search\_analyzer' =\> 'address\_analyzer',  
> > > > > > 'include\_in\_all' =\> true  
> > > > > > ),  
> > > > > > 'country' =\> array(  
> > > > > > 'type' =\> 'string',  
> > > > > > 'analyzer' =\> 'country\_analyzer',  
> > > > > > 'index\_analyzer' =\> 'country\_analyzer',  
> > > > > > 'search\_analyzer' =\> 'country\_analyzer',  
> > > > > > 'include\_in\_all' =\> true  
> > > > > > )  
> > > > > > )  
> > > > > > )
> > > > > > 
> > > > > > A GET request to myindex/\_settings shows that the values are set  
> > > > > > correctly.  
> > > > > > Example:
> > > > > > 
> > > > > > "index.analysis.filter.my\_stopwords.stopwords.90":"llc"
> > > > > > 
> > > > > > I feel pretty lost here. So any help would be really appreciated!
> > > > > > 
> > > > > > Thanks in advance,
> > > > > > 
> > > > > > Hannes
> > > > > > 
> > > > > > --  
> > > > > > You received this message because you are subscribed to the Google  
> > > > > > Groups  
> > > > > > "elasticsearch" group.  
> > > > > > To unsubscribe from this group, send email to  
> > > > > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).  
> > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > > > > 
> > > > > --  
> > > > > Met vriendelijke groet,
> > > > > 
> > > > > Martijn van Groningen
> > > 
> > > --  
> > > Met vriendelijke groet,
> > > 
> > > Martijn van Groningen
> 
> --  
> Met vriendelijke groet,
> 
> Martijn van Groningen

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Haensel](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/haensel/32/2595_2.png) [@Haensel](https://discuss.elastic.co/u/Haensel)\
**Post date:** [January 29, 2013, 7:11pm UTC](https://discuss.elastic.co/t/stop-words-filter-not-working/10530/8 "2013-01-29T19:11:58Z")

</div>

Ok, so after fiddling around a bit I seem to have found the reason for this  
behaviour:

curl -XGET  
'[http://127.0.0.1:9200/sanctionlists/\_analyze?pretty=1&text=gmbh&analyzer=name\_analyzer](http://127.0.0.1:9200/sanctionlists/_analyze?pretty=1&text=gmbh&analyzer=name_analyzer)'

results in:

{  
"tokens" : []  
}

whereas

curl -XGET  
'[http://127.0.0.1:9200/sanctionlists/\_analyze?pretty=1&text=Gmbh&analyzer=name\_analyzer](http://127.0.0.1:9200/sanctionlists/_analyze?pretty=1&text=Gmbh&analyzer=name_analyzer)'

results in:

{  
"tokens" : [ {  
"token" : "g",  
"start\_offset" : 0,  
"end\_offset" : 1,  
"type" : "word",  
"position" : 1  
}, {  
"token" : "m",  
"start\_offset" : 1,  
"end\_offset" : 2,  
"type" : "word",  
"position" : 2  
}, {  
"token" : "b",  
"start\_offset" : 2,  
"end\_offset" : 3,  
"type" : "word",  
"position" : 3  
}, {  
"token" : "h",  
"start\_offset" : 3,  
"end\_offset" : 4,  
"type" : "word",  
"position" : 4  
}, {  
"token" : "gm",  
"start\_offset" : 0,  
"end\_offset" : 2,  
"type" : "word",  
"position" : 5  
}, {  
"token" : "mb",  
"start\_offset" : 1,  
"end\_offset" : 3,  
"type" : "word",  
"position" : 6  
}, {  
"token" : "bh",  
"start\_offset" : 2,  
"end\_offset" : 4,  
"type" : "word",  
"position" : 7  
}, {  
"token" : "gmb",  
"start\_offset" : 0,  
"end\_offset" : 3,  
"type" : "word",  
"position" : 8  
}, {  
"token" : "mbh",  
"start\_offset" : 1,  
"end\_offset" : 4,  
"type" : "word",  
"position" : 9  
}, {  
"token" : "gmbh",  
"start\_offset" : 0,  
"end\_offset" : 4,  
"type" : "word",  
"position" : 10  
} ]

So the analyzer seems not to ignore the case of the stopword even if  
ignore\_case is set to true. So I set a lowercase filter in FRONT of the  
stopword filter and this did the trick for the examples above. The next  
thing was that the fuzzy\_like\_this query does not analyze a query  
(splitting it into tokens) as it seems. So I tried a match query on the  
names field with the proper fuzziness (0.5), setting the analyzer to  
"name\_analyzer" and this actually worked!

I am not sure wheter this is a bug or intended when using FLT queries. Any  
thoughts?

Thanks,

Hannes

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![mvg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mvg/32/98890_2.png) [@mvg](https://discuss.elastic.co/u/mvg)\
**Post date:** [January 29, 2013, 10:03pm UTC](https://discuss.elastic.co/t/stop-words-filter-not-working/10530/9 "2013-01-29T22:03:05Z")

</div>

> So the analyzer seems not to ignore the case of the stopword even if  
> ignore\_case is set to true. So I set a lowercase filter in FRONT of the  
> stopword filter and this did the trick for the examples above.  
> Off course! Too bad I missed that one...

> thing was that the fuzzy\_like\_this query does not analyze a query (splitting  
> it into tokens) as it seems. So I tried a match query on the names field  
> with the proper fuzziness (0.5), setting the analyzer to "name\_analyzer" and  
> this actually worked!
> 
> I am not sure wheter this is a bug or intended when using FLT queries. Any  
> thoughts?  
> This is not a bug, but just how the `fuzzy_like_this` query works. It  
> does analyze the query, but uses a default analyzer (unless you  
> override it via the `analyzer` option). The `match` query uses by  
> default the analyzer of the specified field. Maybe in your case it is  
> better to use the `fuzzy_like_this_field` query instead of the  
> `fuzzy_like_this` query, that needs a field as required parameter.  
> This query by default uses the search analyzer of the specified field.

Martijn

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 2:54am UTC](https://discuss.elastic.co/t/stop-words-filter-not-working/10530/10 "2017-07-06T02:54:06Z")

</div>