# Stop word filter problem

**URL:** https://discuss.elastic.co/t/stop-word-filter-problem/11071
**Category:** Elasticsearch
**Created:** [March 8, 2013, 8:37am UTC](https://discuss.elastic.co/t/stop-word-filter-problem/11071 "2013-03-08T08:37:46Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![Rajesh\_Tarle](https://avatars.discourse-cdn.com/v4/letter/r/3da27b/32.png) [@Rajesh\_Tarle](https://discuss.elastic.co/u/Rajesh_Tarle)
#### Post date: [March 8, 2013, 8:37am UTC](https://discuss.elastic.co/t/stop-word-filter-problem/11071/1 "2013-03-08T08:37:46Z")

</div>

Hi,

I am using elastic search 0.20.5. when user stop word filter is not working.

my analyzer setting is in following link

> <https://gist.github.com/rajesh07/16f4940de3afefba716e>

and my stop word file is

> <https://gist.github.com/rajesh07/0c9865d8abbbf0335b89>

I am creating index using following values  
The oracle  
The Rajesh  
oracle company  
The  
oracle  
Rajesh company  
company  
Rajesh

I got following indexes  
it didn't remove stop word from values.  
{id=4, CompanyName=The oracle}  
{id=5, CompanyName=The Rajesh}  
{id=1, CompanyName=oracle company}  
{id=6, CompanyName=The}  
{id=2, CompanyName=oracle}  
{id=7, CompanyName=Rajesh company}  
{id=3, CompanyName=company}  
{id=8, CompanyName=Rajesh}

and when I search using "_company_" I got all record that contains "\*company  
\*" like  
{id=1, CompanyName=oracle company}  
{id=7, CompanyName=Rajesh company}  
{id=3, CompanyName=company}

i think it's wrong because company in stop word file.

I want the output like following  
if i search for "company" then i must "get no record".  
if i search for "oracle" the i must got  
oracle  
oralce(without "the" because "the" is also available in stopword file )  
oralce(without "company" because "company" is also available in stopword  
file )

and when i search for rajesh then search result is  
Rajesh  
Rajesh(without "the" because "the" is also available in stopword file )  
Rajesh(without "company" because "company" is also available in stopword  
file )

But i didn't get this result.  
i think stopword is not working.  
please help

Thanks  
Rajesh

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

### Author: ![simonw\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simonw_2/32/1130_2.png) [@simonw\_2](https://discuss.elastic.co/u/simonw_2)
#### Post date: [March 8, 2013, 10:43am UTC](https://discuss.elastic.co/t/stop-word-filter-problem/11071/2 "2013-03-08T10:43:37Z")

</div>

hey,

are you using the analyzers in your mapping? how do you create the index  
ie. how does you mapping look like?

simon

On Friday, March 8, 2013 9:37:46 AM UTC+1, raj wrote:

> Hi,
> 
> I am using Elasticsearch 0.20.5. when user stop word filter is not  
> working.
> 
> my analyzer setting is in following link  
> [This is my mainanalyzer code that I use at index and search tyme · GitHub](https://gist.github.com/rajesh07/16f4940de3afefba716e)
> 
> and my stop word file is  
> [This is my stop word file that contain stop words · GitHub](https://gist.github.com/rajesh07/0c9865d8abbbf0335b89)
> 
> I am creating index using following values  
> The oracle  
> The Rajesh  
> oracle company  
> The  
> oracle  
> Rajesh company  
> company  
> Rajesh
> 
> I got following indexes  
> it didn't remove stop word from values.  
> {id=4, CompanyName=The oracle}  
> {id=5, CompanyName=The Rajesh}  
> {id=1, CompanyName=oracle company}  
> {id=6, CompanyName=The}  
> {id=2, CompanyName=oracle}  
> {id=7, CompanyName=Rajesh company}  
> {id=3, CompanyName=company}  
> {id=8, CompanyName=Rajesh}
> 
> and when I search using "_company_" I got all record that contains "\*  
> company\*" like  
> {id=1, CompanyName=oracle company}  
> {id=7, CompanyName=Rajesh company}  
> {id=3, CompanyName=company}
> 
> i think it's wrong because company in stop word file.
> 
> I want the output like following  
> if i search for "company" then i must "get no record".  
> if i search for "oracle" the i must got  
> oracle  
> oralce(without "the" because "the" is also available in stopword file )  
> oralce(without "company" because "company" is also available in stopword  
> file )
> 
> and when i search for rajesh then search result is  
> Rajesh  
> Rajesh(without "the" because "the" is also available in stopword file )  
> Rajesh(without "company" because "company" is also available in stopword  
> file )
> 
> But i didn't get this result.  
> i think stopword is not working.  
> please help
> 
> Thanks  
> Rajesh

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

### Author: ![Rajesh\_Tarle](https://avatars.discourse-cdn.com/v4/letter/r/3da27b/32.png) [@Rajesh\_Tarle](https://discuss.elastic.co/u/Rajesh_Tarle)
#### Post date: [March 8, 2013, 10:56am UTC](https://discuss.elastic.co/t/stop-word-filter-problem/11071/3 "2013-03-08T10:56:51Z")

</div>

hi,  
The mapping of analyzer look like

_"primary" : {_

- "index\_analyzer" : "mainindexanalyzer",\*  
_"search\_analyzer" : "mainsearchanalyzer"_  
_}_

and create index look like

_{ "index" : { "\_index" :"idx", "\_type" :"primary", "\_id" :"1"}}_  
_{ "primary" :{"id":1,"CompanyName":"oracle company"}}_  
_{ "index" : { "\_index" :"idx", "\_type" :"primary", "\_id" :"2"}}_  
_{ "primary" :{"id":2,"CompanyName":"oracle"}}_  
_{ "index" : { "\_index" :"idx", "\_type" :"primary", "\_id" :"3"}}_  
_{ "primary" :{"id":3,"CompanyName":"company"}}_  
_{ "index" : { "\_index" :"idx", "\_type" :"primary", "\_id" :"4"}}_  
_{ "primary" :{"id":4,"CompanyName":"The oracle"}}_  
_{ "index" : { "\_index" :"idx", "\_type" :"primary", "\_id" :"5"}}_  
_{ "primary" :{"id":5,"CompanyName":"The Rajesh"}}_  
_{ "index" : { "\_index" :"idx", "\_type" :"primary", "\_id" :"6"}}_  
_{ "primary" :{"id":6,"CompanyName":"The"}}_  
_{ "index" : { "\_index" :"idx", "\_type" :"primary", "\_id" :"7"}}_  
_{ "primary" :{"id":7,"CompanyName":"Rajesh company"}}_  
_{ "index" : { "\_index" :"idx", "\_type" :"primary", "\_id" :"8"}}_  
_{ "primary" :{"id":8,"CompanyName":"Rajesh"}}_

Thanks  
Rajesh

On Friday, March 8, 2013 4:13:37 PM UTC+5:30, simonw wrote:

> hey,
> 
> are you using the analyzers in your mapping? how do you create the index  
> ie. how does you mapping look like?
> 
> simon
> 
> On Friday, March 8, 2013 9:37:46 AM UTC+1, raj wrote:
> 
> > Hi,
> > 
> > I am using Elasticsearch 0.20.5. when user stop word filter is not  
> > working.
> > 
> > my analyzer setting is in following link  
> > [This is my mainanalyzer code that I use at index and search tyme · GitHub](https://gist.github.com/rajesh07/16f4940de3afefba716e)
> > 
> > and my stop word file is  
> > [This is my stop word file that contain stop words · GitHub](https://gist.github.com/rajesh07/0c9865d8abbbf0335b89)
> > 
> > I am creating index using following values  
> > The oracle  
> > The Rajesh  
> > oracle company  
> > The  
> > oracle  
> > Rajesh company  
> > company  
> > Rajesh
> > 
> > I got following indexes  
> > it didn't remove stop word from values.  
> > {id=4, CompanyName=The oracle}  
> > {id=5, CompanyName=The Rajesh}  
> > {id=1, CompanyName=oracle company}  
> > {id=6, CompanyName=The}  
> > {id=2, CompanyName=oracle}  
> > {id=7, CompanyName=Rajesh company}  
> > {id=3, CompanyName=company}  
> > {id=8, CompanyName=Rajesh}
> > 
> > and when I search using "_company_" I got all record that contains "\*  
> > company\*" like  
> > {id=1, CompanyName=oracle company}  
> > {id=7, CompanyName=Rajesh company}  
> > {id=3, CompanyName=company}
> > 
> > i think it's wrong because company in stop word file.
> > 
> > I want the output like following  
> > if i search for "company" then i must "get no record".  
> > if i search for "oracle" the i must got  
> > oracle  
> > oralce(without "the" because "the" is also available in stopword file )  
> > oralce(without "company" because "company" is also available in stopword  
> > file )
> > 
> > and when i search for rajesh then search result is  
> > Rajesh  
> > Rajesh(without "the" because "the" is also available in stopword file )  
> > Rajesh(without "company" because "company" is also available in stopword  
> > file )
> > 
> > But i didn't get this result.  
> > i think stopword is not working.  
> > please help
> > 
> > Thanks  
> > Rajesh

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

### Author: ![Rajesh\_Tarle](https://avatars.discourse-cdn.com/v4/letter/r/3da27b/32.png) [@Rajesh\_Tarle](https://discuss.elastic.co/u/Rajesh_Tarle)
#### Post date: [March 8, 2013, 11:11am UTC](https://discuss.elastic.co/t/stop-word-filter-problem/11071/4 "2013-03-08T11:11:57Z")

</div>

I am using rabbitmq river for creating an index.

The sequence of defining analyzer first define analyzer  
mapping(index/search) on index then use rabbitmq river to insert the data  
on index.

1.CreateIndexResponse createindexresponseactionGet =  
client.admin().indices().create(new  
CreateIndexRequest(indexName).settings(analyzerSettingSources)).actionGet();

1. client.prepareIndex("\_river", "river",  
"\_meta").setSource(jsonBuilder().startObject().field("type",  
"rabbitmq").field("rabbitmq", rabbitMqRiversettings).field("index",  
riverIndexSetting).endObject()).execute().actionGet();

please look on it and provide any suggestion.

Thanks

On Friday, March 8, 2013 4:13:37 PM UTC+5:30, simonw wrote:

> hey,
> 
> are you using the analyzers in your mapping? how do you create the index  
> ie. how does you mapping look like?
> 
> simon
> 
> On Friday, March 8, 2013 9:37:46 AM UTC+1, raj wrote:
> 
> > Hi,
> > 
> > I am using Elasticsearch 0.20.5. when user stop word filter is not  
> > working.
> > 
> > my analyzer setting is in following link  
> > [This is my mainanalyzer code that I use at index and search tyme · GitHub](https://gist.github.com/rajesh07/16f4940de3afefba716e)
> > 
> > and my stop word file is  
> > [This is my stop word file that contain stop words · GitHub](https://gist.github.com/rajesh07/0c9865d8abbbf0335b89)
> > 
> > I am creating index using following values  
> > The oracle  
> > The Rajesh  
> > oracle company  
> > The  
> > oracle  
> > Rajesh company  
> > company  
> > Rajesh
> > 
> > I got following indexes  
> > it didn't remove stop word from values.  
> > {id=4, CompanyName=The oracle}  
> > {id=5, CompanyName=The Rajesh}  
> > {id=1, CompanyName=oracle company}  
> > {id=6, CompanyName=The}  
> > {id=2, CompanyName=oracle}  
> > {id=7, CompanyName=Rajesh company}  
> > {id=3, CompanyName=company}  
> > {id=8, CompanyName=Rajesh}
> > 
> > and when I search using "_company_" I got all record that contains "\*  
> > company\*" like  
> > {id=1, CompanyName=oracle company}  
> > {id=7, CompanyName=Rajesh company}  
> > {id=3, CompanyName=company}
> > 
> > i think it's wrong because company in stop word file.
> > 
> > I want the output like following  
> > if i search for "company" then i must "get no record".  
> > if i search for "oracle" the i must got  
> > oracle  
> > oralce(without "the" because "the" is also available in stopword file )  
> > oralce(without "company" because "company" is also available in stopword  
> > file )
> > 
> > and when i search for rajesh then search result is  
> > Rajesh  
> > Rajesh(without "the" because "the" is also available in stopword file )  
> > Rajesh(without "company" because "company" is also available in stopword  
> > file )
> > 
> > But i didn't get this result.  
> > i think stopword is not working.  
> > please help
> > 
> > Thanks  
> > Rajesh

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

### Author: ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)
#### Post date: [March 8, 2013, 3:11pm UTC](https://discuss.elastic.co/t/stop-word-filter-problem/11071/5 "2013-03-08T15:11:02Z")

</div>

The stopword filter comes after the lowercase filter in your analyzer,  
therefore the stopwords should all be lowercased in the file or set  
ignore\_case to true.

--  
Ivan

On Fri, Mar 8, 2013 at 3:11 AM, raj [rajesh.elasticsearch@gmail.com](mailto:rajesh.elasticsearch@gmail.com) wrote:

> I am using rabbitmq river for creating an index.
> 
> The sequence of defining analyzer first define analyzer  
> mapping(index/search) on index then use rabbitmq river to insert the data  
> on index.
> 
> 1.CreateIndexResponse createindexresponseactionGet =  
> client.admin().indices().create(new  
> CreateIndexRequest(indexName).settings(analyzerSettingSources)).actionGet();
> 
> 1. client.prepareIndex("\_river", "river",  
> "\_meta").setSource(jsonBuilder().startObject().field("type",  
> "rabbitmq").field("rabbitmq", rabbitMqRiversettings).field("index",  
> riverIndexSetting).endObject()).execute().actionGet();
> 
> please look on it and provide any suggestion.
> 
> Thanks
> 
> On Friday, March 8, 2013 4:13:37 PM UTC+5:30, simonw wrote:
> 
> > hey,
> > 
> > are you using the analyzers in your mapping? how do you create the index  
> > ie. how does you mapping look like?
> > 
> > simon
> > 
> > On Friday, March 8, 2013 9:37:46 AM UTC+1, raj wrote:
> > 
> > > Hi,
> > > 
> > > I am using Elasticsearch 0.20.5. when user stop word filter is not  
> > > working.
> > > 
> > > my analyzer setting is in following link  
> > > [https://gist.github.com/\*\*rajesh07/16f4940de3afefba716e](https://gist.github.com/**rajesh07/16f4940de3afefba716e)[https://gist.github.com/rajesh07/16f4940de3afefba716e](https://gist.github.com/rajesh07/16f4940de3afefba716e)
> > > 
> > > and my stop word file is  
> > > [https://gist.github.com/\*\*rajesh07/0c9865d8abbbf0335b89](https://gist.github.com/**rajesh07/0c9865d8abbbf0335b89)[https://gist.github.com/rajesh07/0c9865d8abbbf0335b89](https://gist.github.com/rajesh07/0c9865d8abbbf0335b89)
> > > 
> > > I am creating index using following values  
> > > The oracle  
> > > The Rajesh  
> > > oracle company  
> > > The  
> > > oracle  
> > > Rajesh company  
> > > company  
> > > Rajesh
> > > 
> > > I got following indexes  
> > > it didn't remove stop word from values.  
> > > {id=4, CompanyName=The oracle}  
> > > {id=5, CompanyName=The Rajesh}  
> > > {id=1, CompanyName=oracle company}  
> > > {id=6, CompanyName=The}  
> > > {id=2, CompanyName=oracle}  
> > > {id=7, CompanyName=Rajesh company}  
> > > {id=3, CompanyName=company}  
> > > {id=8, CompanyName=Rajesh}
> > > 
> > > and when I search using "_company_" I got all record that contains "\*  
> > > company\*" like  
> > > {id=1, CompanyName=oracle company}  
> > > {id=7, CompanyName=Rajesh company}  
> > > {id=3, CompanyName=company}
> > > 
> > > i think it's wrong because company in stop word file.
> > > 
> > > I want the output like following  
> > > if i search for "company" then i must "get no record".  
> > > if i search for "oracle" the i must got  
> > > oracle  
> > > oralce(without "the" because "the" is also available in stopword file )  
> > > oralce(without "company" because "company" is also available in stopword  
> > > file )
> > > 
> > > and when i search for rajesh then search result is  
> > > Rajesh  
> > > Rajesh(without "the" because "the" is also available in stopword file )  
> > > Rajesh(without "company" because "company" is also available in stopword  
> > > file )
> > > 
> > > But i didn't get this result.  
> > > i think stopword is not working.  
> > > please help
> > > 
> > > Thanks  
> > > Rajesh
> > > 
> > > --  
> > > You received this message because you are subscribed to the Google Groups  
> > > "elasticsearch" group.  
> > > To unsubscribe from this group and stop receiving emails from it, send an  
> > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 2:47am UTC](https://discuss.elastic.co/t/stop-word-filter-problem/11071/6 "2017-07-06T02:47:30Z")

</div>


