Stop word filter problem

Rajesh_Tarle · March 8, 2013, 8:37am

Hi,

I am using elastic search 0.20.5. when user stop word filter is not working.

my analyzer setting is in following link

gist.github.com

https://gist.github.com/rajesh07/16f4940de3afefba716e

AnalyzerMapping.json

 {  
  "index" : {
"analysis" : {
"analyzer" : {
 "mainindexanalyzer" : {
"type":"custom",
"tokenizer" : "whitespace",
"filter" : ["lowercase","asciifolding","mystopword"],
"char_filter" :["html_strip"]
 },

This file has been truncated. show original

and my stop word file is

I am creating index using following values
The oracle
The Rajesh
oracle company
The
oracle
Rajesh company
company
Rajesh

I got following indexes
it didn't remove stop word from values.
{id=4, CompanyName=The oracle}
{id=5, CompanyName=The Rajesh}
{id=1, CompanyName=oracle company}
{id=6, CompanyName=The}
{id=2, CompanyName=oracle}
{id=7, CompanyName=Rajesh company}
{id=3, CompanyName=company}
{id=8, CompanyName=Rajesh}

and when I search using "company" I got all record that contains "*company
*" like
{id=1, CompanyName=oracle company}
{id=7, CompanyName=Rajesh company}
{id=3, CompanyName=company}

i think it's wrong because company in stop word file.

I want the output like following
if i search for "company" then i must "get no record".
if i search for "oracle" the i must got
oracle
oralce(without "the" because "the" is also available in stopword file )
oralce(without "company" because "company" is also available in stopword
file )

and when i search for rajesh then search result is
Rajesh
Rajesh(without "the" because "the" is also available in stopword file )
Rajesh(without "company" because "company" is also available in stopword
file )

But i didn't get this result.
i think stopword is not working.
please help

Thanks
Rajesh

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

simonw_2 · March 8, 2013, 10:43am

hey,

are you using the analyzers in your mapping? how do you create the index
ie. how does you mapping look like?

simon

On Friday, March 8, 2013 9:37:46 AM UTC+1, raj wrote:

Hi,

I am using Elasticsearch 0.20.5. when user stop word filter is not
working.

my analyzer setting is in following link
This is my mainanalyzer code that I use at index and search tyme · GitHub

and my stop word file is
This is my stop word file that contain stop words · GitHub

I am creating index using following values
The oracle
The Rajesh
oracle company
The
oracle
Rajesh company
company
Rajesh

I got following indexes
it didn't remove stop word from values.
{id=4, CompanyName=The oracle}
{id=5, CompanyName=The Rajesh}
{id=1, CompanyName=oracle company}
{id=6, CompanyName=The}
{id=2, CompanyName=oracle}
{id=7, CompanyName=Rajesh company}
{id=3, CompanyName=company}
{id=8, CompanyName=Rajesh}

and when I search using "company" I got all record that contains "*
company*" like
{id=1, CompanyName=oracle company}
{id=7, CompanyName=Rajesh company}
{id=3, CompanyName=company}

i think it's wrong because company in stop word file.

I want the output like following
if i search for "company" then i must "get no record".
if i search for "oracle" the i must got
oracle
oralce(without "the" because "the" is also available in stopword file )
oralce(without "company" because "company" is also available in stopword
file )

and when i search for rajesh then search result is
Rajesh
Rajesh(without "the" because "the" is also available in stopword file )
Rajesh(without "company" because "company" is also available in stopword
file )

But i didn't get this result.
i think stopword is not working.
please help

Thanks
Rajesh

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Rajesh_Tarle · March 8, 2013, 10:56am

hi,
The mapping of analyzer look like

"primary" : {

"index_analyzer" : "mainindexanalyzer",*
"search_analyzer" : "mainsearchanalyzer"
}

and create index look like

{ "index" : { "_index" :"idx", "_type" :"primary", "_id" :"1"}}
{ "primary" :{"id":1,"CompanyName":"oracle company"}}
{ "index" : { "_index" :"idx", "_type" :"primary", "_id" :"2"}}
{ "primary" :{"id":2,"CompanyName":"oracle"}}
{ "index" : { "_index" :"idx", "_type" :"primary", "_id" :"3"}}
{ "primary" :{"id":3,"CompanyName":"company"}}
{ "index" : { "_index" :"idx", "_type" :"primary", "_id" :"4"}}
{ "primary" :{"id":4,"CompanyName":"The oracle"}}
{ "index" : { "_index" :"idx", "_type" :"primary", "_id" :"5"}}
{ "primary" :{"id":5,"CompanyName":"The Rajesh"}}
{ "index" : { "_index" :"idx", "_type" :"primary", "_id" :"6"}}
{ "primary" :{"id":6,"CompanyName":"The"}}
{ "index" : { "_index" :"idx", "_type" :"primary", "_id" :"7"}}
{ "primary" :{"id":7,"CompanyName":"Rajesh company"}}
{ "index" : { "_index" :"idx", "_type" :"primary", "_id" :"8"}}
{ "primary" :{"id":8,"CompanyName":"Rajesh"}}

Thanks
Rajesh

On Friday, March 8, 2013 4:13:37 PM UTC+5:30, simonw wrote:

hey,

are you using the analyzers in your mapping? how do you create the index
ie. how does you mapping look like?

simon

On Friday, March 8, 2013 9:37:46 AM UTC+1, raj wrote:

Hi,

I am using Elasticsearch 0.20.5. when user stop word filter is not
working.

my analyzer setting is in following link
This is my mainanalyzer code that I use at index and search tyme · GitHub

and my stop word file is
This is my stop word file that contain stop words · GitHub

I am creating index using following values
The oracle
The Rajesh
oracle company
The
oracle
Rajesh company
company
Rajesh

I got following indexes
it didn't remove stop word from values.
{id=4, CompanyName=The oracle}
{id=5, CompanyName=The Rajesh}
{id=1, CompanyName=oracle company}
{id=6, CompanyName=The}
{id=2, CompanyName=oracle}
{id=7, CompanyName=Rajesh company}
{id=3, CompanyName=company}
{id=8, CompanyName=Rajesh}

and when I search using "company" I got all record that contains "*
company*" like
{id=1, CompanyName=oracle company}
{id=7, CompanyName=Rajesh company}
{id=3, CompanyName=company}

i think it's wrong because company in stop word file.

I want the output like following
if i search for "company" then i must "get no record".
if i search for "oracle" the i must got
oracle
oralce(without "the" because "the" is also available in stopword file )
oralce(without "company" because "company" is also available in stopword
file )

and when i search for rajesh then search result is
Rajesh
Rajesh(without "the" because "the" is also available in stopword file )
Rajesh(without "company" because "company" is also available in stopword
file )

But i didn't get this result.
i think stopword is not working.
please help

Thanks
Rajesh

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Rajesh_Tarle · March 8, 2013, 11:11am

I am using rabbitmq river for creating an index.

The sequence of defining analyzer first define analyzer
mapping(index/search) on index then use rabbitmq river to insert the data
on index.

1.CreateIndexResponse createindexresponseactionGet =
client.admin().indices().create(new
CreateIndexRequest(indexName).settings(analyzerSettingSources)).actionGet();

client.prepareIndex("_river", "river",
"_meta").setSource(jsonBuilder().startObject().field("type",
"rabbitmq").field("rabbitmq", rabbitMqRiversettings).field("index",
riverIndexSetting).endObject()).execute().actionGet();

please look on it and provide any suggestion.

Thanks

On Friday, March 8, 2013 4:13:37 PM UTC+5:30, simonw wrote:

hey,

are you using the analyzers in your mapping? how do you create the index
ie. how does you mapping look like?

simon

On Friday, March 8, 2013 9:37:46 AM UTC+1, raj wrote:

Hi,

I am using Elasticsearch 0.20.5. when user stop word filter is not
working.

my analyzer setting is in following link
This is my mainanalyzer code that I use at index and search tyme · GitHub

and my stop word file is
This is my stop word file that contain stop words · GitHub

I am creating index using following values
The oracle
The Rajesh
oracle company
The
oracle
Rajesh company
company
Rajesh

I got following indexes
it didn't remove stop word from values.
{id=4, CompanyName=The oracle}
{id=5, CompanyName=The Rajesh}
{id=1, CompanyName=oracle company}
{id=6, CompanyName=The}
{id=2, CompanyName=oracle}
{id=7, CompanyName=Rajesh company}
{id=3, CompanyName=company}
{id=8, CompanyName=Rajesh}

and when I search using "company" I got all record that contains "*
company*" like
{id=1, CompanyName=oracle company}
{id=7, CompanyName=Rajesh company}
{id=3, CompanyName=company}

i think it's wrong because company in stop word file.

I want the output like following
if i search for "company" then i must "get no record".
if i search for "oracle" the i must got
oracle
oralce(without "the" because "the" is also available in stopword file )
oralce(without "company" because "company" is also available in stopword
file )

and when i search for rajesh then search result is
Rajesh
Rajesh(without "the" because "the" is also available in stopword file )
Rajesh(without "company" because "company" is also available in stopword
file )

But i didn't get this result.
i think stopword is not working.
please help

Thanks
Rajesh

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Ivan · March 8, 2013, 3:11pm

The stopword filter comes after the lowercase filter in your analyzer,
therefore the stopwords should all be lowercased in the file or set
ignore_case to true.

--
Ivan

On Fri, Mar 8, 2013 at 3:11 AM, raj rajesh.elasticsearch@gmail.com wrote:

I am using rabbitmq river for creating an index.

The sequence of defining analyzer first define analyzer
mapping(index/search) on index then use rabbitmq river to insert the data
on index.

1.CreateIndexResponse createindexresponseactionGet =
client.admin().indices().create(new
CreateIndexRequest(indexName).settings(analyzerSettingSources)).actionGet();

client.prepareIndex("_river", "river",
"_meta").setSource(jsonBuilder().startObject().field("type",
"rabbitmq").field("rabbitmq", rabbitMqRiversettings).field("index",
riverIndexSetting).endObject()).execute().actionGet();

please look on it and provide any suggestion.

Thanks

On Friday, March 8, 2013 4:13:37 PM UTC+5:30, simonw wrote:

hey,

are you using the analyzers in your mapping? how do you create the index
ie. how does you mapping look like?

simon

On Friday, March 8, 2013 9:37:46 AM UTC+1, raj wrote:

Hi,

I am using Elasticsearch 0.20.5. when user stop word filter is not
working.

my analyzer setting is in following link
https://gist.github.com/**rajesh07/16f4940de3afefba716e https://gist.github.com/rajesh07/16f4940de3afefba716e

and my stop word file is
https://gist.github.com/**rajesh07/0c9865d8abbbf0335b89 https://gist.github.com/rajesh07/0c9865d8abbbf0335b89

I am creating index using following values
The oracle
The Rajesh
oracle company
The
oracle
Rajesh company
company
Rajesh

I got following indexes
it didn't remove stop word from values.
{id=4, CompanyName=The oracle}
{id=5, CompanyName=The Rajesh}
{id=1, CompanyName=oracle company}
{id=6, CompanyName=The}
{id=2, CompanyName=oracle}
{id=7, CompanyName=Rajesh company}
{id=3, CompanyName=company}
{id=8, CompanyName=Rajesh}

and when I search using "company" I got all record that contains "*
company*" like
{id=1, CompanyName=oracle company}
{id=7, CompanyName=Rajesh company}
{id=3, CompanyName=company}

i think it's wrong because company in stop word file.

I want the output like following
if i search for "company" then i must "get no record".
if i search for "oracle" the i must got
oracle
oralce(without "the" because "the" is also available in stopword file )
oralce(without "company" because "company" is also available in stopword
file )

and when i search for rajesh then search result is
Rajesh
Rajesh(without "the" because "the" is also available in stopword file )
Rajesh(without "company" because "company" is also available in stopword
file )

But i didn't get this result.
i think stopword is not working.
please help

Thanks
Rajesh

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Topic		Replies	Views
Stopwords in analyzer doesn't seem to work Elasticsearch	3	383	June 26, 2020
Stop words are not working Elasticsearch	7	2569	July 5, 2017
Stop words filter not working Elasticsearch	9	2335	July 6, 2017
My stopwords filter is not working Elasticsearch	5	1935	July 6, 2017
It does not work "Stop Word" Elasticsearch	7	393	July 6, 2017

Stop word filter problem

Related topics