Problem with synonym token filter

Rajesh_Tarle · February 19, 2013, 1:39pm

Hi

I am newbie with elastic search.i need to use synonym filter to meet my
requirement.

my used mapping details and synonym text file on :

gist.github.com

https://gist.github.com/rajesh07/4985901

mapping.json

  "index" : {
"analysis" : {
"analyzer" : {
 "manualindexanalyzer" : {
"type":"custom",
"tokenizer" : "whitespace",
"filter" : ["lowercase","asciifolding","length","mystopword","myworddelimiter","myshingle","mystemmer","myelision"],
"char_filter" :["html_strip"]
 },
"manualsearchanalyzer" : {

This file has been truncated. show original

synonym.txt

bit-depth,bit depth,bitdepth
technologies,technology,tech
Association,Assc
Department,Dept,Dpt
Bank,Banc,Banking,Banco
services,service
Goverment,Govt
oracle company,sap

Requirement:

I need to search both direction means*

oracle=sap and sap=oracle*
This it's works nicely but when using below one :

oracle company=sap and sap =oracle company

It doesn't work.

when i remove the space between oracle company (like oraclecompany) it's
give me desired results.

so final conclusion from my side that synonym does not support space
between word combination.

please help me to sorted out this problem.

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

egaumer · February 19, 2013, 1:51pm

You need to use the proper tokenizer.

"The tokenizer parameter controls the tokenizers that will be used to
tokenize the synonym, and defaults to the whitespace tokenizer."

On Tuesday, February 19, 2013 8:39:14 AM UTC-5, raj wrote:

Hi

I am newbie with Elasticsearch.i need to use synonym filter to meet my
requirement.

my used mapping details and synonym text file on :
Analyzer Mapping · GitHub

Requirement:

I need to search both direction means*

oracle=sap and sap=oracle*
This it's works nicely but when using below one :

oracle company=sap and sap =oracle company

It doesn't work.

when i remove the space between oracle company (like oraclecompany) it's
give me desired results.

so final conclusion from my side that synonym does not support space
between word combination.

please help me to sorted out this problem.

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Rajesh_Tarle · February 20, 2013, 6:45am

hi,

Thanks for reply,
may you explain me. which type of tokenizer should use for this purpose.
I have used used letter,standard and whitespacedelimeter but not got any
solution.

Thanks

Raj

On Tuesday, February 19, 2013 7:21:14 PM UTC+5:30, egaumer wrote:

You need to use the proper tokenizer.

Elasticsearch Platform — Find real-time answers at scale | Elastic

"The tokenizer parameter controls the tokenizers that will be used to
tokenize the synonym, and defaults to the whitespace tokenizer."

On Tuesday, February 19, 2013 8:39:14 AM UTC-5, raj wrote:

Hi

I am newbie with Elasticsearch.i need to use synonym filter to meet my
requirement.

my used mapping details and synonym text file on :
Analyzer Mapping · GitHub

Requirement:

I need to search both direction means*

oracle=sap and sap=oracle*
This it's works nicely but when using below one :

oracle company=sap and sap =oracle company

It doesn't work.

when i remove the space between oracle company (like oraclecompany) it's
give me desired results.

so final conclusion from my side that synonym does not support space
between word combination.

please help me to sorted out this problem.

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

ppearcy · February 20, 2013, 3:01pm

Sounds like you want synonym phrases. My recommendation is to:

Perform index time synonym expansion
On the search side do NOT include the synonym filter in your analyzer

If this doesn't help, please provide examples showing what you believe to
be incorrect behavior.

Thanks,
Paul

On Tuesday, February 19, 2013 11:45:43 PM UTC-7, raj wrote:

hi,

Thanks for reply,
may you explain me. which type of tokenizer should use for this purpose.
I have used used letter,standard and whitespacedelimeter but not got any
solution.

Thanks

Raj

On Tuesday, February 19, 2013 7:21:14 PM UTC+5:30, egaumer wrote:

You need to use the proper tokenizer.

Elasticsearch Platform — Find real-time answers at scale | Elastic

"The tokenizer parameter controls the tokenizers that will be used to
tokenize the synonym, and defaults to the whitespace tokenizer."

On Tuesday, February 19, 2013 8:39:14 AM UTC-5, raj wrote:

Hi

I am newbie with Elasticsearch.i need to use synonym filter to meet my
requirement.

my used mapping details and synonym text file on :
Analyzer Mapping · GitHub

Requirement:

I need to search both direction means*

oracle=sap and sap=oracle*
This it's works nicely but when using below one :

oracle company=sap and sap =oracle company

It doesn't work.

when i remove the space between oracle company (like oraclecompany) it's
give me desired results.

so final conclusion from my side that synonym does not support space
between word combination.

please help me to sorted out this problem.

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Rajesh_Tarle · February 27, 2013, 12:21pm

On Tuesday, February 19, 2013 7:09:14 PM UTC+5:30, raj wrote:

Hi

I am newbie with Elasticsearch.i need to use synonym filter to meet my
requirement.

my used mapping details and synonym text file on :
Analyzer Mapping · GitHub

Requirement:

I need to search both direction means*

oracle=sap and sap=oracle*
This it's works nicely but when using below one :

oracle company=sap and sap =oracle company

It doesn't work.

when i remove the space between oracle company (like oraclecompany) it's
give me desired results.

so final conclusion from my side that synonym does not support space
between word combination.

please help me to sorted out this problem.

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Rajesh_Tarle · February 27, 2013, 12:22pm

On Tuesday, February 19, 2013 7:09:14 PM UTC+5:30, raj wrote:

Hi

I am newbie with Elasticsearch.i need to use synonym filter to meet my
requirement.

my used mapping details and synonym text file on :
Analyzer Mapping · GitHub

Requirement:

I need to search both direction means*

oracle=sap and sap=oracle*
This it's works nicely but when using below one :

oracle company=sap and sap =oracle company

It doesn't work.

when i remove the space between oracle company (like oraclecompany) it's
give me desired results.

so final conclusion from my side that synonym does not support space
between word combination.

please help me to sorted out this problem.

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Rajesh_Tarle · February 27, 2013, 12:25pm

Hi ppearcy,

I have changed my Elasticsearch version to 0.20.5 it's working good and
solve my problem.
but now If I search using query "oracle company", i got both "sap" and
"oracle company" reverse also working proper.
but when I am searching using query "oracle" or "company" i got also same
result. it's wrong.
in my synonym file I have
oracle company,sap

why "sap" is match to single word "oracle" or "company"?

please suggest.

Thanks
Rajesh

On Tuesday, February 19, 2013 7:09:14 PM UTC+5:30, raj wrote:

Hi

I am newbie with Elasticsearch.i need to use synonym filter to meet my
requirement.

my used mapping details and synonym text file on :
Analyzer Mapping · GitHub

Requirement:

I need to search both direction means*

oracle=sap and sap=oracle*
This it's works nicely but when using below one :

oracle company=sap and sap =oracle company

It doesn't work.

when i remove the space between oracle company (like oraclecompany) it's
give me desired results.

so final conclusion from my side that synonym does not support space
between word combination.

please help me to sorted out this problem.

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Ivan · February 27, 2013, 3:35pm

What changes have you made to the original mapping? I noticed a few issues,
but you must have changed it after you originally posted if you got it to
work somewhat.

Your synonym filter is being applied to the analyzed tokens of the stream.
Try using the analyze API on the text using the analyzer used to index it.

--
Ivan

On Wed, Feb 27, 2013 at 4:25 AM, raj rajesh.elasticsearch@gmail.com wrote:

Hi ppearcy,

I have changed my Elasticsearch version to 0.20.5 it's working good and
solve my problem.
but now If I search using query "oracle company", i got both "sap" and
"oracle company" reverse also working proper.
but when I am searching using query "oracle" or "company" i got also same
result. it's wrong.
in my synonym file I have
oracle company,sap

why "sap" is match to single word "oracle" or "company"?

please suggest.

Thanks
Rajesh

On Tuesday, February 19, 2013 7:09:14 PM UTC+5:30, raj wrote:

Hi

I am newbie with Elasticsearch.i need to use synonym filter to meet my
requirement.

my used mapping details and synonym text file on :
https://gist.github.com/**rajesh07/4985901 https://gist.github.com/rajesh07/4985901

Requirement:

I need to search both direction means*

oracle=sap and sap=oracle*
This it's works nicely but when using below one :

oracle company=sap and sap =oracle company

It doesn't work.

when i remove the space between oracle company (like oraclecompany) it's
give me desired results.

so final conclusion from my side that synonym does not support space
between word combination.

please help me to sorted out this problem.

Thanks

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Topic		Replies	Views
Confusion in using synonym token filter in elasticsearch 6.x Elasticsearch	8	1549	May 18, 2018
Elasticsearch can't hanlde space after add analyzer Elasticsearch	3	405	April 21, 2022
Synonym Filter Elasticsearch	2	352	July 6, 2017
Synonym analyzer issue in elastic search Elasticsearch	1	770	September 9, 2019
Keyword analyzer but allow redundant white spaces Elasticsearch	3	4081	January 15, 2018

Problem with synonym token filter

Requirement:

Requirement:

Requirement:

Requirement:

Requirement:

Requirement:

Requirement:

Requirement:

Related topics