Synonyms in a query


(Alex K) #1

Hello there,

i have a query, example is this:
{
"query": {
"bool": {
"should": [
{
"multi_match": {
"query": "foo",
"fields": [
"TITLE",
"SHORTDESC"
],
"type": "phrase_prefix"
}
},
{
"multi_match": {
"query": "foo",
"cutoff_frequency": null,
"fields": [
"TITLE",
"SHORTDESC"
]
}
}
]
}
},
"filter": {
"term": {
"ACTIVE": 1
}
},
"sort": {
"TITLE": {
"order": "asc"
}
},
"size": 7
}

Now I have the question if I can use synonyms here?

I already saw that you can use a synonym-token inside an analyzer.
But I have a query here, not an analyzer.
Do I have to put an analyzer inside the query?

I don't know much about ES yet, so this may be a total stupid question.
Thank you in advance :slight_smile:

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5fe2d157-b437-4bd8-8a18-8aa4f41f63fe%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Luiz Guilherme Santos) #2

Hi Alex,

If the fields you are searching ("TITLE","SHORTDESC") were indexed using an
analyzer that declares synonyms the "multi_match" query will use it. It
happens because the multi_match query use the analyzer explicit defined in
mapping or the default analyzer.[1]

You can explicit declare the analyzer in the query if you want:

POST test/_search
{
"query": {
"multi_match": {
"query": "test",
"fields": ["title"],
"analyzer": "standard"
}
}
}

[1]
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-match-query.html#_boolean

On Mon, Apr 7, 2014 at 4:29 AM, Alex K sniper.foxde@googlemail.com wrote:

Hello there,

i have a query, example is this:
{
"query": {
"bool": {
"should": [
{
"multi_match": {
"query": "foo",
"fields": [
"TITLE",
"SHORTDESC"
],
"type": "phrase_prefix"
}
},
{
"multi_match": {
"query": "foo",
"cutoff_frequency": null,
"fields": [
"TITLE",
"SHORTDESC"
]
}
}
]
}
},
"filter": {
"term": {
"ACTIVE": 1
}
},
"sort": {
"TITLE": {
"order": "asc"
}
},
"size": 7
}

Now I have the question if I can use synonyms here?

I already saw that you can use a synonym-token inside an analyzer.
But I have a query here, not an analyzer.
Do I have to put an analyzer inside the query?

I don't know much about ES yet, so this may be a total stupid question.
Thank you in advance :slight_smile:

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/5fe2d157-b437-4bd8-8a18-8aa4f41f63fe%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/5fe2d157-b437-4bd8-8a18-8aa4f41f63fe%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Luiz Guilherme P. Santos

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAMdL%3DZHOw14zbQuROWbam7%3Dd%3DYkSH3K5DJHEzsz73kqdJ0nsUg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Alex K) #3

Hello Luiz, thank you for your reply!

As we use rivers, I was told to declare the analyzer there.
It looks like this for me:
{
"index" : {
"analysis" : {
"filter" : {
"synonym_filter" : {
"type" : "synonym",
"synonyms" : [
"foo, foo bar => core"
]
}
},
"analyzer" : {
"synonym" : {
"tokenizer" : "whitespace",
"filter" : [
"synonym_filter"
],
"type" : "custom",
}
}
}
}
}
which acctually says, for testing-purpose, 'if someone searches for 'foo'
or 'foo bar', search for 'core' '

Now my query uses the analyzer:
{
"query": {
"bool": {
"should": [
{
"multi_match": {
"query": "foo",
"fields": [
"TITLE",
"SHORTDESC"
],
"type": "phrase_prefix",
"analyzer": "synonym"
}
},
{
"multi_match": {
"query": "foo",
"cutoff_frequency": null,
"fields": [
"TITLE",
"SHORTDESC"
]
}
}
]
}
},
"filter": {
"term": {
"ACTIVE": 1
}
},
"sort": {
"TITLE": {
"order": "asc"
}
},
"size": 7
}

But I get an error there:
[...]nested: QueryParsingException[[test484] [multi_match] analyzer
[synonym] not found];[...]

What am I doing wrong here?

Am Montag, 7. April 2014 09:29:17 UTC+2 schrieb Alex K:

Hello there,

i have a query, example is this:
{
"query": {
"bool": {
"should": [
{
"multi_match": {
"query": "foo",
"fields": [
"TITLE",
"SHORTDESC"
],
"type": "phrase_prefix"
}
},
{
"multi_match": {
"query": "foo",
"cutoff_frequency": null,
"fields": [
"TITLE",
"SHORTDESC"
]
}
}
]
}
},
"filter": {
"term": {
"ACTIVE": 1
}
},
"sort": {
"TITLE": {
"order": "asc"
}
},
"size": 7
}

Now I have the question if I can use synonyms here?

I already saw that you can use a synonym-token inside an analyzer.
But I have a query here, not an analyzer.
Do I have to put an analyzer inside the query?

I don't know much about ES yet, so this may be a total stupid question.
Thank you in advance :slight_smile:

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/3ec9e97d-f210-4a88-a269-f6306bf0266c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Luiz Guilherme Santos) #4

Hi Alex,

First create your index with the synonym filter:

curl -XPUT 'localhost:9200/myindex/' -d '{
"settings": {
"index": {
"analysis": {
"filter": {
"synonym_filter": {
"type": "synonym",
"synonyms": [
"foo, foo bar => core"
]
}
},
"analyzer": {
"synonym": {
"tokenizer": "whitespace",
"filter": [
"synonym_filter"
],
"type": "custom"
}
}
}
}
}
}'

After index one document with the title equals do "core":

curl -XPOST 'localhost:9200/myindex/test/1' -d '{
"title": "core"
}'

Then search for "foo":

curl -XPOST localhost:1980/myindex/_search -d '{
"query": {
"multi_match": {
"query": "foo",
"fields": [
"title"
],
"type": "phrase_prefix",
"analyzer": "synonym"
}
}
}'

The response is the previous indexed document:
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 0.30685282,
"hits": [
{
"_index": "myindex",
"_type": "test",
"_id": "1",
"_score": 0.30685282,
"_source": {
"title": "core"
}
}
]
}
}

On Mon, Apr 7, 2014 at 11:03 AM, Alex K sniper.foxde@googlemail.com wrote:

Hello Luiz, thank you for your reply!

As we use rivers, I was told to declare the analyzer there.
It looks like this for me:
{
"index" : {
"analysis" : {
"filter" : {
"synonym_filter" : {
"type" : "synonym",
"synonyms" : [
"foo, foo bar => core"
]
}
},
"analyzer" : {
"synonym" : {
"tokenizer" : "whitespace",
"filter" : [
"synonym_filter"
],
"type" : "custom",
}
}
}
}
}
which acctually says, for testing-purpose, 'if someone searches for 'foo'
or 'foo bar', search for 'core' '

Now my query uses the analyzer:
{
"query": {
"bool": {
"should": [
{
"multi_match": {
"query": "foo",
"fields": [
"TITLE",
"SHORTDESC"
],
"type": "phrase_prefix",

                    "analyzer": "synonym"

                }
            },
            {
                "multi_match": {
                    "query": "foo",
                    "cutoff_frequency": null,
                    "fields": [
                        "TITLE",
                        "SHORTDESC"
                    ]
                }
            }
        ]
    }
},
"filter": {
    "term": {
        "ACTIVE": 1
    }
},
"sort": {
    "TITLE": {
        "order": "asc"
    }
},
"size": 7

}

But I get an error there:
[...]nested: QueryParsingException[[test484] [multi_match] analyzer
[synonym] not found];[...]

What am I doing wrong here?

Am Montag, 7. April 2014 09:29:17 UTC+2 schrieb Alex K:

Hello there,

i have a query, example is this:
{
"query": {
"bool": {
"should": [
{
"multi_match": {
"query": "foo",
"fields": [
"TITLE",
"SHORTDESC"
],
"type": "phrase_prefix"
}
},
{
"multi_match": {
"query": "foo",
"cutoff_frequency": null,
"fields": [
"TITLE",
"SHORTDESC"
]
}
}
]
}
},
"filter": {
"term": {
"ACTIVE": 1
}
},
"sort": {
"TITLE": {
"order": "asc"
}
},
"size": 7
}

Now I have the question if I can use synonyms here?

I already saw that you can use a synonym-token inside an analyzer.
But I have a query here, not an analyzer.
Do I have to put an analyzer inside the query?

I don't know much about ES yet, so this may be a total stupid question.
Thank you in advance :slight_smile:

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/3ec9e97d-f210-4a88-a269-f6306bf0266c%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/3ec9e97d-f210-4a88-a269-f6306bf0266c%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Luiz Guilherme P. Santos

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAMdL%3DZG3sZzwM7CrXC%2BQrXu93vJuiPBL%3DW5UfYkkDXhGDpCOnA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Alex K) #5

Hi Luiz,

thank you again for your reply.

I don't fully understand the part you mentioned:

After index one document with the title equals do "core":

curl -XPOST 'localhost:9200/myindex/test/1' -d '{
"title": "core"
}'

Sorry, I am pretty new to ES and haven't understand pretty much.

Now what happens there?
And what if I don't have hardcoded synonyms, but a file which someone can
fill out.
I need something like

"synonyms_path" : "analysis/synonym.txt"

in my filter, but then what about the setp you mentioned that I did not
understand?

Sorry for all the trouble

Am Montag, 7. April 2014 09:29:17 UTC+2 schrieb Alex K:

Hello there,

i have a query, example is this:
{
"query": {
"bool": {
"should": [
{
"multi_match": {
"query": "foo",
"fields": [
"TITLE",
"SHORTDESC"
],
"type": "phrase_prefix"
}
},
{
"multi_match": {
"query": "foo",
"cutoff_frequency": null,
"fields": [
"TITLE",
"SHORTDESC"
]
}
}
]
}
},
"filter": {
"term": {
"ACTIVE": 1
}
},
"sort": {
"TITLE": {
"order": "asc"
}
},
"size": 7
}

Now I have the question if I can use synonyms here?

I already saw that you can use a synonym-token inside an analyzer.
But I have a query here, not an analyzer.
Do I have to put an analyzer inside the query?

I don't know much about ES yet, so this may be a total stupid question.
Thank you in advance :slight_smile:

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/637c5a0a-89cc-47d3-9f59-785ddf6ccfc3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Alex K) #6

Hi Luiz,

thank you again for your reply.

A colleague of mine told me that I might miss a plugin to use my
settings-file.
I will check this out and later write down here what I found out.

Sorry for all the trouble

Am Montag, 7. April 2014 09:29:17 UTC+2 schrieb Alex K:

Hello there,

i have a query, example is this:
{
"query": {
"bool": {
"should": [
{
"multi_match": {
"query": "foo",
"fields": [
"TITLE",
"SHORTDESC"
],
"type": "phrase_prefix"
}
},
{
"multi_match": {
"query": "foo",
"cutoff_frequency": null,
"fields": [
"TITLE",
"SHORTDESC"
]
}
}
]
}
},
"filter": {
"term": {
"ACTIVE": 1
}
},
"sort": {
"TITLE": {
"order": "asc"
}
},
"size": 7
}

Now I have the question if I can use synonyms here?

I already saw that you can use a synonym-token inside an analyzer.
But I have a query here, not an analyzer.
Do I have to put an analyzer inside the query?

I don't know much about ES yet, so this may be a total stupid question.
Thank you in advance :slight_smile:

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ba838067-1277-4db9-a8f9-e306d47d6591%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Alex K) #7

Me again,

seems it was a local problem for me.
The way Luiz mentioned is the exact correct way.
Thank you very much, Luiz!!!!
You helped me really out of this!

Am Montag, 7. April 2014 09:29:17 UTC+2 schrieb Alex K:

Hello there,

i have a query, example is this:
{
"query": {
"bool": {
"should": [
{
"multi_match": {
"query": "foo",
"fields": [
"TITLE",
"SHORTDESC"
],
"type": "phrase_prefix"
}
},
{
"multi_match": {
"query": "foo",
"cutoff_frequency": null,
"fields": [
"TITLE",
"SHORTDESC"
]
}
}
]
}
},
"filter": {
"term": {
"ACTIVE": 1
}
},
"sort": {
"TITLE": {
"order": "asc"
}
},
"size": 7
}

Now I have the question if I can use synonyms here?

I already saw that you can use a synonym-token inside an analyzer.
But I have a query here, not an analyzer.
Do I have to put an analyzer inside the query?

I don't know much about ES yet, so this may be a total stupid question.
Thank you in advance :slight_smile:

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/7a1878b4-6c79-42a3-b0d7-5562d0cbdece%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(system) #8