Edge-Ngram returns irrelevant result

cyrilforce · April 28, 2014, 8:50am

Hi All,

I am trying to use *edge-ngram filter *to achieve auto-completion
behaviour. However when i run the following query :

{
"from" : "0",
"size" : "20",
"explain" : true,

"query" : {
*"multi_match": {*

```
 "query": "happy",*
```
```
 "fields": [ "DISPLAY_NAME.NAME"]*
```
```
}*
```
},
"sort" : [{
"_score" : { "order" : "desc"}}]
}

It returns me documents :

gist.github.com

https://gist.github.com/cheehoo/11365794

irrevelent_response_edge_ngram.json

      "_shard": 1,
                "_node": "Db5OO2IRQqKtr9dTLnM6qQ",
                "_index": "jdbc_dev",
                "_type": "media",
                "_id": "yPz6U2iWSPCKCLVQpvd_Zw",
                "_score": 1.5775237,
                "_source": {
                    "DISPLAY_NAME": "Happier?",
                    "PRICE": 4,
                    "AUDIO": "http://musicube.thecube.my/base/preview/088/889023/889023-MP3-TT-P.mp3",

This file has been truncated. show original

May i know how can i achieve the behaviour when i search "happy" then is
should return me following result instead :

1)Happy
2)Happy birthday
3)Happy valentine

The analzyer configure :

"analysis" : {
"analyzer" : {
"edge_ngram_keyword_lowercase_analyzer" : {
"tokenizer" : "keyword",
"filter" : ["my_edge_ngram_filter","lowercase"]
}
},
"filter" : {
"my_edge_ngram_filter" : {
"type" : "edgeNGram",
"min_gram" : "2",
"max_gram" : "30"

                }
            }

}

The mapping :

gist.github.com

https://gist.github.com/cheehoo/11365917

irrevelent_response_ngram_edge_mapping.json

{
    "media": {
        "properties": {
            "AUDIO": {
                "type": "string"
            },
            "BILLINGTYPE_ID": {
                "type": "long"
            },
            "CATMEDIA_CDATE": {

This file has been truncated. show original

Thanks.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e8a4446c-38c9-4c6b-8d2a-15374ef041a9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

jpountz · April 28, 2014, 9:23am

Hi,

I think your issue is that you are also applying the edge n-gram filter at
search time. So "happy" is translated to ("ha" OR "hap" OR "happ" OR
"happy"). You need to configure your mappings with a search analyzer[1]
which is the same analyzer but without the edge-ngram filter (so just
keyword and lowercase).

On a side note, you would benefit from putting the lowercase filter in
front of your n-gram filter. This way, the lowercasing would be done only
once.

[1]

On Mon, Apr 28, 2014 at 10:50 AM, cyrilforce cheehoo84@gmail.com wrote:

Hi All,

I am trying to use *edge-ngram filter *to achieve auto-completion
behaviour. However when i run the following query :

{
"from" : "0",
"size" : "20",
"explain" : true,
"query" : {
*"multi_match": {*
 "query": "happy",*
 "fields": [ "DISPLAY_NAME.NAME <http://DISPLAY_NAME.NAME>"]*
}*
},
"sort" : [{
"_score" : { "order" : "desc"}}]
}
It returns me documents :

irrevelent_response_edge_ngram.json · GitHub

May i know how can i achieve the behaviour when i search "happy" then
is should return me following result instead :

1)Happy
2)Happy birthday
3)Happy valentine

The analzyer configure :

"analysis" : {
"analyzer" : {
"edge_ngram_keyword_lowercase_analyzer" : {
"tokenizer" : "keyword",
"filter" : ["my_edge_ngram_filter","lowercase"]
}
},
"filter" : {
"my_edge_ngram_filter" : {
"type" : "edgeNGram",
"min_gram" : "2",
"max_gram" : "30"
                }
            }
}

The mapping :
irrevelent_response_ngram_edge_mapping.json · GitHub

Thanks.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/e8a4446c-38c9-4c6b-8d2a-15374ef041a9%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/e8a4446c-38c9-4c6b-8d2a-15374ef041a9%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Adrien Grand

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j5iMm-e%3Dk-vf6_Hw%3D7EDgNzoG%2BrQLfAW6-pJdw1JbJYYQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

cyrilforce · April 28, 2014, 10:29am

Hi Adrien,

But now when i type "hap" it doesn't return the following after i
configure with search_analzyer :

1)Happy
2)Happy birthday
3)Happy valentine

Sorry maybe i have missed the requirements at the first place. Basically i
wanted to use the edge_ngram to achieve auto-completion not full-text
matching.

Thanks.

On Mon, Apr 28, 2014 at 5:23 PM, Adrien Grand <
adrien.grand@elasticsearch.com> wrote:

Hi,

I think your issue is that you are also applying the edge n-gram filter at
search time. So "happy" is translated to ("ha" OR "hap" OR "happ" OR
"happy"). You need to configure your mappings with a search analyzer[1]
which is the same analyzer but without the edge-ngram filter (so just
keyword and lowercase).

On a side note, you would benefit from putting the lowercase filter in
front of your n-gram filter. This way, the lowercasing would be done only
once.

[1]
Elasticsearch Platform — Find real-time answers at scale | Elastic

On Mon, Apr 28, 2014 at 10:50 AM, cyrilforce cheehoo84@gmail.com wrote:
Hi All,

I am trying to use *edge-ngram filter *to achieve auto-completion
behaviour. However when i run the following query :

{
"from" : "0",
"size" : "20",
"explain" : true,
"query" : {
*"multi_match": {*
 "query": "happy",*
 "fields": [ "DISPLAY_NAME.NAME <http://DISPLAY_NAME.NAME>"]*
}*
},
"sort" : [{
"_score" : { "order" : "desc"}}]
}
It returns me documents :

irrevelent_response_edge_ngram.json · GitHub

May i know how can i achieve the behaviour when i search "happy" then
is should return me following result instead :

1)Happy
2)Happy birthday
3)Happy valentine

The analzyer configure :

"analysis" : {
"analyzer" : {
"edge_ngram_keyword_lowercase_analyzer" : {
"tokenizer" : "keyword",
"filter" : ["my_edge_ngram_filter","lowercase"]
}
},
"filter" : {
"my_edge_ngram_filter" : {
"type" : "edgeNGram",
"min_gram" : "2",
"max_gram" : "30"
                }
            }
}

The mapping :
irrevelent_response_ngram_edge_mapping.json · GitHub

Thanks.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/e8a4446c-38c9-4c6b-8d2a-15374ef041a9%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/e8a4446c-38c9-4c6b-8d2a-15374ef041a9%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.
--
Adrien Grand

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/fhsCgFBF76E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j5iMm-e%3Dk-vf6_Hw%3D7EDgNzoG%2BrQLfAW6-pJdw1JbJYYQ%40mail.gmail.com https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j5iMm-e%3Dk-vf6_Hw%3D7EDgNzoG%2BrQLfAW6-pJdw1JbJYYQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Regards,

Chee Hoo

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGS0%2Bg-6noA3oFDQiGMQD1BdfxEK2w5rFpdXB9-iaeRie4ak%3DA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

dantuff · April 28, 2014, 11:36am

To return the query results in the order you want, execute the sort on a
not_analyzed version of the term, i.e

"DISPLAY_NAME": {
"type": "string",
"fields": {
"NAME": {
"type": "string",
"analyzer": "edge_ngram_keyword_lowercase_analyzer"
},
"untouched":{
"type": "string",
"analyzer":"not_analyzed"
}
}
}

Reindex the data. Now in your query you can sort on the 'untouched' field:

{
"from" : "0",
"size" : "20",
"explain" : true,

"query" : {
"multi_match": {
     "query": "happy",
      "fields": [ "DISPLAY_NAME.name"]
 }
},
"sort" : [{ 
  "DISPLAY_NAME.untouched" : { "order" : "desc"}}]

}

which will return

1)Happy
2)Happy birthday
3)Happy valentine

Dan

On Monday, April 28, 2014 9:50:44 AM UTC+1, cyrilforce wrote:

Hi All,

I am trying to use *edge-ngram filter *to achieve auto-completion
behaviour. However when i run the following query :

{
"from" : "0",
"size" : "20",
"explain" : true,
"query" : {
*"multi_match": {*
 "query": "happy",*
 "fields": [ "DISPLAY_NAME.NAME <http://DISPLAY_NAME.NAME>"]*
}*
},
"sort" : [{
"_score" : { "order" : "desc"}}]
}
It returns me documents :

irrevelent_response_edge_ngram.json · GitHub

May i know how can i achieve the behaviour when i search "happy" then
is should return me following result instead :

1)Happy
2)Happy birthday
3)Happy valentine

The analzyer configure :

"analysis" : {
"analyzer" : {
"edge_ngram_keyword_lowercase_analyzer" : {
"tokenizer" : "keyword",
"filter" : ["my_edge_ngram_filter","lowercase"]
}
},
"filter" : {
"my_edge_ngram_filter" : {
"type" : "edgeNGram",
"min_gram" : "2",
"max_gram" : "30"
                }
            }
}

The mapping :
irrevelent_response_ngram_edge_mapping.json · GitHub

Thanks.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/7ad16d4e-aed3-474e-893d-92cc788003b0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

cyrilforce · April 28, 2014, 11:41am

Hi dan

I tried this before however it returns me some irrelevant result as well :

gist.github.com

https://gist.github.com/cheehoo/11365794

irrevelent_response_edge_ngram.json

      "_shard": 1,
                "_node": "Db5OO2IRQqKtr9dTLnM6qQ",
                "_index": "jdbc_dev",
                "_type": "media",
                "_id": "yPz6U2iWSPCKCLVQpvd_Zw",
                "_score": 1.5775237,
                "_source": {
                    "DISPLAY_NAME": "Happier?",
                    "PRICE": 4,
                    "AUDIO": "http://musicube.thecube.my/base/preview/088/889023/889023-MP3-TT-P.mp3",

This file has been truncated. show original

How I could get only the relevant result with the sorting via edge ngram ?
Thanks
On Apr 28, 2014 7:36 PM, "Dan Tuffery" dan.tuffery@gmail.com wrote:

To return the query results in the order you want, execute the sort on a
not_analyzed version of the term, i.e

"DISPLAY_NAME": {
"type": "string",
"fields": {
"NAME": {
"type": "string",
"analyzer": "edge_ngram_keyword_lowercase_analyzer"
},
"untouched":{
"type": "string",
"analyzer":"not_analyzed"
}
}
}

Reindex the data. Now in your query you can sort on the 'untouched' field:

{
"from" : "0",
"size" : "20",
"explain" : true,
"query" : {
"multi_match": {
     "query": "happy",
      "fields": [ "DISPLAY_NAME.name"]
 }
},
"sort" : [{
  "DISPLAY_NAME.untouched" : { "order" : "desc"}}]
}

which will return

1)Happy
2)Happy birthday
3)Happy valentine

Dan

On Monday, April 28, 2014 9:50:44 AM UTC+1, cyrilforce wrote:
Hi All,

I am trying to use *edge-ngram filter *to achieve auto-completion
behaviour. However when i run the following query :

{
"from" : "0",
"size" : "20",
"explain" : true,
"query" : {
*"multi_match": {*
 "query": "happy",*
 "fields": [ "DISPLAY_NAME.NAME <http://DISPLAY_NAME.NAME>"]*
}*
},
"sort" : [{
"_score" : { "order" : "desc"}}]
}
It returns me documents :

irrevelent_response_edge_ngram.json · GitHub

May i know how can i achieve the behaviour when i search "happy" then
is should return me following result instead :

1)Happy
2)Happy birthday
3)Happy valentine

The analzyer configure :

"analysis" : {
"analyzer" : {
"edge_ngram_keyword_lowercase_analyzer" : {
"tokenizer" : "keyword",
"filter" : ["my_edge_ngram_filter","lowercase"]
}
},
"filter" : {
"my_edge_ngram_filter" : {
"type" : "edgeNGram",
"min_gram" : "2",
"max_gram" : "30"
                }
            }
}

The mapping :
irrevelent_response_ngram_edge_mapping.json · GitHub

Thanks.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/fhsCgFBF76E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/7ad16d4e-aed3-474e-893d-92cc788003b0%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/7ad16d4e-aed3-474e-893d-92cc788003b0%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGS0%2Bg-yN%3DBa5%2BUgCcSwz4hVeAbo8O0c2BV1Wod1zvsKb_QzKg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

dantuff · April 28, 2014, 12:17pm

As Adrien said don't apply the edge n-gram filter at search time. When you
tried it what does you mapping look like, because "hap" should return a
result.

Dan

On Monday, April 28, 2014 12:41:38 PM UTC+1, cyrilforce wrote:

Hi dan

I tried this before however it returns me some irrelevant result as well :
irrevelent_response_edge_ngram.json · GitHub

How I could get only the relevant result with the sorting via edge ngram ?
Thanks
On Apr 28, 2014 7:36 PM, "Dan Tuffery" <dan.t...@gmail.com <javascript:>>
wrote:
To return the query results in the order you want, execute the sort on a
not_analyzed version of the term, i.e

"DISPLAY_NAME": {
"type": "string",
"fields": {
"NAME": {
"type": "string",
"analyzer": "edge_ngram_keyword_lowercase_analyzer"
},
"untouched":{
"type": "string",
"analyzer":"not_analyzed"
}
}
}

Reindex the data. Now in your query you can sort on the 'untouched' field:

{
"from" : "0",
"size" : "20",
"explain" : true,
"query" : {
"multi_match": {
     "query": "happy",
      "fields": [ "DISPLAY_NAME.name"]
 }
},
"sort" : [{ 
  "DISPLAY_NAME.untouched" : { "order" : "desc"}}] 
}

which will return

1)Happy
2)Happy birthday
3)Happy valentine

Dan

On Monday, April 28, 2014 9:50:44 AM UTC+1, cyrilforce wrote:
Hi All,

I am trying to use *edge-ngram filter *to achieve auto-completion
behaviour. However when i run the following query :

{
"from" : "0",
"size" : "20",
"explain" : true,
"query" : {
*"multi_match": {*
 "query": "happy",*
 "fields": [ "DISPLAY_NAME.NAME <http://DISPLAY_NAME.NAME>"]*
}*
},
"sort" : [{
"_score" : { "order" : "desc"}}]
}
It returns me documents :

irrevelent_response_edge_ngram.json · GitHub

May i know how can i achieve the behaviour when i search "happy" then
is should return me following result instead :

1)Happy
2)Happy birthday
3)Happy valentine

The analzyer configure :

"analysis" : {
"analyzer" : {
"edge_ngram_keyword_lowercase_analyzer" : {
"tokenizer" : "keyword",
"filter" : ["my_edge_ngram_filter","lowercase"]
}
},
"filter" : {
"my_edge_ngram_filter" : {
"type" : "edgeNGram",
"min_gram" : "2",
"max_gram" : "30"
                }
            }
}

The mapping :
irrevelent_response_ngram_edge_mapping.json · GitHub

Thanks.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/fhsCgFBF76E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/7ad16d4e-aed3-474e-893d-92cc788003b0%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/7ad16d4e-aed3-474e-893d-92cc788003b0%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f7b7df20-4136-4cb6-9dcc-318502ac05f9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

cyrilforce · April 28, 2014, 1:31pm

Hi Dan,

Not sure whether i have same understanding "edge n-gram filter at search
time" refer to apply edge n-gram filter in the query ? If this is the case
i think i didn't apply any analyzer in the query :

*Here my query : *

{
"query" : {
"multi_match": {
"query": "happy",
"fields": [ "DISPLAY_NAME.AUTOCOMPLETE" ]
}
},
"sort" : [{
"_score" : { "order" : "desc"},
"DISPLAY_NAME.NAME" : { "order" : "asc" }}]
}

*The result : *

gist.github.com

https://gist.github.com/cheehoo/11371837

edge_ngram_original.json

{
    "took": 22,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "failed": 0
    },
    "hits": {
        "total": 8,

This file has been truncated. show original

As you notice in the result it returned :

Happenings
Happier?

which aren't part of what i searched for.

*The mapping : *

gist.github.com

https://gist.github.com/cheehoo/11371897

edge_ngram_mapping_original.json

{
    "media": {
        "properties": {
            "AUDIO": {
                "type": "string"
            },
            "BILLINGTYPE_ID": {
                "type": "long"
            },
            "CATMEDIA_CDATE": {

This file has been truncated. show original

*The index : *
{
"jdbc_dev": {
"settings": {
"index.number_of_shards": "5",
"index.number_of_replicas": "1",
"index.version.created": "900299",
"index.analysis.filter.my_edge_ngram_filter.type": "edgeNGram",

"index.analysis.analyzer.edge_ngram_keyword_lowercase_analyzer.filter.1":
"lowercase",

"index.analysis.analyzer.edge_ngram_keyword_lowercase_analyzer.filter.0":
"my_edge_ngram_filter",

"index.analysis.analyzer.edge_ngram_keyword_lowercase_analyzer.tokenizer":
"keyword",
"index.analysis.filter.my_edge_ngram_filter.min_gram": "2",
"index.analysis.filter.my_edge_ngram_filter.max_gram": "30"
}
}
}

After that with Adrien suggestion i tried create the analyzer

"analysis" : {
"analyzer" : {
"keyword_lowercase_analyzer" : {
"tokenizer" : "keyword",
"filter" : ["lowercase"]
}
}
}

and recreate the mapping with :

"DISPLAY_NAME": {
"type": "string",
"search_analyzer" : "keyword_lowercase_analyzer"

}

however with this configuration it wasn't able to search via "hap" to
return the following result :

1)Happy
2)Happy birthday
3)Happy valentine

Thanks

On Mon, Apr 28, 2014 at 8:17 PM, Dan Tuffery dan.tuffery@gmail.com wrote:

As Adrien said don't apply the edge n-gram filter at search time. When you
tried it what does you mapping look like, because "hap" should return a
result.

Dan

On Monday, April 28, 2014 12:41:38 PM UTC+1, cyrilforce wrote:
Hi dan

I tried this before however it returns me some irrelevant result as well :
irrevelent_response_edge_ngram.json · GitHub

How I could get only the relevant result with the sorting via edge ngram
? Thanks
On Apr 28, 2014 7:36 PM, "Dan Tuffery" dan.t...@gmail.com wrote:
To return the query results in the order you want, execute the sort on a
not_analyzed version of the term, i.e

"DISPLAY_NAME": {
"type": "string",
"fields": {
"NAME": {
"type": "string",
"analyzer": "edge_ngram_keyword_lowercase_analyzer"
},
"untouched":{
"type": "string",
"analyzer":"not_analyzed"
}
}
}

Reindex the data. Now in your query you can sort on the 'untouched'
field:

{
"from" : "0",
"size" : "20",
"explain" : true,
"query" : {
"multi_match": {
     "query": "happy",
      "fields": [ "DISPLAY_NAME.name"]
 }
},
"sort" : [{
  "DISPLAY_NAME.untouched" : { "order" : "desc"}}]
}

which will return

1)Happy
2)Happy birthday
3)Happy valentine

Dan

On Monday, April 28, 2014 9:50:44 AM UTC+1, cyrilforce wrote:
Hi All,

I am trying to use *edge-ngram filter *to achieve auto-completion
behaviour. However when i run the following query :

{
"from" : "0",
"size" : "20",
"explain" : true,
"query" : {
*"multi_match": {*
 "query": "happy",*
 "fields": [ "DISPLAY_NAME.NAME <http://DISPLAY_NAME.NAME>"]*
}*
},
"sort" : [{
"_score" : { "order" : "desc"}}]
}
It returns me documents :

irrevelent_response_edge_ngram.json · GitHub

May i know how can i achieve the behaviour when i search "happy"
then is should return me following result instead :

1)Happy
2)Happy birthday
3)Happy valentine

The analzyer configure :

"analysis" : {
"analyzer" : {
"edge_ngram_keyword_lowercase_analyzer" : {
"tokenizer" : "keyword",
"filter" : ["my_edge_ngram_filter","lowercase"]
}
},
"filter" : {
"my_edge_ngram_filter" : {
"type" : "edgeNGram",
"min_gram" : "2",
"max_gram" : "30"
                }
            }
}

The mapping :
irrevelent_response_ngram_edge_mapping.json · GitHub

Thanks.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/
topic/elasticsearch/fhsCgFBF76E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/7ad16d4e-aed3-474e-893d-92cc788003b0%
40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/7ad16d4e-aed3-474e-893d-92cc788003b0%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/fhsCgFBF76E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/f7b7df20-4136-4cb6-9dcc-318502ac05f9%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/f7b7df20-4136-4cb6-9dcc-318502ac05f9%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Regards,

Chee Hoo

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGS0%2Bg95SAL--gSTfUm6gJRYq2pEeDgQiVqH%3DcoV%2BffLSn56tg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

dantuff · April 28, 2014, 1:43pm

Change your mapping to:

DISPLAY_NAME": {
"type": "multi_field",
"fields": {
"AUTOCOMPLETE": {
"type": "string",
"index_analyzer":
"edge_ngram_keyword_lowercase_analyzer",
"include_in_all": false
},
"NAME": {
"type": "string",
"analyzer": "standard",
"include_in_all": false
}
}
}

the DISPLAY_NAME term will be indexed using
'edge_ngram_keyword_lowercase_analyzer'. When you search on DISPLAY_NAME

On Monday, April 28, 2014 2:31:24 PM UTC+1, cyrilforce wrote:

Hi Dan,

Not sure whether i have same understanding "edge n-gram filter at search
time" refer to apply edge n-gram filter in the query ? If this is the
case i think i didn't apply any analyzer in the query :

*Here my query : *

{
"query" : {
"multi_match": {
"query": "happy",
"fields": [ "DISPLAY_NAME.AUTOCOMPLETE" ]
}
},
"sort" : [{
"_score" : { "order" : "desc"},
"DISPLAY_NAME.NAME" : { "order" : "asc" }}]
}

*The result : *
edge_ngram_original.json · GitHub

As you notice in the result it returned :

Happenings
Happier?

which aren't part of what i searched for.

*The mapping : *
edge_ngram_mapping_original.json · GitHub

*The index : *
{
"jdbc_dev": {
"settings": {
"index.number_of_shards": "5",
"index.number_of_replicas": "1",
"index.version.created": "900299",
"index.analysis.filter.my_edge_ngram_filter.type": "edgeNGram",

"index.analysis.analyzer.edge_ngram_keyword_lowercase_analyzer.filter.1":
"lowercase",

"index.analysis.analyzer.edge_ngram_keyword_lowercase_analyzer.filter.0":
"my_edge_ngram_filter",

"index.analysis.analyzer.edge_ngram_keyword_lowercase_analyzer.tokenizer":
"keyword",
"index.analysis.filter.my_edge_ngram_filter.min_gram": "2",
"index.analysis.filter.my_edge_ngram_filter.max_gram": "30"
}
}
}

After that with Adrien suggestion i tried create the analyzer

"analysis" : {
"analyzer" : {
"keyword_lowercase_analyzer" : {
"tokenizer" : "keyword",
"filter" : ["lowercase"]
}
}
}

and recreate the mapping with :

"DISPLAY_NAME": {
            "type": "string",
            "search_analyzer" : "keyword_lowercase_analyzer"
}

however with this configuration it wasn't able to search via "hap" to
return the following result :

1)Happy
2)Happy birthday
3)Happy valentine

Thanks

On Mon, Apr 28, 2014 at 8:17 PM, Dan Tuffery <dan.t...@gmail.com<javascript:>

wrote:
As Adrien said don't apply the edge n-gram filter at search time. When
you tried it what does you mapping look like, because "hap" should return a
result.

Dan

On Monday, April 28, 2014 12:41:38 PM UTC+1, cyrilforce wrote:
Hi dan

I tried this before however it returns me some irrelevant result as well
:
irrevelent_response_edge_ngram.json · GitHub

How I could get only the relevant result with the sorting via edge ngram
? Thanks
On Apr 28, 2014 7:36 PM, "Dan Tuffery" dan.t...@gmail.com wrote:
To return the query results in the order you want, execute the sort
on a not_analyzed version of the term, i.e

"DISPLAY_NAME": {
"type": "string",
"fields": {
"NAME": {
"type": "string",
"analyzer": "edge_ngram_keyword_lowercase_analyzer"
},
"untouched":{
"type": "string",
"analyzer":"not_analyzed"
}
}
}

Reindex the data. Now in your query you can sort on the 'untouched'
field:

{
"from" : "0",
"size" : "20",
"explain" : true,
"query" : {
"multi_match": {
     "query": "happy",
      "fields": [ "DISPLAY_NAME.name"]
 }
},
"sort" : [{ 
  "DISPLAY_NAME.untouched" : { "order" : "desc"}}] 
}

which will return

1)Happy
2)Happy birthday
3)Happy valentine

Dan

On Monday, April 28, 2014 9:50:44 AM UTC+1, cyrilforce wrote:
Hi All,

I am trying to use *edge-ngram filter *to achieve auto-completion
behaviour. However when i run the following query :

{
"from" : "0",
"size" : "20",
"explain" : true,
"query" : {
*"multi_match": {*
 "query": "happy",*
 "fields": [ "DISPLAY_NAME.NAME <http://DISPLAY_NAME.NAME>"]*
}*
},
"sort" : [{
"_score" : { "order" : "desc"}}]
}
It returns me documents :

irrevelent_response_edge_ngram.json · GitHub

May i know how can i achieve the behaviour when i search "happy"
then is should return me following result instead :

1)Happy
2)Happy birthday
3)Happy valentine

The analzyer configure :

"analysis" : {
"analyzer" : {
"edge_ngram_keyword_lowercase_analyzer" : {
"tokenizer" : "keyword",
"filter" : ["my_edge_ngram_filter","lowercase"]
}
},
"filter" : {
"my_edge_ngram_filter" : {
"type" : "edgeNGram",
"min_gram" : "2",
"max_gram" : "30"
                }
            }
}

The mapping :
irrevelent_response_ngram_edge_mapping.json · GitHub

Thanks.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/
topic/elasticsearch/fhsCgFBF76E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/7ad16d4e-aed3-474e-893d-92cc788003b0%
40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/7ad16d4e-aed3-474e-893d-92cc788003b0%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/fhsCgFBF76E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/f7b7df20-4136-4cb6-9dcc-318502ac05f9%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/f7b7df20-4136-4cb6-9dcc-318502ac05f9%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.
--
Regards,

Chee Hoo

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f296c0d1-0248-4d62-aba4-057373596343%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

cyrilforce · April 28, 2014, 2:08pm

Hi Dan,

It worked ! Btw one more questions. Now the result returned me the same
scores for all :

1)"_score": 1.6931472,
"_source": {
"DISPLAY_NAME": "Happy Tears",

"_score": 1.6931472,
"_source": {
"DISPLAY_NAME": "Happy Happy Sangeethalu",

3)"_score": 1.6931472,
"_source": {
"DISPLAY_NAME": "Happy",

Anyway to manipulate the 3) to appear first ?

*The query : *

{
"explain" : true,
"query" : {
"multi_match": {
"query": "happy",
"fields": [ "DISPLAY_NAME.AUTOCOMPLETE" ]
}
},
"sort" : [{
"_score" : { "order" : "desc"},
"DISPLAY_NAME.NAME" : { "order" : "asc" }}]
}

Thanks.

On Mon, Apr 28, 2014 at 9:43 PM, Dan Tuffery dan.tuffery@gmail.com wrote:

Change your mapping to:

DISPLAY_NAME": {
"type": "multi_field",
"fields": {
"AUTOCOMPLETE": {
"type": "string",
"index_analyzer":
"edge_ngram_keyword_lowercase_analyzer",
"include_in_all": false
},
"NAME": {
"type": "string",
"analyzer": "standard",
"include_in_all": false
}
}
}

the DISPLAY_NAME term will be indexed using
'edge_ngram_keyword_lowercase_analyzer'. When you search on DISPLAY_NAME

On Monday, April 28, 2014 2:31:24 PM UTC+1, cyrilforce wrote:
Hi Dan,

Not sure whether i have same understanding "edge n-gram filter at
search time" refer to apply edge n-gram filter in the query ? If this
is the case i think i didn't apply any analyzer in the query :

*Here my query : *

{
"query" : {
"multi_match": {
"query": "happy",
"fields": [ "DISPLAY_NAME.AUTOCOMPLETE" ]
}
},
"sort" : [{
"_score" : { "order" : "desc"},
"DISPLAY_NAME.NAME" : { "order" : "asc" }}]
}

*The result : *
edge_ngram_original.json · GitHub

As you notice in the result it returned :

Happenings
Happier?

which aren't part of what i searched for.

*The mapping : *
edge_ngram_mapping_original.json · GitHub

*The index : *
{
"jdbc_dev": {
"settings": {
"index.number_of_shards": "5",
"index.number_of_replicas": "1",
"index.version.created": "900299",
"index.analysis.filter.my_edge_ngram_filter.type":
"edgeNGram",
"index.analysis.analyzer.edge_ngram_keyword_lowercase_analyzer.filter.1":
"lowercase",
"index.analysis.analyzer.edge_ngram_keyword_lowercase_analyzer.filter.0":
"my_edge_ngram_filter",
"index.analysis.analyzer.edge_ngram_keyword_lowercase_analyzer.tokenizer":
"keyword",
"index.analysis.filter.my_edge_ngram_filter.min_gram": "2",
"index.analysis.filter.my_edge_ngram_filter.max_gram": "30"
}
}
}

After that with Adrien suggestion i tried create the analyzer

"analysis" : {
"analyzer" : {
"keyword_lowercase_analyzer" : {
"tokenizer" : "keyword",
"filter" : ["lowercase"]
}
}
}

and recreate the mapping with :

"DISPLAY_NAME": {
            "type": "string",
            "search_analyzer" : "keyword_lowercase_analyzer"
}

however with this configuration it wasn't able to search via "hap" to
return the following result :

1)Happy
2)Happy birthday
3)Happy valentine

Thanks

On Mon, Apr 28, 2014 at 8:17 PM, Dan Tuffery dan.t...@gmail.com wrote:
As Adrien said don't apply the edge n-gram filter at search time. When
you tried it what does you mapping look like, because "hap" should return a
result.

Dan

On Monday, April 28, 2014 12:41:38 PM UTC+1, cyrilforce wrote:
Hi dan

I tried this before however it returns me some irrelevant result as
well :
irrevelent_response_edge_ngram.json · GitHub

How I could get only the relevant result with the sorting via edge
ngram ? Thanks
On Apr 28, 2014 7:36 PM, "Dan Tuffery" dan.t...@gmail.com wrote:
To return the query results in the order you want, execute the sort
on a not_analyzed version of the term, i.e

"DISPLAY_NAME": {
"type": "string",
"fields": {
"NAME": {
"type": "string",
"analyzer": "edge_ngram_keyword_lowercase_analyzer"
},
"untouched":{
"type": "string",
"analyzer":"not_analyzed"
}
}
}

Reindex the data. Now in your query you can sort on the 'untouched'
field:

{
"from" : "0",
"size" : "20",
"explain" : true,
"query" : {
"multi_match": {
     "query": "happy",
      "fields": [ "DISPLAY_NAME.name"]
 }
},
"sort" : [{
  "DISPLAY_NAME.untouched" : { "order" : "desc"}}]
}

which will return

1)Happy
2)Happy birthday
3)Happy valentine

Dan

On Monday, April 28, 2014 9:50:44 AM UTC+1, cyrilforce wrote:
Hi All,

I am trying to use *edge-ngram filter *to achieve auto-completion
behaviour. However when i run the following query :

{
"from" : "0",
"size" : "20",
"explain" : true,
"query" : {
*"multi_match": {*
 "query": "happy",*
 "fields": [ "DISPLAY_NAME.NAME <http://DISPLAY_NAME.NAME>"]*
}*
},
"sort" : [{
"_score" : { "order" : "desc"}}]
}
It returns me documents :

irrevelent_response_edge_ngram.json · GitHub

May i know how can i achieve the behaviour when i search "happy"
then is should return me following result instead :

1)Happy
2)Happy birthday
3)Happy valentine

The analzyer configure :

"analysis" : {
"analyzer" : {
"edge_ngram_keyword_lowercase_analyzer" : {
"tokenizer" : "keyword",
"filter" : ["my_edge_ngram_filter","lowercase"]
}
},
"filter" : {
"my_edge_ngram_filter" : {
"type" : "edgeNGram",
"min_gram" : "2",
"max_gram" : "30"
                }
            }
}

The mapping :
irrevelent_response_ngram_edge_mapping.json · GitHub

Thanks.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/to
pic/elasticsearch/fhsCgFBF76E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/7ad16d4e-aed3-474e-893d-92cc788003b0%40goo
glegroups.com https://groups.google.com/d/msgid/elasticsearch/7ad16d4e-aed3-474e-893d-92cc788003b0%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/
topic/elasticsearch/fhsCgFBF76E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/f7b7df20-4136-4cb6-9dcc-318502ac05f9%
40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/f7b7df20-4136-4cb6-9dcc-318502ac05f9%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.
--
Regards,

Chee Hoo
--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/fhsCgFBF76E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/f296c0d1-0248-4d62-aba4-057373596343%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/f296c0d1-0248-4d62-aba4-057373596343%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Regards,

Chee Hoo

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGS0%2Bg-Hc1PSLqQpd%2Bs2d9hzQ0b2MREUtXWoFh_-%3DqjtqNYmOA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

dantuff · April 28, 2014, 6:03pm

You could use a function score query and use the field length as the score:

{
"from": "0",
"size": "20",
"explain": true,
"query": {
"function_score": {
"query": {
"multi_match": {
"query": "happy",
"fields": [
"DISPLAY_NAME.name"
]
}
},
"script_score": {
"script": "_score =
pow(doc['DISPLAY_NAME.untouched'].value.length(), -1)"
}
}
}
}

Dan

On Monday, April 28, 2014 3:08:24 PM UTC+1, cyrilforce wrote:

Hi Dan,

It worked ! Btw one more questions. Now the result returned me the same
scores for all :

1)"_score": 1.6931472,
"_source": {
"DISPLAY_NAME": "Happy Tears",

"_score": 1.6931472,
"_source": {
"DISPLAY_NAME": "Happy Happy Sangeethalu",

3)"_score": 1.6931472,
"_source": {
"DISPLAY_NAME": "Happy",

Anyway to manipulate the 3) to appear first ?

*The query : *

{
"explain" : true,
"query" : {
"multi_match": {
"query": "happy",
"fields": [ "DISPLAY_NAME.AUTOCOMPLETE" ]
}
},
"sort" : [{
"_score" : { "order" : "desc"},
"DISPLAY_NAME.NAME" : { "order" : "asc" }}]
}

Thanks.

On Mon, Apr 28, 2014 at 9:43 PM, Dan Tuffery <dan.t...@gmail.com<javascript:>

wrote:
Change your mapping to:

DISPLAY_NAME": {
"type": "multi_field",
"fields": {
"AUTOCOMPLETE": {
"type": "string",
"index_analyzer":
"edge_ngram_keyword_lowercase_analyzer",
"include_in_all": false
},
"NAME": {
"type": "string",
"analyzer": "standard",
"include_in_all": false
}
}
}

the DISPLAY_NAME term will be indexed using
'edge_ngram_keyword_lowercase_analyzer'. When you search on DISPLAY_NAME

On Monday, April 28, 2014 2:31:24 PM UTC+1, cyrilforce wrote:
Hi Dan,

Not sure whether i have same understanding "edge n-gram filter at
search time" refer to apply edge n-gram filter in the query ? If this
is the case i think i didn't apply any analyzer in the query :

*Here my query : *

{
"query" : {
"multi_match": {
"query": "happy",
"fields": [ "DISPLAY_NAME.AUTOCOMPLETE" ]
}
},
"sort" : [{
"_score" : { "order" : "desc"},
"DISPLAY_NAME.NAME" : { "order" : "asc" }}]
}

*The result : *
edge_ngram_original.json · GitHub

As you notice in the result it returned :

Happenings
Happier?

which aren't part of what i searched for.

*The mapping : *
edge_ngram_mapping_original.json · GitHub

*The index : *
{
"jdbc_dev": {
"settings": {
"index.number_of_shards": "5",
"index.number_of_replicas": "1",
"index.version.created": "900299",
"index.analysis.filter.my_edge_ngram_filter.type":
"edgeNGram",
"index.analysis.analyzer.edge_ngram_keyword_lowercase_analyzer.filter.1":
"lowercase",
"index.analysis.analyzer.edge_ngram_keyword_lowercase_analyzer.filter.0":
"my_edge_ngram_filter",
"index.analysis.analyzer.edge_ngram_keyword_lowercase_analyzer.tokenizer":
"keyword",
"index.analysis.filter.my_edge_ngram_filter.min_gram": "2",
"index.analysis.filter.my_edge_ngram_filter.max_gram": "30"
}
}
}

After that with Adrien suggestion i tried create the analyzer

"analysis" : {
"analyzer" : {
"keyword_lowercase_analyzer" : {
"tokenizer" : "keyword",
"filter" : ["lowercase"]
}
}
}

and recreate the mapping with :

"DISPLAY_NAME": {
            "type": "string",
            "search_analyzer" : "keyword_lowercase_analyzer"
}

however with this configuration it wasn't able to search via "hap" to
return the following result :

1)Happy
2)Happy birthday
3)Happy valentine

Thanks

On Mon, Apr 28, 2014 at 8:17 PM, Dan Tuffery dan.t...@gmail.com wrote:
As Adrien said don't apply the edge n-gram filter at search time. When
you tried it what does you mapping look like, because "hap" should return a
result.

Dan

On Monday, April 28, 2014 12:41:38 PM UTC+1, cyrilforce wrote:
Hi dan

I tried this before however it returns me some irrelevant result as
well :
irrevelent_response_edge_ngram.json · GitHub

How I could get only the relevant result with the sorting via edge
ngram ? Thanks
On Apr 28, 2014 7:36 PM, "Dan Tuffery" dan.t...@gmail.com wrote:
To return the query results in the order you want, execute the sort
on a not_analyzed version of the term, i.e

"DISPLAY_NAME": {
"type": "string",
"fields": {
"NAME": {
"type": "string",
"analyzer": "edge_ngram_keyword_lowercase_analyzer"
},
"untouched":{
"type": "string",
"analyzer":"not_analyzed"
}
}
}

Reindex the data. Now in your query you can sort on the 'untouched'
field:

{
"from" : "0",
"size" : "20",
"explain" : true,
"query" : {
"multi_match": {
     "query": "happy",
      "fields": [ "DISPLAY_NAME.name"]
 }
},
"sort" : [{ 
  "DISPLAY_NAME.untouched" : { "order" : "desc"}}] 
}

which will return

1)Happy
2)Happy birthday
3)Happy valentine

Dan

On Monday, April 28, 2014 9:50:44 AM UTC+1, cyrilforce wrote:
Hi All,

I am trying to use *edge-ngram filter *to achieve auto-completion
behaviour. However when i run the following query :

{
"from" : "0",
"size" : "20",
"explain" : true,
"query" : {
*"multi_match": {*
 "query": "happy",*
 "fields": [ "DISPLAY_NAME.NAME <http://DISPLAY_NAME.NAME>"]*
}*
},
"sort" : [{
"_score" : { "order" : "desc"}}]
}
It returns me documents :

irrevelent_response_edge_ngram.json · GitHub

May i know how can i achieve the behaviour when i search "happy"
then is should return me following result instead :

1)Happy
2)Happy birthday
3)Happy valentine

The analzyer configure :

"analysis" : {
"analyzer" : {
"edge_ngram_keyword_lowercase_analyzer" : {
"tokenizer" : "keyword",
"filter" : ["my_edge_ngram_filter","lowercase"]
}
},
"filter" : {
"my_edge_ngram_filter" : {
"type" : "edgeNGram",
"min_gram" : "2",
"max_gram" : "30"
                }
            }
}

The mapping :
irrevelent_response_ngram_edge_mapping.json · GitHub

Thanks.

--
You received this message because you are subscribed to a topic in
the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/to
pic/elasticsearch/fhsCgFBF76E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/7ad16d4e-aed3-474e-893d-92cc788003b0%40goo
glegroups.com https://groups.google.com/d/msgid/elasticsearch/7ad16d4e-aed3-474e-893d-92cc788003b0%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/
topic/elasticsearch/fhsCgFBF76E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/f7b7df20-4136-4cb6-9dcc-318502ac05f9%
40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/f7b7df20-4136-4cb6-9dcc-318502ac05f9%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.
--
Regards,

Chee Hoo
--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/fhsCgFBF76E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/f296c0d1-0248-4d62-aba4-057373596343%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/f296c0d1-0248-4d62-aba4-057373596343%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.
--
Regards,

Chee Hoo

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a49fed20-b734-42ac-ad51-ee9947ebe909%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

cyrilforce · April 30, 2014, 2:21am

Nice. Thanks.

On Tue, Apr 29, 2014 at 2:03 AM, Dan Tuffery dan.tuffery@gmail.com wrote:

You could use a function score query and use the field length as the score:

{
"from": "0",
"size": "20",
"explain": true,
"query": {
"function_score": {
"query": {
"multi_match": {
"query": "happy",
"fields": [
"DISPLAY_NAME.name"
]
}
},
"script_score": {
"script": "_score =
pow(doc['DISPLAY_NAME.untouched'].value.length(), -1)"
}
}
}
}

Dan

On Monday, April 28, 2014 3:08:24 PM UTC+1, cyrilforce wrote:
Hi Dan,

It worked ! Btw one more questions. Now the result returned me the same
scores for all :

1)"_score": 1.6931472,
"_source": {
"DISPLAY_NAME": "Happy Tears",

"_score": 1.6931472,
"_source": {
"DISPLAY_NAME": "Happy Happy Sangeethalu",

3)"_score": 1.6931472,
"_source": {
"DISPLAY_NAME": "Happy",

Anyway to manipulate the 3) to appear first ?

*The query : *

{
"explain" : true,
"query" : {
"multi_match": {
"query": "happy",
"fields": [ "DISPLAY_NAME.AUTOCOMPLETE" ]
}
},
"sort" : [{
"_score" : { "order" : "desc"},
"DISPLAY_NAME.NAME" : { "order" : "asc" }}]
}

Thanks.

On Mon, Apr 28, 2014 at 9:43 PM, Dan Tuffery dan.t...@gmail.com wrote:
Change your mapping to:

DISPLAY_NAME": {
"type": "multi_field",
"fields": {
"AUTOCOMPLETE": {
"type": "string",
"index_analyzer": "edge_ngram_keyword_lowercase_
analyzer",
"include_in_all": false
},
"NAME": {
"type": "string",
"analyzer": "standard",
"include_in_all": false
}
}
}

the DISPLAY_NAME term will be indexed using
'edge_ngram_keyword_lowercase_analyzer'. When you search on DISPLAY_NAME

On Monday, April 28, 2014 2:31:24 PM UTC+1, cyrilforce wrote:
Hi Dan,

Not sure whether i have same understanding "edge n-gram filter at
search time" refer to apply edge n-gram filter in the query ? If this
is the case i think i didn't apply any analyzer in the query :

*Here my query : *

{
"query" : {
"multi_match": {
"query": "happy",
"fields": [ "DISPLAY_NAME.AUTOCOMPLETE" ]
}
},
"sort" : [{
"_score" : { "order" : "desc"},
"DISPLAY_NAME.NAME" : { "order" : "asc" }}]
}

*The result : *
edge_ngram_original.json · GitHub

As you notice in the result it returned :

Happenings
Happier?

which aren't part of what i searched for.

*The mapping : *
edge_ngram_mapping_original.json · GitHub

*The index : *
{
"jdbc_dev": {
"settings": {
"index.number_of_shards": "5",
"index.number_of_replicas": "1",
"index.version.created": "900299",
"index.analysis.filter.my_edge_ngram_filter.type":
"edgeNGram",
"index.analysis.analyzer.edge_ngram_keyword_lowercase_
analyzer.filter.1": "lowercase",
"index.analysis.analyzer.edge_ngram_keyword_lowercase_
analyzer.filter.0": "my_edge_ngram_filter",
"index.analysis.analyzer.edge_ngram_keyword_lowercase_
analyzer.tokenizer": "keyword",
"index.analysis.filter.my_edge_ngram_filter.min_gram": "2",
"index.analysis.filter.my_edge_ngram_filter.max_gram": "30"
}
}
}

After that with Adrien suggestion i tried create the analyzer

"analysis" : {
"analyzer" : {
"keyword_lowercase_analyzer" : {
"tokenizer" : "keyword",
"filter" : ["lowercase"]
}
}
}

and recreate the mapping with :

"DISPLAY_NAME": {
            "type": "string",
            "search_analyzer" : "keyword_lowercase_analyzer"
}

however with this configuration it wasn't able to search via "hap" to
return the following result :

1)Happy
2)Happy birthday
3)Happy valentine

Thanks

On Mon, Apr 28, 2014 at 8:17 PM, Dan Tuffery dan.t...@gmail.comwrote:
As Adrien said don't apply the edge n-gram filter at search time. When
you tried it what does you mapping look like, because "hap" should return a
result.

Dan

On Monday, April 28, 2014 12:41:38 PM UTC+1, cyrilforce wrote:
Hi dan

I tried this before however it returns me some irrelevant result as
well :
irrevelent_response_edge_ngram.json · GitHub

How I could get only the relevant result with the sorting via edge
ngram ? Thanks
On Apr 28, 2014 7:36 PM, "Dan Tuffery" dan.t...@gmail.com wrote:
To return the query results in the order you want, execute the
sort on a not_analyzed version of the term, i.e

"DISPLAY_NAME": {
"type": "string",
"fields": {
"NAME": {
"type": "string",
"analyzer": "edge_ngram_keyword_lowercase_analyzer"
},
"untouched":{
"type": "string",
"analyzer":"not_analyzed"
}
}
}

Reindex the data. Now in your query you can sort on the 'untouched'
field:

{
"from" : "0",
"size" : "20",
"explain" : true,
"query" : {
"multi_match": {
     "query": "happy",
      "fields": [ "DISPLAY_NAME.name"]
 }
},
"sort" : [{
  "DISPLAY_NAME.untouched" : { "order" : "desc"}}]
}

which will return

1)Happy
2)Happy birthday
3)Happy valentine

Dan

On Monday, April 28, 2014 9:50:44 AM UTC+1, cyrilforce wrote:
Hi All,

I am trying to use *edge-ngram filter *to achieve auto-completion
behaviour. However when i run the following query :

{
"from" : "0",
"size" : "20",
"explain" : true,
"query" : {
*"multi_match": {*
 "query": "happy",*
 "fields": [ "DISPLAY_NAME.NAME <http://DISPLAY_NAME.NAME>"]*
}*
},
"sort" : [{
"_score" : { "order" : "desc"}}]
}
It returns me documents :

irrevelent_response_edge_ngram.json · GitHub

May i know how can i achieve the behaviour when i search "happy"
then is should return me following result instead :

1)Happy
2)Happy birthday
3)Happy valentine

The analzyer configure :

"analysis" : {
"analyzer" : {
"edge_ngram_keyword_lowercase_analyzer" : {
"tokenizer" : "keyword",
"filter" : ["my_edge_ngram_filter","lowercase"]
}
},
"filter" : {
"my_edge_ngram_filter" : {
"type" : "edgeNGram",
"min_gram" : "2",
"max_gram" : "30"
                }
            }
}

The mapping :
irrevelent_response_ngram_edge_mapping.json · GitHub

Thanks.

--
You received this message because you are subscribed to a topic in
the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/to
pic/elasticsearch/fhsCgFBF76E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/7ad16d4e-aed
3-474e-893d-92cc788003b0%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/7ad16d4e-aed3-474e-893d-92cc788003b0%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/to
pic/elasticsearch/fhsCgFBF76E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/f7b7df20-4136-4cb6-9dcc-318502ac05f9%40goo
glegroups.com https://groups.google.com/d/msgid/elasticsearch/f7b7df20-4136-4cb6-9dcc-318502ac05f9%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.
--
Regards,

Chee Hoo
--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/
topic/elasticsearch/fhsCgFBF76E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/f296c0d1-0248-4d62-aba4-057373596343%
40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/f296c0d1-0248-4d62-aba4-057373596343%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.
--
Regards,

Chee Hoo
--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/fhsCgFBF76E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/a49fed20-b734-42ac-ad51-ee9947ebe909%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/a49fed20-b734-42ac-ad51-ee9947ebe909%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Regards,

Chee Hoo

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGS0%2Bg-7j%3Dwbw9kx6iVMuQfXoJt5k1JU3M%3Dk77B-rwb0-1RkKQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Topic		Replies	Views
Relevant data not coming in elasticsearch Elasticsearch	2	265	June 28, 2021
Edge-ngram and irrelevant scoring Elasticsearch	2	563	March 27, 2020
Issue with elasticsearch edge_ngram query Elasticsearch	1	343	May 29, 2020
Edge-ngram not working for single edge case! Elasticsearch	2	176	August 4, 2022
Edge Ngram not working on querying all fields Elasticsearch	1	621	July 4, 2017

Edge-Ngram returns irrelevant result

Related topics