Edge-Ngram returns irrelevant result


(cyrilforce) #1

Hi All,

I am trying to use *edge-ngram filter *to achieve auto-completion
behaviour. However when i run the following query :

{
"from" : "0",
"size" : "20",
"explain" : true,

"query" : {
*"multi_match": {*
  •  "query": "happy",*
    
  •  "fields": [ "DISPLAY_NAME.NAME"]*
    
  • }*
    
    },
    "sort" : [{
    "_score" : { "order" : "desc"}}]
    }

It returns me documents :

May i know how can i achieve the behaviour when i search "happy" then is
should return me following result instead :

1)Happy
2)Happy birthday
3)Happy valentine

The analzyer configure :

"analysis" : {
"analyzer" : {
"edge_ngram_keyword_lowercase_analyzer" : {
"tokenizer" : "keyword",
"filter" : ["my_edge_ngram_filter","lowercase"]
}
},
"filter" : {
"my_edge_ngram_filter" : {
"type" : "edgeNGram",
"min_gram" : "2",
"max_gram" : "30"

                }
            }

}

The mapping :

Thanks.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e8a4446c-38c9-4c6b-8d2a-15374ef041a9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Adrien Grand) #2

Hi,

I think your issue is that you are also applying the edge n-gram filter at
search time. So "happy" is translated to ("ha" OR "hap" OR "happ" OR
"happy"). You need to configure your mappings with a search analyzer[1]
which is the same analyzer but without the edge-ngram filter (so just
keyword and lowercase).

On a side note, you would benefit from putting the lowercase filter in
front of your n-gram filter. This way, the lowercasing would be done only
once.

[1]
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-core-types.html#string

On Mon, Apr 28, 2014 at 10:50 AM, cyrilforce cheehoo84@gmail.com wrote:

Hi All,

I am trying to use *edge-ngram filter *to achieve auto-completion
behaviour. However when i run the following query :

{
"from" : "0",
"size" : "20",
"explain" : true,

"query" : {
*"multi_match": {*
  •  "query": "happy",*
    
  •  "fields": [ "DISPLAY_NAME.NAME <http://DISPLAY_NAME.NAME>"]*
    
  • }*
    
    },
    "sort" : [{
    "_score" : { "order" : "desc"}}]
    }

It returns me documents :

https://gist.github.com/cheehoo/11365794

May i know how can i achieve the behaviour when i search "happy" then
is should return me following result instead :

1)Happy
2)Happy birthday
3)Happy valentine

The analzyer configure :

"analysis" : {
"analyzer" : {
"edge_ngram_keyword_lowercase_analyzer" : {
"tokenizer" : "keyword",
"filter" : ["my_edge_ngram_filter","lowercase"]
}
},
"filter" : {
"my_edge_ngram_filter" : {
"type" : "edgeNGram",
"min_gram" : "2",
"max_gram" : "30"

                }
            }

}

The mapping :
https://gist.github.com/cheehoo/11365917

Thanks.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/e8a4446c-38c9-4c6b-8d2a-15374ef041a9%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/e8a4446c-38c9-4c6b-8d2a-15374ef041a9%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Adrien Grand

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j5iMm-e%3Dk-vf6_Hw%3D7EDgNzoG%2BrQLfAW6-pJdw1JbJYYQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(cyrilforce) #3

Hi Adrien,

But now when i type "hap" it doesn't return the following after i
configure with search_analzyer :

1)Happy
2)Happy birthday
3)Happy valentine

Sorry maybe i have missed the requirements at the first place. Basically i
wanted to use the edge_ngram to achieve auto-completion not full-text
matching.

Thanks.

On Mon, Apr 28, 2014 at 5:23 PM, Adrien Grand <
adrien.grand@elasticsearch.com> wrote:

Hi,

I think your issue is that you are also applying the edge n-gram filter at
search time. So "happy" is translated to ("ha" OR "hap" OR "happ" OR
"happy"). You need to configure your mappings with a search analyzer[1]
which is the same analyzer but without the edge-ngram filter (so just
keyword and lowercase).

On a side note, you would benefit from putting the lowercase filter in
front of your n-gram filter. This way, the lowercasing would be done only
once.

[1]
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-core-types.html#string

On Mon, Apr 28, 2014 at 10:50 AM, cyrilforce cheehoo84@gmail.com wrote:

Hi All,

I am trying to use *edge-ngram filter *to achieve auto-completion
behaviour. However when i run the following query :

{
"from" : "0",
"size" : "20",
"explain" : true,

"query" : {
*"multi_match": {*
  •  "query": "happy",*
    
  •  "fields": [ "DISPLAY_NAME.NAME <http://DISPLAY_NAME.NAME>"]*
    
  • }*
    
    },
    "sort" : [{
    "_score" : { "order" : "desc"}}]
    }

It returns me documents :

https://gist.github.com/cheehoo/11365794

May i know how can i achieve the behaviour when i search "happy" then
is should return me following result instead :

1)Happy
2)Happy birthday
3)Happy valentine

The analzyer configure :

"analysis" : {
"analyzer" : {
"edge_ngram_keyword_lowercase_analyzer" : {
"tokenizer" : "keyword",
"filter" : ["my_edge_ngram_filter","lowercase"]
}
},
"filter" : {
"my_edge_ngram_filter" : {
"type" : "edgeNGram",
"min_gram" : "2",
"max_gram" : "30"

                }
            }

}

The mapping :
https://gist.github.com/cheehoo/11365917

Thanks.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/e8a4446c-38c9-4c6b-8d2a-15374ef041a9%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/e8a4446c-38c9-4c6b-8d2a-15374ef041a9%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Adrien Grand

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/fhsCgFBF76E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j5iMm-e%3Dk-vf6_Hw%3D7EDgNzoG%2BrQLfAW6-pJdw1JbJYYQ%40mail.gmail.comhttps://groups.google.com/d/msgid/elasticsearch/CAL6Z4j5iMm-e%3Dk-vf6_Hw%3D7EDgNzoG%2BrQLfAW6-pJdw1JbJYYQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Regards,

Chee Hoo

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGS0%2Bg-6noA3oFDQiGMQD1BdfxEK2w5rFpdXB9-iaeRie4ak%3DA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Dan Tuffery) #4

To return the query results in the order you want, execute the sort on a
not_analyzed version of the term, i.e

"DISPLAY_NAME": {
"type": "string",
"fields": {
"NAME": {
"type": "string",
"analyzer": "edge_ngram_keyword_lowercase_analyzer"
},
"untouched":{
"type": "string",
"analyzer":"not_analyzed"
}
}
}

Reindex the data. Now in your query you can sort on the 'untouched' field:

{
"from" : "0",
"size" : "20",
"explain" : true,

"query" : {
"multi_match": {
     "query": "happy",
      "fields": [ "DISPLAY_NAME.name"]
 }
},
"sort" : [{ 
  "DISPLAY_NAME.untouched" : { "order" : "desc"}}] 

}

which will return

1)Happy
2)Happy birthday
3)Happy valentine

Dan

On Monday, April 28, 2014 9:50:44 AM UTC+1, cyrilforce wrote:

Hi All,

I am trying to use *edge-ngram filter *to achieve auto-completion
behaviour. However when i run the following query :

{
"from" : "0",
"size" : "20",
"explain" : true,

"query" : {
*"multi_match": {*
  •  "query": "happy",*
    
  •  "fields": [ "DISPLAY_NAME.NAME <http://DISPLAY_NAME.NAME>"]*
    
  • }*
    
    },
    "sort" : [{
    "_score" : { "order" : "desc"}}]
    }

It returns me documents :

https://gist.github.com/cheehoo/11365794

May i know how can i achieve the behaviour when i search "happy" then
is should return me following result instead :

1)Happy
2)Happy birthday
3)Happy valentine

The analzyer configure :

"analysis" : {
"analyzer" : {
"edge_ngram_keyword_lowercase_analyzer" : {
"tokenizer" : "keyword",
"filter" : ["my_edge_ngram_filter","lowercase"]
}
},
"filter" : {
"my_edge_ngram_filter" : {
"type" : "edgeNGram",
"min_gram" : "2",
"max_gram" : "30"

                }
            }

}

The mapping :
https://gist.github.com/cheehoo/11365917

Thanks.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/7ad16d4e-aed3-474e-893d-92cc788003b0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(cyrilforce) #5

Hi dan

I tried this before however it returns me some irrelevant result as well :

How I could get only the relevant result with the sorting via edge ngram ?
Thanks
On Apr 28, 2014 7:36 PM, "Dan Tuffery" dan.tuffery@gmail.com wrote:

To return the query results in the order you want, execute the sort on a
not_analyzed version of the term, i.e

"DISPLAY_NAME": {
"type": "string",
"fields": {
"NAME": {
"type": "string",
"analyzer": "edge_ngram_keyword_lowercase_analyzer"
},
"untouched":{
"type": "string",
"analyzer":"not_analyzed"
}
}
}

Reindex the data. Now in your query you can sort on the 'untouched' field:

{
"from" : "0",
"size" : "20",
"explain" : true,

"query" : {
"multi_match": {
     "query": "happy",
      "fields": [ "DISPLAY_NAME.name"]
 }
},
"sort" : [{
  "DISPLAY_NAME.untouched" : { "order" : "desc"}}]

}

which will return

1)Happy
2)Happy birthday
3)Happy valentine

Dan

On Monday, April 28, 2014 9:50:44 AM UTC+1, cyrilforce wrote:

Hi All,

I am trying to use *edge-ngram filter *to achieve auto-completion
behaviour. However when i run the following query :

{
"from" : "0",
"size" : "20",
"explain" : true,

"query" : {
*"multi_match": {*
  •  "query": "happy",*
    
  •  "fields": [ "DISPLAY_NAME.NAME <http://DISPLAY_NAME.NAME>"]*
    
  • }*
    
    },
    "sort" : [{
    "_score" : { "order" : "desc"}}]
    }

It returns me documents :

https://gist.github.com/cheehoo/11365794

May i know how can i achieve the behaviour when i search "happy" then
is should return me following result instead :

1)Happy
2)Happy birthday
3)Happy valentine

The analzyer configure :

"analysis" : {
"analyzer" : {
"edge_ngram_keyword_lowercase_analyzer" : {
"tokenizer" : "keyword",
"filter" : ["my_edge_ngram_filter","lowercase"]
}
},
"filter" : {
"my_edge_ngram_filter" : {
"type" : "edgeNGram",
"min_gram" : "2",
"max_gram" : "30"

                }
            }

}

The mapping :
https://gist.github.com/cheehoo/11365917

Thanks.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/fhsCgFBF76E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/7ad16d4e-aed3-474e-893d-92cc788003b0%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/7ad16d4e-aed3-474e-893d-92cc788003b0%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGS0%2Bg-yN%3DBa5%2BUgCcSwz4hVeAbo8O0c2BV1Wod1zvsKb_QzKg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Dan Tuffery) #6

As Adrien said don't apply the edge n-gram filter at search time. When you
tried it what does you mapping look like, because "hap" should return a
result.

Dan

On Monday, April 28, 2014 12:41:38 PM UTC+1, cyrilforce wrote:

Hi dan

I tried this before however it returns me some irrelevant result as well :
https://gist.github.com/cheehoo/11365794

How I could get only the relevant result with the sorting via edge ngram ?
Thanks
On Apr 28, 2014 7:36 PM, "Dan Tuffery" <dan.t...@gmail.com <javascript:>>
wrote:

To return the query results in the order you want, execute the sort on a
not_analyzed version of the term, i.e

"DISPLAY_NAME": {
"type": "string",
"fields": {
"NAME": {
"type": "string",
"analyzer": "edge_ngram_keyword_lowercase_analyzer"
},
"untouched":{
"type": "string",
"analyzer":"not_analyzed"
}
}
}

Reindex the data. Now in your query you can sort on the 'untouched' field:

{
"from" : "0",
"size" : "20",
"explain" : true,

"query" : {
"multi_match": {
     "query": "happy",
      "fields": [ "DISPLAY_NAME.name"]
 }
},
"sort" : [{ 
  "DISPLAY_NAME.untouched" : { "order" : "desc"}}] 

}

which will return

1)Happy
2)Happy birthday
3)Happy valentine

Dan

On Monday, April 28, 2014 9:50:44 AM UTC+1, cyrilforce wrote:

Hi All,

I am trying to use *edge-ngram filter *to achieve auto-completion
behaviour. However when i run the following query :

{
"from" : "0",
"size" : "20",
"explain" : true,

"query" : {
*"multi_match": {*
  •  "query": "happy",*
    
  •  "fields": [ "DISPLAY_NAME.NAME <http://DISPLAY_NAME.NAME>"]*
    
  • }*
    
    },
    "sort" : [{
    "_score" : { "order" : "desc"}}]
    }

It returns me documents :

https://gist.github.com/cheehoo/11365794

May i know how can i achieve the behaviour when i search "happy" then
is should return me following result instead :

1)Happy
2)Happy birthday
3)Happy valentine

The analzyer configure :

"analysis" : {
"analyzer" : {
"edge_ngram_keyword_lowercase_analyzer" : {
"tokenizer" : "keyword",
"filter" : ["my_edge_ngram_filter","lowercase"]
}
},
"filter" : {
"my_edge_ngram_filter" : {
"type" : "edgeNGram",
"min_gram" : "2",
"max_gram" : "30"

                }
            }

}

The mapping :
https://gist.github.com/cheehoo/11365917

Thanks.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/fhsCgFBF76E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/7ad16d4e-aed3-474e-893d-92cc788003b0%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/7ad16d4e-aed3-474e-893d-92cc788003b0%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f7b7df20-4136-4cb6-9dcc-318502ac05f9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(cyrilforce) #7

Hi Dan,

Not sure whether i have same understanding "edge n-gram filter at search
time
" refer to apply edge n-gram filter in the query ? If this is the case
i think i didn't apply any analyzer in the query :

*Here my query : *

{
"query" : {
"multi_match": {
"query": "happy",
"fields": [ "DISPLAY_NAME.AUTOCOMPLETE" ]
}
},
"sort" : [{
"_score" : { "order" : "desc"},
"DISPLAY_NAME.NAME" : { "order" : "asc" }}]
}

*The result : *

As you notice in the result it returned :

Happenings
Happier?

which aren't part of what i searched for.

*The mapping : *

*The index : *
{
"jdbc_dev": {
"settings": {
"index.number_of_shards": "5",
"index.number_of_replicas": "1",
"index.version.created": "900299",
"index.analysis.filter.my_edge_ngram_filter.type": "edgeNGram",

"index.analysis.analyzer.edge_ngram_keyword_lowercase_analyzer.filter.1":
"lowercase",

"index.analysis.analyzer.edge_ngram_keyword_lowercase_analyzer.filter.0":
"my_edge_ngram_filter",

"index.analysis.analyzer.edge_ngram_keyword_lowercase_analyzer.tokenizer":
"keyword",
"index.analysis.filter.my_edge_ngram_filter.min_gram": "2",
"index.analysis.filter.my_edge_ngram_filter.max_gram": "30"
}
}
}

After that with Adrien suggestion i tried create the analyzer

"analysis" : {
"analyzer" : {
"keyword_lowercase_analyzer" : {
"tokenizer" : "keyword",
"filter" : ["lowercase"]
}
}
}

and recreate the mapping with :

"DISPLAY_NAME": {
"type": "string",
"search_analyzer" : "keyword_lowercase_analyzer"

}

however with this configuration it wasn't able to search via "hap" to
return the following result :

1)Happy
2)Happy birthday
3)Happy valentine

Thanks

On Mon, Apr 28, 2014 at 8:17 PM, Dan Tuffery dan.tuffery@gmail.com wrote:

As Adrien said don't apply the edge n-gram filter at search time. When you
tried it what does you mapping look like, because "hap" should return a
result.

Dan

On Monday, April 28, 2014 12:41:38 PM UTC+1, cyrilforce wrote:

Hi dan

I tried this before however it returns me some irrelevant result as well :
https://gist.github.com/cheehoo/11365794

How I could get only the relevant result with the sorting via edge ngram
? Thanks
On Apr 28, 2014 7:36 PM, "Dan Tuffery" dan.t...@gmail.com wrote:

To return the query results in the order you want, execute the sort on a
not_analyzed version of the term, i.e

"DISPLAY_NAME": {
"type": "string",
"fields": {
"NAME": {
"type": "string",
"analyzer": "edge_ngram_keyword_lowercase_analyzer"
},
"untouched":{
"type": "string",
"analyzer":"not_analyzed"
}
}
}

Reindex the data. Now in your query you can sort on the 'untouched'
field:

{
"from" : "0",
"size" : "20",
"explain" : true,

"query" : {
"multi_match": {
     "query": "happy",
      "fields": [ "DISPLAY_NAME.name"]
 }
},
"sort" : [{
  "DISPLAY_NAME.untouched" : { "order" : "desc"}}]

}

which will return

1)Happy
2)Happy birthday
3)Happy valentine

Dan

On Monday, April 28, 2014 9:50:44 AM UTC+1, cyrilforce wrote:

Hi All,

I am trying to use *edge-ngram filter *to achieve auto-completion
behaviour. However when i run the following query :

{
"from" : "0",
"size" : "20",
"explain" : true,

"query" : {
*"multi_match": {*
  •  "query": "happy",*
    
  •  "fields": [ "DISPLAY_NAME.NAME <http://DISPLAY_NAME.NAME>"]*
    
  • }*
    
    },
    "sort" : [{
    "_score" : { "order" : "desc"}}]
    }

It returns me documents :

https://gist.github.com/cheehoo/11365794

May i know how can i achieve the behaviour when i search "happy"
then is should return me following result instead :

1)Happy
2)Happy birthday
3)Happy valentine

The analzyer configure :

"analysis" : {
"analyzer" : {
"edge_ngram_keyword_lowercase_analyzer" : {
"tokenizer" : "keyword",
"filter" : ["my_edge_ngram_filter","lowercase"]
}
},
"filter" : {
"my_edge_ngram_filter" : {
"type" : "edgeNGram",
"min_gram" : "2",
"max_gram" : "30"

                }
            }

}

The mapping :
https://gist.github.com/cheehoo/11365917

Thanks.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/
topic/elasticsearch/fhsCgFBF76E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/7ad16d4e-aed3-474e-893d-92cc788003b0%
40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/7ad16d4e-aed3-474e-893d-92cc788003b0%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/fhsCgFBF76E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/f7b7df20-4136-4cb6-9dcc-318502ac05f9%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/f7b7df20-4136-4cb6-9dcc-318502ac05f9%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Regards,

Chee Hoo

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGS0%2Bg95SAL--gSTfUm6gJRYq2pEeDgQiVqH%3DcoV%2BffLSn56tg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Dan Tuffery) #8

Change your mapping to:

DISPLAY_NAME": {
"type": "multi_field",
"fields": {
"AUTOCOMPLETE": {
"type": "string",
"index_analyzer":
"edge_ngram_keyword_lowercase_analyzer",
"include_in_all": false
},
"NAME": {
"type": "string",
"analyzer": "standard",
"include_in_all": false
}
}
}

the DISPLAY_NAME term will be indexed using
'edge_ngram_keyword_lowercase_analyzer'. When you search on DISPLAY_NAME

On Monday, April 28, 2014 2:31:24 PM UTC+1, cyrilforce wrote:

Hi Dan,

Not sure whether i have same understanding "edge n-gram filter at search
time
" refer to apply edge n-gram filter in the query ? If this is the
case i think i didn't apply any analyzer in the query :

*Here my query : *

{
"query" : {
"multi_match": {
"query": "happy",
"fields": [ "DISPLAY_NAME.AUTOCOMPLETE" ]
}
},
"sort" : [{
"_score" : { "order" : "desc"},
"DISPLAY_NAME.NAME" : { "order" : "asc" }}]
}

*The result : *
https://gist.github.com/cheehoo/11371837

As you notice in the result it returned :

Happenings
Happier?

which aren't part of what i searched for.

*The mapping : *
https://gist.github.com/cheehoo/11371897

*The index : *
{
"jdbc_dev": {
"settings": {
"index.number_of_shards": "5",
"index.number_of_replicas": "1",
"index.version.created": "900299",
"index.analysis.filter.my_edge_ngram_filter.type": "edgeNGram",

"index.analysis.analyzer.edge_ngram_keyword_lowercase_analyzer.filter.1":
"lowercase",

"index.analysis.analyzer.edge_ngram_keyword_lowercase_analyzer.filter.0":
"my_edge_ngram_filter",

"index.analysis.analyzer.edge_ngram_keyword_lowercase_analyzer.tokenizer":
"keyword",
"index.analysis.filter.my_edge_ngram_filter.min_gram": "2",
"index.analysis.filter.my_edge_ngram_filter.max_gram": "30"
}
}
}

After that with Adrien suggestion i tried create the analyzer

"analysis" : {
"analyzer" : {
"keyword_lowercase_analyzer" : {
"tokenizer" : "keyword",
"filter" : ["lowercase"]
}
}
}

and recreate the mapping with :

"DISPLAY_NAME": {

            "type": "string",
            "search_analyzer" : "keyword_lowercase_analyzer"

}

however with this configuration it wasn't able to search via "hap" to
return the following result :

1)Happy
2)Happy birthday
3)Happy valentine

Thanks

On Mon, Apr 28, 2014 at 8:17 PM, Dan Tuffery <dan.t...@gmail.com<javascript:>

wrote:

As Adrien said don't apply the edge n-gram filter at search time. When
you tried it what does you mapping look like, because "hap" should return a
result.

Dan

On Monday, April 28, 2014 12:41:38 PM UTC+1, cyrilforce wrote:

Hi dan

I tried this before however it returns me some irrelevant result as well
:
https://gist.github.com/cheehoo/11365794

How I could get only the relevant result with the sorting via edge ngram
? Thanks
On Apr 28, 2014 7:36 PM, "Dan Tuffery" dan.t...@gmail.com wrote:

To return the query results in the order you want, execute the sort
on a not_analyzed version of the term, i.e

"DISPLAY_NAME": {
"type": "string",
"fields": {
"NAME": {
"type": "string",
"analyzer": "edge_ngram_keyword_lowercase_analyzer"
},
"untouched":{
"type": "string",
"analyzer":"not_analyzed"
}
}
}

Reindex the data. Now in your query you can sort on the 'untouched'
field:

{
"from" : "0",
"size" : "20",
"explain" : true,

"query" : {
"multi_match": {
     "query": "happy",
      "fields": [ "DISPLAY_NAME.name"]
 }
},
"sort" : [{ 
  "DISPLAY_NAME.untouched" : { "order" : "desc"}}] 

}

which will return

1)Happy
2)Happy birthday
3)Happy valentine

Dan

On Monday, April 28, 2014 9:50:44 AM UTC+1, cyrilforce wrote:

Hi All,

I am trying to use *edge-ngram filter *to achieve auto-completion
behaviour. However when i run the following query :

{
"from" : "0",
"size" : "20",
"explain" : true,

"query" : {
*"multi_match": {*
  •  "query": "happy",*
    
  •  "fields": [ "DISPLAY_NAME.NAME <http://DISPLAY_NAME.NAME>"]*
    
  • }*
    
    },
    "sort" : [{
    "_score" : { "order" : "desc"}}]
    }

It returns me documents :

https://gist.github.com/cheehoo/11365794

May i know how can i achieve the behaviour when i search "happy"
then is should return me following result instead :

1)Happy
2)Happy birthday
3)Happy valentine

The analzyer configure :

"analysis" : {
"analyzer" : {
"edge_ngram_keyword_lowercase_analyzer" : {
"tokenizer" : "keyword",
"filter" : ["my_edge_ngram_filter","lowercase"]
}
},
"filter" : {
"my_edge_ngram_filter" : {
"type" : "edgeNGram",
"min_gram" : "2",
"max_gram" : "30"

                }
            }

}

The mapping :
https://gist.github.com/cheehoo/11365917

Thanks.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/
topic/elasticsearch/fhsCgFBF76E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/7ad16d4e-aed3-474e-893d-92cc788003b0%
40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/7ad16d4e-aed3-474e-893d-92cc788003b0%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/fhsCgFBF76E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/f7b7df20-4136-4cb6-9dcc-318502ac05f9%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/f7b7df20-4136-4cb6-9dcc-318502ac05f9%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Regards,

Chee Hoo

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f296c0d1-0248-4d62-aba4-057373596343%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(cyrilforce) #9

Hi Dan,

It worked ! Btw one more questions. Now the result returned me the same
scores for all :

1)"_score": 1.6931472,
"_source": {
"DISPLAY_NAME": "Happy Tears",

  1. "_score": 1.6931472,
    "_source": {
    "DISPLAY_NAME": "Happy Happy Sangeethalu",

3)"_score": 1.6931472,
"_source": {
"DISPLAY_NAME": "Happy",

Anyway to manipulate the 3) to appear first ?

*The query : *

{
"explain" : true,
"query" : {
"multi_match": {
"query": "happy",
"fields": [ "DISPLAY_NAME.AUTOCOMPLETE" ]
}
},
"sort" : [{
"_score" : { "order" : "desc"},
"DISPLAY_NAME.NAME" : { "order" : "asc" }}]
}

Thanks.

On Mon, Apr 28, 2014 at 9:43 PM, Dan Tuffery dan.tuffery@gmail.com wrote:

Change your mapping to:

DISPLAY_NAME": {
"type": "multi_field",
"fields": {
"AUTOCOMPLETE": {
"type": "string",
"index_analyzer":
"edge_ngram_keyword_lowercase_analyzer",
"include_in_all": false
},
"NAME": {
"type": "string",
"analyzer": "standard",
"include_in_all": false
}
}
}

the DISPLAY_NAME term will be indexed using
'edge_ngram_keyword_lowercase_analyzer'. When you search on DISPLAY_NAME

On Monday, April 28, 2014 2:31:24 PM UTC+1, cyrilforce wrote:

Hi Dan,

Not sure whether i have same understanding "edge n-gram filter at
search time
" refer to apply edge n-gram filter in the query ? If this
is the case i think i didn't apply any analyzer in the query :

*Here my query : *

{
"query" : {
"multi_match": {
"query": "happy",
"fields": [ "DISPLAY_NAME.AUTOCOMPLETE" ]
}
},
"sort" : [{
"_score" : { "order" : "desc"},
"DISPLAY_NAME.NAME" : { "order" : "asc" }}]
}

*The result : *
https://gist.github.com/cheehoo/11371837

As you notice in the result it returned :

Happenings
Happier?

which aren't part of what i searched for.

*The mapping : *
https://gist.github.com/cheehoo/11371897

*The index : *
{
"jdbc_dev": {
"settings": {
"index.number_of_shards": "5",
"index.number_of_replicas": "1",
"index.version.created": "900299",
"index.analysis.filter.my_edge_ngram_filter.type":
"edgeNGram",
"index.analysis.analyzer.edge_ngram_keyword_lowercase_analyzer.filter.1":
"lowercase",
"index.analysis.analyzer.edge_ngram_keyword_lowercase_analyzer.filter.0":
"my_edge_ngram_filter",
"index.analysis.analyzer.edge_ngram_keyword_lowercase_analyzer.tokenizer":
"keyword",
"index.analysis.filter.my_edge_ngram_filter.min_gram": "2",
"index.analysis.filter.my_edge_ngram_filter.max_gram": "30"
}
}
}

After that with Adrien suggestion i tried create the analyzer

"analysis" : {
"analyzer" : {
"keyword_lowercase_analyzer" : {
"tokenizer" : "keyword",
"filter" : ["lowercase"]
}
}
}

and recreate the mapping with :

"DISPLAY_NAME": {

            "type": "string",
            "search_analyzer" : "keyword_lowercase_analyzer"

}

however with this configuration it wasn't able to search via "hap" to
return the following result :

1)Happy
2)Happy birthday
3)Happy valentine

Thanks

On Mon, Apr 28, 2014 at 8:17 PM, Dan Tuffery dan.t...@gmail.com wrote:

As Adrien said don't apply the edge n-gram filter at search time. When
you tried it what does you mapping look like, because "hap" should return a
result.

Dan

On Monday, April 28, 2014 12:41:38 PM UTC+1, cyrilforce wrote:

Hi dan

I tried this before however it returns me some irrelevant result as
well :
https://gist.github.com/cheehoo/11365794

How I could get only the relevant result with the sorting via edge
ngram ? Thanks
On Apr 28, 2014 7:36 PM, "Dan Tuffery" dan.t...@gmail.com wrote:

To return the query results in the order you want, execute the sort
on a not_analyzed version of the term, i.e

"DISPLAY_NAME": {
"type": "string",
"fields": {
"NAME": {
"type": "string",
"analyzer": "edge_ngram_keyword_lowercase_analyzer"
},
"untouched":{
"type": "string",
"analyzer":"not_analyzed"
}
}
}

Reindex the data. Now in your query you can sort on the 'untouched'
field:

{
"from" : "0",
"size" : "20",
"explain" : true,

"query" : {
"multi_match": {
     "query": "happy",
      "fields": [ "DISPLAY_NAME.name"]
 }
},
"sort" : [{
  "DISPLAY_NAME.untouched" : { "order" : "desc"}}]

}

which will return

1)Happy
2)Happy birthday
3)Happy valentine

Dan

On Monday, April 28, 2014 9:50:44 AM UTC+1, cyrilforce wrote:

Hi All,

I am trying to use *edge-ngram filter *to achieve auto-completion
behaviour. However when i run the following query :

{
"from" : "0",
"size" : "20",
"explain" : true,

"query" : {
*"multi_match": {*
  •  "query": "happy",*
    
  •  "fields": [ "DISPLAY_NAME.NAME <http://DISPLAY_NAME.NAME>"]*
    
  • }*
    
    },
    "sort" : [{
    "_score" : { "order" : "desc"}}]
    }

It returns me documents :

https://gist.github.com/cheehoo/11365794

May i know how can i achieve the behaviour when i search "happy"
then is should return me following result instead :

1)Happy
2)Happy birthday
3)Happy valentine

The analzyer configure :

"analysis" : {
"analyzer" : {
"edge_ngram_keyword_lowercase_analyzer" : {
"tokenizer" : "keyword",
"filter" : ["my_edge_ngram_filter","lowercase"]
}
},
"filter" : {
"my_edge_ngram_filter" : {
"type" : "edgeNGram",
"min_gram" : "2",
"max_gram" : "30"

                }
            }

}

The mapping :
https://gist.github.com/cheehoo/11365917

Thanks.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/to
pic/elasticsearch/fhsCgFBF76E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/7ad16d4e-aed3-474e-893d-92cc788003b0%40goo
glegroups.comhttps://groups.google.com/d/msgid/elasticsearch/7ad16d4e-aed3-474e-893d-92cc788003b0%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/
topic/elasticsearch/fhsCgFBF76E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/f7b7df20-4136-4cb6-9dcc-318502ac05f9%
40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/f7b7df20-4136-4cb6-9dcc-318502ac05f9%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Regards,

Chee Hoo

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/fhsCgFBF76E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/f296c0d1-0248-4d62-aba4-057373596343%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/f296c0d1-0248-4d62-aba4-057373596343%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Regards,

Chee Hoo

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGS0%2Bg-Hc1PSLqQpd%2Bs2d9hzQ0b2MREUtXWoFh_-%3DqjtqNYmOA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Dan Tuffery) #10

You could use a function score query and use the field length as the score:

{
"from": "0",
"size": "20",
"explain": true,
"query": {
"function_score": {
"query": {
"multi_match": {
"query": "happy",
"fields": [
"DISPLAY_NAME.name"
]
}
},
"script_score": {
"script": "_score =
pow(doc['DISPLAY_NAME.untouched'].value.length(), -1)"
}
}
}
}

Dan

On Monday, April 28, 2014 3:08:24 PM UTC+1, cyrilforce wrote:

Hi Dan,

It worked ! Btw one more questions. Now the result returned me the same
scores for all :

1)"_score": 1.6931472,
"_source": {
"DISPLAY_NAME": "Happy Tears",

  1. "_score": 1.6931472,
    "_source": {
    "DISPLAY_NAME": "Happy Happy Sangeethalu",

3)"_score": 1.6931472,
"_source": {
"DISPLAY_NAME": "Happy",

Anyway to manipulate the 3) to appear first ?

*The query : *

{
"explain" : true,
"query" : {
"multi_match": {
"query": "happy",
"fields": [ "DISPLAY_NAME.AUTOCOMPLETE" ]
}
},
"sort" : [{
"_score" : { "order" : "desc"},
"DISPLAY_NAME.NAME" : { "order" : "asc" }}]
}

Thanks.

On Mon, Apr 28, 2014 at 9:43 PM, Dan Tuffery <dan.t...@gmail.com<javascript:>

wrote:

Change your mapping to:

DISPLAY_NAME": {
"type": "multi_field",
"fields": {
"AUTOCOMPLETE": {
"type": "string",
"index_analyzer":
"edge_ngram_keyword_lowercase_analyzer",
"include_in_all": false
},
"NAME": {
"type": "string",
"analyzer": "standard",
"include_in_all": false
}
}
}

the DISPLAY_NAME term will be indexed using
'edge_ngram_keyword_lowercase_analyzer'. When you search on DISPLAY_NAME

On Monday, April 28, 2014 2:31:24 PM UTC+1, cyrilforce wrote:

Hi Dan,

Not sure whether i have same understanding "edge n-gram filter at
search time
" refer to apply edge n-gram filter in the query ? If this
is the case i think i didn't apply any analyzer in the query :

*Here my query : *

{
"query" : {
"multi_match": {
"query": "happy",
"fields": [ "DISPLAY_NAME.AUTOCOMPLETE" ]
}
},
"sort" : [{
"_score" : { "order" : "desc"},
"DISPLAY_NAME.NAME" : { "order" : "asc" }}]
}

*The result : *
https://gist.github.com/cheehoo/11371837

As you notice in the result it returned :

Happenings
Happier?

which aren't part of what i searched for.

*The mapping : *
https://gist.github.com/cheehoo/11371897

*The index : *
{
"jdbc_dev": {
"settings": {
"index.number_of_shards": "5",
"index.number_of_replicas": "1",
"index.version.created": "900299",
"index.analysis.filter.my_edge_ngram_filter.type":
"edgeNGram",
"index.analysis.analyzer.edge_ngram_keyword_lowercase_analyzer.filter.1":
"lowercase",
"index.analysis.analyzer.edge_ngram_keyword_lowercase_analyzer.filter.0":
"my_edge_ngram_filter",
"index.analysis.analyzer.edge_ngram_keyword_lowercase_analyzer.tokenizer":
"keyword",
"index.analysis.filter.my_edge_ngram_filter.min_gram": "2",
"index.analysis.filter.my_edge_ngram_filter.max_gram": "30"
}
}
}

After that with Adrien suggestion i tried create the analyzer

"analysis" : {
"analyzer" : {
"keyword_lowercase_analyzer" : {
"tokenizer" : "keyword",
"filter" : ["lowercase"]
}
}
}

and recreate the mapping with :

"DISPLAY_NAME": {

            "type": "string",
            "search_analyzer" : "keyword_lowercase_analyzer"

}

however with this configuration it wasn't able to search via "hap" to
return the following result :

1)Happy
2)Happy birthday
3)Happy valentine

Thanks

On Mon, Apr 28, 2014 at 8:17 PM, Dan Tuffery dan.t...@gmail.com wrote:

As Adrien said don't apply the edge n-gram filter at search time. When
you tried it what does you mapping look like, because "hap" should return a
result.

Dan

On Monday, April 28, 2014 12:41:38 PM UTC+1, cyrilforce wrote:

Hi dan

I tried this before however it returns me some irrelevant result as
well :
https://gist.github.com/cheehoo/11365794

How I could get only the relevant result with the sorting via edge
ngram ? Thanks
On Apr 28, 2014 7:36 PM, "Dan Tuffery" dan.t...@gmail.com wrote:

To return the query results in the order you want, execute the sort
on a not_analyzed version of the term, i.e

"DISPLAY_NAME": {
"type": "string",
"fields": {
"NAME": {
"type": "string",
"analyzer": "edge_ngram_keyword_lowercase_analyzer"
},
"untouched":{
"type": "string",
"analyzer":"not_analyzed"
}
}
}

Reindex the data. Now in your query you can sort on the 'untouched'
field:

{
"from" : "0",
"size" : "20",
"explain" : true,

"query" : {
"multi_match": {
     "query": "happy",
      "fields": [ "DISPLAY_NAME.name"]
 }
},
"sort" : [{ 
  "DISPLAY_NAME.untouched" : { "order" : "desc"}}] 

}

which will return

1)Happy
2)Happy birthday
3)Happy valentine

Dan

On Monday, April 28, 2014 9:50:44 AM UTC+1, cyrilforce wrote:

Hi All,

I am trying to use *edge-ngram filter *to achieve auto-completion
behaviour. However when i run the following query :

{
"from" : "0",
"size" : "20",
"explain" : true,

"query" : {
*"multi_match": {*
  •  "query": "happy",*
    
  •  "fields": [ "DISPLAY_NAME.NAME <http://DISPLAY_NAME.NAME>"]*
    
  • }*
    
    },
    "sort" : [{
    "_score" : { "order" : "desc"}}]
    }

It returns me documents :

https://gist.github.com/cheehoo/11365794

May i know how can i achieve the behaviour when i search "happy"
then is should return me following result instead :

1)Happy
2)Happy birthday
3)Happy valentine

The analzyer configure :

"analysis" : {
"analyzer" : {
"edge_ngram_keyword_lowercase_analyzer" : {
"tokenizer" : "keyword",
"filter" : ["my_edge_ngram_filter","lowercase"]
}
},
"filter" : {
"my_edge_ngram_filter" : {
"type" : "edgeNGram",
"min_gram" : "2",
"max_gram" : "30"

                }
            }

}

The mapping :
https://gist.github.com/cheehoo/11365917

Thanks.

--
You received this message because you are subscribed to a topic in
the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/to
pic/elasticsearch/fhsCgFBF76E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/7ad16d4e-aed3-474e-893d-92cc788003b0%40goo
glegroups.comhttps://groups.google.com/d/msgid/elasticsearch/7ad16d4e-aed3-474e-893d-92cc788003b0%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/
topic/elasticsearch/fhsCgFBF76E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/f7b7df20-4136-4cb6-9dcc-318502ac05f9%
40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/f7b7df20-4136-4cb6-9dcc-318502ac05f9%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Regards,

Chee Hoo

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/fhsCgFBF76E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/f296c0d1-0248-4d62-aba4-057373596343%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/f296c0d1-0248-4d62-aba4-057373596343%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Regards,

Chee Hoo

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a49fed20-b734-42ac-ad51-ee9947ebe909%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(cyrilforce) #11

Nice. Thanks.

On Tue, Apr 29, 2014 at 2:03 AM, Dan Tuffery dan.tuffery@gmail.com wrote:

You could use a function score query and use the field length as the score:

{
"from": "0",
"size": "20",
"explain": true,
"query": {
"function_score": {
"query": {
"multi_match": {
"query": "happy",
"fields": [
"DISPLAY_NAME.name"
]
}
},
"script_score": {
"script": "_score =
pow(doc['DISPLAY_NAME.untouched'].value.length(), -1)"
}
}
}
}

Dan

On Monday, April 28, 2014 3:08:24 PM UTC+1, cyrilforce wrote:

Hi Dan,

It worked ! Btw one more questions. Now the result returned me the same
scores for all :

1)"_score": 1.6931472,
"_source": {
"DISPLAY_NAME": "Happy Tears",

  1. "_score": 1.6931472,
    "_source": {
    "DISPLAY_NAME": "Happy Happy Sangeethalu",

3)"_score": 1.6931472,
"_source": {
"DISPLAY_NAME": "Happy",

Anyway to manipulate the 3) to appear first ?

*The query : *

{
"explain" : true,
"query" : {
"multi_match": {
"query": "happy",
"fields": [ "DISPLAY_NAME.AUTOCOMPLETE" ]
}
},
"sort" : [{
"_score" : { "order" : "desc"},
"DISPLAY_NAME.NAME" : { "order" : "asc" }}]
}

Thanks.

On Mon, Apr 28, 2014 at 9:43 PM, Dan Tuffery dan.t...@gmail.com wrote:

Change your mapping to:

DISPLAY_NAME": {
"type": "multi_field",
"fields": {
"AUTOCOMPLETE": {
"type": "string",
"index_analyzer": "edge_ngram_keyword_lowercase_
analyzer",
"include_in_all": false
},
"NAME": {
"type": "string",
"analyzer": "standard",
"include_in_all": false
}
}
}

the DISPLAY_NAME term will be indexed using
'edge_ngram_keyword_lowercase_analyzer'. When you search on DISPLAY_NAME

On Monday, April 28, 2014 2:31:24 PM UTC+1, cyrilforce wrote:

Hi Dan,

Not sure whether i have same understanding "edge n-gram filter at
search time
" refer to apply edge n-gram filter in the query ? If this
is the case i think i didn't apply any analyzer in the query :

*Here my query : *

{
"query" : {
"multi_match": {
"query": "happy",
"fields": [ "DISPLAY_NAME.AUTOCOMPLETE" ]
}
},
"sort" : [{
"_score" : { "order" : "desc"},
"DISPLAY_NAME.NAME" : { "order" : "asc" }}]
}

*The result : *
https://gist.github.com/cheehoo/11371837

As you notice in the result it returned :

Happenings
Happier?

which aren't part of what i searched for.

*The mapping : *
https://gist.github.com/cheehoo/11371897

*The index : *
{
"jdbc_dev": {
"settings": {
"index.number_of_shards": "5",
"index.number_of_replicas": "1",
"index.version.created": "900299",
"index.analysis.filter.my_edge_ngram_filter.type":
"edgeNGram",
"index.analysis.analyzer.edge_ngram_keyword_lowercase_
analyzer.filter.1": "lowercase",
"index.analysis.analyzer.edge_ngram_keyword_lowercase_
analyzer.filter.0": "my_edge_ngram_filter",
"index.analysis.analyzer.edge_ngram_keyword_lowercase_
analyzer.tokenizer": "keyword",
"index.analysis.filter.my_edge_ngram_filter.min_gram": "2",
"index.analysis.filter.my_edge_ngram_filter.max_gram": "30"
}
}
}

After that with Adrien suggestion i tried create the analyzer

"analysis" : {
"analyzer" : {
"keyword_lowercase_analyzer" : {
"tokenizer" : "keyword",
"filter" : ["lowercase"]
}
}
}

and recreate the mapping with :

"DISPLAY_NAME": {

            "type": "string",
            "search_analyzer" : "keyword_lowercase_analyzer"

}

however with this configuration it wasn't able to search via "hap" to
return the following result :

1)Happy
2)Happy birthday
3)Happy valentine

Thanks

On Mon, Apr 28, 2014 at 8:17 PM, Dan Tuffery dan.t...@gmail.comwrote:

As Adrien said don't apply the edge n-gram filter at search time. When
you tried it what does you mapping look like, because "hap" should return a
result.

Dan

On Monday, April 28, 2014 12:41:38 PM UTC+1, cyrilforce wrote:

Hi dan

I tried this before however it returns me some irrelevant result as
well :
https://gist.github.com/cheehoo/11365794

How I could get only the relevant result with the sorting via edge
ngram ? Thanks
On Apr 28, 2014 7:36 PM, "Dan Tuffery" dan.t...@gmail.com wrote:

To return the query results in the order you want, execute the
sort on a not_analyzed version of the term, i.e

"DISPLAY_NAME": {
"type": "string",
"fields": {
"NAME": {
"type": "string",
"analyzer": "edge_ngram_keyword_lowercase_analyzer"
},
"untouched":{
"type": "string",
"analyzer":"not_analyzed"
}
}
}

Reindex the data. Now in your query you can sort on the 'untouched'
field:

{
"from" : "0",
"size" : "20",
"explain" : true,

"query" : {
"multi_match": {
     "query": "happy",
      "fields": [ "DISPLAY_NAME.name"]
 }
},
"sort" : [{
  "DISPLAY_NAME.untouched" : { "order" : "desc"}}]

}

which will return

1)Happy
2)Happy birthday
3)Happy valentine

Dan

On Monday, April 28, 2014 9:50:44 AM UTC+1, cyrilforce wrote:

Hi All,

I am trying to use *edge-ngram filter *to achieve auto-completion
behaviour. However when i run the following query :

{
"from" : "0",
"size" : "20",
"explain" : true,

"query" : {
*"multi_match": {*
  •  "query": "happy",*
    
  •  "fields": [ "DISPLAY_NAME.NAME <http://DISPLAY_NAME.NAME>"]*
    
  • }*
    
    },
    "sort" : [{
    "_score" : { "order" : "desc"}}]
    }

It returns me documents :

https://gist.github.com/cheehoo/11365794

May i know how can i achieve the behaviour when i search "happy"
then is should return me following result instead :

1)Happy
2)Happy birthday
3)Happy valentine

The analzyer configure :

"analysis" : {
"analyzer" : {
"edge_ngram_keyword_lowercase_analyzer" : {
"tokenizer" : "keyword",
"filter" : ["my_edge_ngram_filter","lowercase"]
}
},
"filter" : {
"my_edge_ngram_filter" : {
"type" : "edgeNGram",
"min_gram" : "2",
"max_gram" : "30"

                }
            }

}

The mapping :
https://gist.github.com/cheehoo/11365917

Thanks.

--
You received this message because you are subscribed to a topic in
the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/to
pic/elasticsearch/fhsCgFBF76E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/7ad16d4e-aed
3-474e-893d-92cc788003b0%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/7ad16d4e-aed3-474e-893d-92cc788003b0%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/to
pic/elasticsearch/fhsCgFBF76E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/f7b7df20-4136-4cb6-9dcc-318502ac05f9%40goo
glegroups.comhttps://groups.google.com/d/msgid/elasticsearch/f7b7df20-4136-4cb6-9dcc-318502ac05f9%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Regards,

Chee Hoo

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/
topic/elasticsearch/fhsCgFBF76E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/f296c0d1-0248-4d62-aba4-057373596343%
40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/f296c0d1-0248-4d62-aba4-057373596343%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Regards,

Chee Hoo

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/fhsCgFBF76E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/a49fed20-b734-42ac-ad51-ee9947ebe909%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/a49fed20-b734-42ac-ad51-ee9947ebe909%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Regards,

Chee Hoo

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGS0%2Bg-7j%3Dwbw9kx6iVMuQfXoJt5k1JU3M%3Dk77B-rwb0-1RkKQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(system) #12