Question about completion in ES

Hi all,

I try to put an auto completion system based on ES.

This article http://www.elasticsearch.org/blog/you-complete-me/ helps me a
lot, but I don't understand a case:

PUT /hotels

{

"mappings": {

"hotel" : {

  "properties" : {

    "name_suggest" : {

      "type" :     "completion"

    }

  }

}

}

}

PUT /hotels/hotel/1

{

"name_suggest" : {

"input" :      [

  "21 Mercure Hotel Munich",

  "24 Mercure Munich"

]

}

}

PUT hotels/hotel/2

{

"name_suggest" : {

"input" :      [

  "Monaco Munich",

  "Hotel Monaco"

]

}

}

When i post :

POST /hotels/_suggest
{
"hotels" : {
"text" : "m",
"completion" : {
"field" : "name_suggest"
}
}
}

The result is 3 hotels : "21 Mercure Hotel Munich","24 Mercure
Munich", "Monaco Munich".

But when i post this request:

POST /hotels/_suggest

{

"hotels" : {

"text" : "2",

"completion" : {

  "field" : "name_suggest"

}

}

}

There is no result ...

Any Idea ?

Franck

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a8094645-39a5-4160-87d8-8a8bd1eca444%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

You are probably using an analyzer that removes numbers?

On Mon, Oct 13, 2014 at 6:45 PM, Franck B f.belvallette@gmail.com wrote:

Hi all,

I try to put an auto completion system based on ES.

This article http://www.elasticsearch.org/blog/you-complete-me/ helps me
a lot, but I don't understand a case:

PUT /hotels

{

"mappings": {

"hotel" : {

  "properties" : {

    "name_suggest" : {

      "type" :     "completion"

    }

  }

}

}

}

PUT /hotels/hotel/1

{

"name_suggest" : {

"input" :      [

  "21 Mercure Hotel Munich",

  "24 Mercure Munich"

]

}

}

PUT hotels/hotel/2

{

"name_suggest" : {

"input" :      [

  "Monaco Munich",

  "Hotel Monaco"

]

}

}

When i post :

POST /hotels/_suggest
{
"hotels" : {
"text" : "m",
"completion" : {
"field" : "name_suggest"
}
}
}

The result is 3 hotels : "21 Mercure Hotel Munich","24 Mercure
Munich", "Monaco Munich".

But when i post this request:

POST /hotels/_suggest

{

"hotels" : {

"text" : "2",

"completion" : {

  "field" : "name_suggest"

}

}

}

There is no result ...

Any Idea ?

Franck

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/a8094645-39a5-4160-87d8-8a8bd1eca444%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/a8094645-39a5-4160-87d8-8a8bd1eca444%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Adrien Grand

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j6bhn2Z9gMoXUhR0p6_SEaHPXqDi0G4YXxUx8qy0u0rTg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Mapping is very basic. There are no analyzers.

Le mardi 14 octobre 2014 13:24:42 UTC+2, Adrien Grand a écrit :

You are probably using an analyzer that removes numbers?

On Mon, Oct 13, 2014 at 6:45 PM, Franck B <f.belv...@gmail.com
<javascript:>> wrote:

Hi all,

I try to put an auto completion system based on ES.

This article http://www.elasticsearch.org/blog/you-complete-me/ helps me
a lot, but I don't understand a case:

PUT /hotels

{

"mappings": {

"hotel" : {

  "properties" : {

    "name_suggest" : {

      "type" :     "completion"

    }

  }

}

}

}

PUT /hotels/hotel/1

{

"name_suggest" : {

"input" :      [

  "21 Mercure Hotel Munich",

  "24 Mercure Munich"

]

}

}

PUT hotels/hotel/2

{

"name_suggest" : {

"input" :      [

  "Monaco Munich",

  "Hotel Monaco"

]

}

}

When i post :

POST /hotels/_suggest
{
"hotels" : {
"text" : "m",
"completion" : {
"field" : "name_suggest"
}
}
}

The result is 3 hotels : "21 Mercure Hotel Munich","24 Mercure
Munich", "Monaco Munich".

But when i post this request:

POST /hotels/_suggest

{

"hotels" : {

"text" : "2",

"completion" : {

  "field" : "name_suggest"

}

}

}

There is no result ...

Any Idea ?

Franck

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/a8094645-39a5-4160-87d8-8a8bd1eca444%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/a8094645-39a5-4160-87d8-8a8bd1eca444%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Adrien Grand

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f3f2dfbe-ca89-4f30-afe7-f9a9fa8e44c3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Completion fields use the simple analyzer by default, which removes
numbers. If you try another analyzer that keeps numbers such as the
standard analyzer, this should work:

PUT /hotels
{
"mappings": {
"hotel" : {
"properties" : {
"name" : { "type" : "string" },
"city" : { "type" : "string" },
"name_suggest" : {
"type" : "completion",
"analyzer": "standard"
}
}
}
}
}

On Tue, Oct 14, 2014 at 2:03 PM, Franck B f.belvallette@gmail.com wrote:

Mapping is very basic. There are no analyzers.

Le mardi 14 octobre 2014 13:24:42 UTC+2, Adrien Grand a écrit :

You are probably using an analyzer that removes numbers?

On Mon, Oct 13, 2014 at 6:45 PM, Franck B f.belv...@gmail.com wrote:

Hi all,

I try to put an auto completion system based on ES.

This article http://www.elasticsearch.org/blog/you-complete-me/ helps
me a lot, but I don't understand a case:

PUT /hotels

{

"mappings": {

"hotel" : {

  "properties" : {

    "name_suggest" : {

      "type" :     "completion"

    }

  }

}

}

}

PUT /hotels/hotel/1

{

"name_suggest" : {

"input" :      [

  "21 Mercure Hotel Munich",

  "24 Mercure Munich"

]

}

}

PUT hotels/hotel/2

{

"name_suggest" : {

"input" :      [

  "Monaco Munich",

  "Hotel Monaco"

]

}

}

When i post :

POST /hotels/_suggest
{
"hotels" : {
"text" : "m",
"completion" : {
"field" : "name_suggest"
}
}
}

The result is 3 hotels : "21 Mercure Hotel Munich","24 Mercure
Munich", "Monaco Munich".

But when i post this request:

POST /hotels/_suggest

{

"hotels" : {

"text" : "2",

"completion" : {

  "field" : "name_suggest"

}

}

}

There is no result ...

Any Idea ?

Franck

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/a8094645-39a5-4160-87d8-8a8bd1eca444%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/a8094645-39a5-4160-87d8-8a8bd1eca444%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Adrien Grand

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/f3f2dfbe-ca89-4f30-afe7-f9a9fa8e44c3%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/f3f2dfbe-ca89-4f30-afe7-f9a9fa8e44c3%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Adrien Grand

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j5ZEn%3DmTj1CW6PywODQz-DgMsOyJwe454GLGqGWi4ZwpA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

You're right, it works by changing the analyzer.

Now evolving issues, I customize the analyzer

PUT hotels

{

"settings": {

  "analysis": {

     "analyzer": {

        "labelstreet_analyzer": {

           "type": "custom",

           "tokenizer": "nGram_tokenizer",

           "filter": [

              "lowercase",

              "elision",

              "stopwords"

           ]

        }},

        "tokenizer": {

           "nGram_tokenizer": {

              "type": "nGram",

              "min_gram": "2",

              "max_gram": "4",

              "token_chars": [

                 "letter",

                 "digit"

              ]

           }

        },

        "filter": {

           "elision": {

              "type": "elision",

              "article": [

                 "l",

                 "m",

                 "t",

                 "qu",

                 "n",

                 "s",

                 "j",

                 "d"

              ]

           },

           "stopwords": {

              "type": "stop",

              "stopwords": [

                 "_french_"

              ],

              "ignore_case": true

           }

        }

     }

},

"mappings": {

  "hotel": {

     "properties": {

        "name": {

           "type": "string",

           "analyzer": "labelstreet_analyzer"

        },

        "city": {

           "type": "string",

           "analyzer": "labelstreet_analyzer"

        },

        "name_suggest": {

           "type": "completion",

           "analyzer": "labelstreet_analyzer"

        }

     }

  }

}

}

PUT /hotels/hotel/1

{

"name": "Mercure Hotel Munich",

"city": "Munich",

"name_suggest": {

  "input": [

     "Mercure Hotel Munich",

     "Mercure Munich"

  ]

}

}

PUT hotels/hotel/2

{

"name": "Hotel Monaco",

"city": "Munich",

"name_suggest": {

  "input": [

     "Monaco Munich",

     "Hotel Monaco"

  ]

}

}

PUT /hotels/hotel/3

{

"name": "Courtyard by Marriot Munich City",

"city": "Munich",

"name_suggest": {

  "input": [

     "Courtyard by Marriot Munich City",

     "Marriot Munich City"

  ]

}

}

If i check the analyser, I get the result I want. That is, if I type "mun",
the tokenizer gets me a right token

GET /hotels/_analyze?analyzer=labelstreet_analyzer&text=Munich

Yet when I execute this query, the result is null

POST /hotels/_suggest

{

"hotels" : {

"text" : "mun",

"completion" : {

  "field" : "name_suggest"

}

}

}

Why the result with "mun" is not displayed ?

Thanks for your help

Franck
Le mardi 14 octobre 2014 14:21:18 UTC+2, Adrien Grand a écrit :

Completion fields use the simple analyzer by default, which removes
numbers. If you try another analyzer that keeps numbers such as the
standard analyzer, this should work:

PUT /hotels
{
"mappings": {
"hotel" : {
"properties" : {
"name" : { "type" : "string" },
"city" : { "type" : "string" },
"name_suggest" : {
"type" : "completion",
"analyzer": "standard"
}
}
}
}
}

On Tue, Oct 14, 2014 at 2:03 PM, Franck B <f.belv...@gmail.com
<javascript:>> wrote:

Mapping is very basic. There are no analyzers.

Le mardi 14 octobre 2014 13:24:42 UTC+2, Adrien Grand a écrit :

You are probably using an analyzer that removes numbers?

On Mon, Oct 13, 2014 at 6:45 PM, Franck B f.belv...@gmail.com wrote:

Hi all,

I try to put an auto completion system based on ES.

This article http://www.elasticsearch.org/blog/you-complete-me/ helps
me a lot, but I don't understand a case:

PUT /hotels

{

"mappings": {

"hotel" : {

  "properties" : {

    "name_suggest" : {

      "type" :     "completion"

    }

  }

}

}

}

PUT /hotels/hotel/1

{

"name_suggest" : {

"input" :      [

  "21 Mercure Hotel Munich",

  "24 Mercure Munich"

]

}

}

PUT hotels/hotel/2

{

"name_suggest" : {

"input" :      [

  "Monaco Munich",

  "Hotel Monaco"

]

}

}

When i post :

POST /hotels/_suggest
{
"hotels" : {
"text" : "m",
"completion" : {
"field" : "name_suggest"
}
}
}

The result is 3 hotels : "21 Mercure Hotel Munich","24 Mercure
Munich", "Monaco Munich".

But when i post this request:

POST /hotels/_suggest

{

"hotels" : {

"text" : "2",

"completion" : {

  "field" : "name_suggest"

}

}

}

There is no result ...

Any Idea ?

Franck

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/a8094645-39a5-4160-87d8-8a8bd1eca444%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/a8094645-39a5-4160-87d8-8a8bd1eca444%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Adrien Grand

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/f3f2dfbe-ca89-4f30-afe7-f9a9fa8e44c3%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/f3f2dfbe-ca89-4f30-afe7-f9a9fa8e44c3%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Adrien Grand

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8e69449f-f2f4-425b-b49a-6feb0cfe4409%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Hi Franck,

There are two issues here. First the completion suggester is not based on
n-grams but on a FST, so you should not use a n-gram tokenizer. Another
thing to know is that the completion suggester is a prefix suggester, not
an infix suggester, meaning that it can only suggest from the beginning of
the input, yet you don't have any input that starts with "Munich". The only
way to make it recommend hotels based in Munich would be to add more inputs
to your documents that start with "Munich".

On Wed, Oct 15, 2014 at 9:19 PM, Franck B f.belvallette@gmail.com wrote:

You're right, it works by changing the analyzer.

Now evolving issues, I customize the analyzer

PUT hotels

{

"settings": {

  "analysis": {

     "analyzer": {

        "labelstreet_analyzer": {

           "type": "custom",

           "tokenizer": "nGram_tokenizer",

           "filter": [

              "lowercase",

              "elision",

              "stopwords"

           ]

        }},

        "tokenizer": {

           "nGram_tokenizer": {

              "type": "nGram",

              "min_gram": "2",

              "max_gram": "4",

              "token_chars": [

                 "letter",

                 "digit"

              ]

           }

        },

        "filter": {

           "elision": {

              "type": "elision",

              "article": [

                 "l",

                 "m",

                 "t",

                 "qu",

                 "n",

                 "s",

                 "j",

                 "d"

              ]

           },

           "stopwords": {

              "type": "stop",

              "stopwords": [

                 "_french_"

              ],

              "ignore_case": true

           }

        }

     }

},

"mappings": {

  "hotel": {

     "properties": {

        "name": {

           "type": "string",

           "analyzer": "labelstreet_analyzer"

        },

        "city": {

           "type": "string",

           "analyzer": "labelstreet_analyzer"

        },

        "name_suggest": {

           "type": "completion",

           "analyzer": "labelstreet_analyzer"

        }

     }

  }

}

}

PUT /hotels/hotel/1

{

"name": "Mercure Hotel Munich",

"city": "Munich",

"name_suggest": {

  "input": [

     "Mercure Hotel Munich",

     "Mercure Munich"

  ]

}

}

PUT hotels/hotel/2

{

"name": "Hotel Monaco",

"city": "Munich",

"name_suggest": {

  "input": [

     "Monaco Munich",

     "Hotel Monaco"

  ]

}

}

PUT /hotels/hotel/3

{

"name": "Courtyard by Marriot Munich City",

"city": "Munich",

"name_suggest": {

  "input": [

     "Courtyard by Marriot Munich City",

     "Marriot Munich City"

  ]

}

}

If i check the analyser, I get the result I want. That is, if I type
"mun", the tokenizer gets me a right token

GET /hotels/_analyze?analyzer=labelstreet_analyzer&text=Munich

Yet when I execute this query, the result is null

POST /hotels/_suggest

{

"hotels" : {

"text" : "mun",

"completion" : {

  "field" : "name_suggest"

}

}

}

Why the result with "mun" is not displayed ?

Thanks for your help

Franck
Le mardi 14 octobre 2014 14:21:18 UTC+2, Adrien Grand a écrit :

Completion fields use the simple analyzer by default, which removes
numbers. If you try another analyzer that keeps numbers such as the
standard analyzer, this should work:

PUT /hotels
{
"mappings": {
"hotel" : {
"properties" : {
"name" : { "type" : "string" },
"city" : { "type" : "string" },
"name_suggest" : {
"type" : "completion",
"analyzer": "standard"
}
}
}
}
}

On Tue, Oct 14, 2014 at 2:03 PM, Franck B f.belv...@gmail.com wrote:

Mapping is very basic. There are no analyzers.

Le mardi 14 octobre 2014 13:24:42 UTC+2, Adrien Grand a écrit :

You are probably using an analyzer that removes numbers?

On Mon, Oct 13, 2014 at 6:45 PM, Franck B f.belv...@gmail.com wrote:

Hi all,

I try to put an auto completion system based on ES.

This article http://www.elasticsearch.org/blog/you-complete-me/ helps
me a lot, but I don't understand a case:

PUT /hotels

{

"mappings": {

"hotel" : {

  "properties" : {

    "name_suggest" : {

      "type" :     "completion"

    }

  }

}

}

}

PUT /hotels/hotel/1

{

"name_suggest" : {

"input" :      [

  "21 Mercure Hotel Munich",

  "24 Mercure Munich"

]

}

}

PUT hotels/hotel/2

{

"name_suggest" : {

"input" :      [

  "Monaco Munich",

  "Hotel Monaco"

]

}

}

When i post :

POST /hotels/_suggest
{
"hotels" : {
"text" : "m",
"completion" : {
"field" : "name_suggest"
}
}
}

The result is 3 hotels : "21 Mercure Hotel Munich","24 Mercure
Munich", "Monaco Munich".

But when i post this request:

POST /hotels/_suggest

{

"hotels" : {

"text" : "2",

"completion" : {

  "field" : "name_suggest"

}

}

}

There is no result ...

Any Idea ?

Franck

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/a8094645-39a5-4160-87d8-8a8bd1eca444%40goo
glegroups.com
https://groups.google.com/d/msgid/elasticsearch/a8094645-39a5-4160-87d8-8a8bd1eca444%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Adrien Grand

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/f3f2dfbe-ca89-4f30-afe7-f9a9fa8e44c3%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/f3f2dfbe-ca89-4f30-afe7-f9a9fa8e44c3%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Adrien Grand

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/8e69449f-f2f4-425b-b49a-6feb0cfe4409%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/8e69449f-f2f4-425b-b49a-6feb0cfe4409%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Adrien Grand

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j7psbDm72f38yjs%3DcWGJFHJF3CN1mo%3Dn86ed62tuC3A1w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.