Completion suggester caching

Elastic Stack Elasticsearch
1

Hello,

I am using completion suggester for user search on my website. I basically
followed the howto from elasticsearch webpage.

Created the mapping:
post /gruppu/user/_mapping
{
"user" : {
"properties" : {
"name" : { "type" : "string" },
"suggest" : { "type" : "completion",
"index_analyzer" : "simple",
"search_analyzer" : "simple",
"payloads" : true
}
}
}
}

And I put something to the ES:
post /gruppu/user/100000001?refresh=true
{
"name" : "jorge",
"suggest" : {
"input": [ "jjorge"],
"payload" : { "userid" : 100000001, "grouporuser": "group" }
}
}'

*The problem is that when I delete or update above document, the suggester
still seems to cache the data. My question is what is the correct way to
update the documents in ES to have the suggester working correctly? *

For example the above document, after updating the document and changing
the "input" , when I search for it I get the latest version:
{"_index":"gruppu","_type":"user","_id":"100000001","_version":4,"exists":true,
"_source" : {
"name" : "jorge",
"suggest" : {
"input": [ "jorge group"],
"payload" : { "userid" : 100000001, "grouporuser": "group" }
}
}'
}

When I run the suggester I get following:
"user-suggest": [
{
"text": "j",
"offset": 0,
"length": 1,
"options": [
{
"text": "jorge group",
"score": 2,
"payload": {
"userid": 100000001,
"grouporuser": "group"
}
},
{
"text": "jjorge",
"score": 1,
"payload": {
"userid": 100000001,
"grouporuser": "group"
}
},

So what is the correct way of updating the ES documents when using the
suggester ?

Thanks,

Jorge Sanchez

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d968850f-a973-4e78-ab75-8d2558545066%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

Jorge,

May I ask what version of ES are you using? I just tried this on 1.0 RC 2
and the updates/deletes seem to be working properly as expected.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/c9bab198-4177-4063-85be-d023d299d686%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

Hi,

I have this following version:
elasticsearch-0.90.10

PS: I can open the server with public IP to be reachable from internet if
you would like to see that behavior yourself.
Regards,

Jorge

Dne pátek, 7. února 2014 12:25:13 UTC-6 Binh Ly napsal(a):

Jorge,

May I ask what version of ES are you using? I just tried this on 1.0 RC 2
and the updates/deletes seem to be working properly as expected.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/25cfabbe-7041-400d-b4e4-5fda31dfd40e%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

4

Strange, I tried it on 0.90.10 and it works as expected also. Can you
please post the complete sequence of steps (exact commands starting with an
empty cluster) from beginning to end so that I can try it and see if I can
reproduce your problem? Thanks.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/991828e1-e933-4adb-be0a-ce88dd2afc2d%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

5

Hi,

here is the sequence:
put /gruppu
post /gruppu/user/_mapping
{
"user" : {
"properties" : {
"name" : { "type" : "string" },
"suggest" : { "type" : "completion",
"index_analyzer" : "simple",
"search_analyzer" : "simple",
"payloads" : true
}
}
}
}
post /gruppu/user/3?refresh=true
{
"name" : "markoff",
"suggest" : {
"input": [ "markoff" ],
"payload" : { "userid" : 3, "grouporuser": "user" }
}
}'
post /gruppu/user/2?refresh=true
{
"name" : "jorge",
"suggest" : {
"input": [ "jorge", "sanchez", "jorge sanchez" ],
"payload" : { "userid" : 2, "grouporuser": "user" }
}
}'
post /gruppu/user/100000001?refresh=true
{
"name" : "group jorge",
"suggest" : {
"input": [ "jjorge"],
"payload" : { "userid" : 100000001, "grouporuser": "group" }
}
}'
post /gruppu/user/100000001?refresh=true
{
"name" : "group jorge",
"suggest" : {
"input": [ "jorge"],
"payload" : { "userid" : 100000001, "grouporuser": "group" }
}
}'
post /gruppu/_suggest?pretty
{"user-suggest" : {
"text" : "j",
"completion" : {
"field" : "suggest"
}
}
}'

I even tried to delete the records the suggester still sees those records...

Regards,

Jorge

Dne pátek, 7. února 2014 15:04:47 UTC-6 Binh Ly napsal(a):

Strange, I tried it on 0.90.10 and it works as expected also. Can you
please post the complete sequence of steps (exact commands starting with an
empty cluster) from beginning to end so that I can try it and see if I can
reproduce your problem? Thanks.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/12efae85-f71a-4227-bab4-2dd065d480ca%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

6

Jorge,

I ran your sequence and I got these results which look correct:

{
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"user-suggest" : [ {
"text" : "j",
"offset" : 0,
"length" : 1,
"options" : [ {
"text" : "jorge",
"score" : 1.0, "payload" : {"userid":2,"grouporuser":"user"}
}, {
"text" : "jorge sanchez",
"score" : 1.0, "payload" : {"userid":2,"grouporuser":"user"}
} ]
} ]
}

Then I did this:

DELETE http://localhost:9200/gruppu/user/2?refresh=true

And query again, and I got this:

{
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"user-suggest" : [ {
"text" : "j",
"offset" : 0,
"length" : 1,
"options" : [ {
"text" : "jorge",
"score" : 1.0, "payload" : {"userid":100000001,"grouporuser":"group"}
} ]
} ]
}

And then

DELETE http://localhost:9200/gruppu/user/100000001?refresh=true

And query again, and I got this:

{
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"user-suggest" : [ {
"text" : "j",
"offset" : 0,
"length" : 1,
"options" : [ ]
} ]
}

Keep in mind that when you suggest on "j", this entry will produce 2
suggestions because it matches both "jorge" and "jorge sanchez":

post /gruppu/user/2?refresh=true
{
"name" : "jorge",
"suggest" : {
"input": [ "jorge", "sanchez", "jorge sanchez" ],
"payload" : { "userid" : 2, "grouporuser": "user" }
}
}

If you don't want that behavior, set the output property to a single string
that will uniquely identify a single suggestion (instead of 2 suggestions
for the above entry). So for example:

{
"name" : "jorge",
"suggest" : {
"input": [ "jorge", "sanchez", "jorge sanchez" ],
"payload" : { "userid" : 2, "grouporuser": "user" },
"output": "jorge sanchez"
}
}

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5829c521-c0ff-4a9b-bbad-8bccff77feca%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

7

Hi,

really appreciate that you are looking into this. I got it working with elasticsearch-1.0.0.RC2
release. I don't know what was wrong with 0.90. Now the deletes and updates
are ok anyway I have a question. If there are two documents with "exactly"
the same input - in my case i put two documents with input 'jorge' , is
there a way how to get both of the documents returned for suggest query ? I
only get one back which probably has the highest priority.

Thanks,

Jorge

Dne pátek, 7. února 2014 16:24:30 UTC-6 Binh Ly napsal(a):

Jorge,

I ran your sequence and I got these results which look correct:

{
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"user-suggest" : [ {
"text" : "j",
"offset" : 0,
"length" : 1,
"options" : [ {
"text" : "jorge",
"score" : 1.0, "payload" : {"userid":2,"grouporuser":"user"}
}, {
"text" : "jorge sanchez",
"score" : 1.0, "payload" : {"userid":2,"grouporuser":"user"}
} ]
} ]
}

Then I did this:

DELETE http://localhost:9200/gruppu/user/2?refresh=true

And query again, and I got this:

{
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"user-suggest" : [ {
"text" : "j",
"offset" : 0,
"length" : 1,
"options" : [ {
"text" : "jorge",
"score" : 1.0, "payload" : {"userid":100000001,"grouporuser":"group"}
} ]
} ]
}

And then

DELETE http://localhost:9200/gruppu/user/100000001?refresh=true

And query again, and I got this:

{
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"user-suggest" : [ {
"text" : "j",
"offset" : 0,
"length" : 1,
"options" :
} ]
}

Keep in mind that when you suggest on "j", this entry will produce 2
suggestions because it matches both "jorge" and "jorge sanchez":

post /gruppu/user/2?refresh=true
{
"name" : "jorge",
"suggest" : {
"input": [ "jorge", "sanchez", "jorge sanchez" ],
"payload" : { "userid" : 2, "grouporuser": "user" }
}
}

If you don't want that behavior, set the output property to a single
string that will uniquely identify a single suggestion (instead of 2
suggestions for the above entry). So for example:

{
"name" : "jorge",
"suggest" : {
"input": [ "jorge", "sanchez", "jorge sanchez" ],
"payload" : { "userid" : 2, "grouporuser": "user" },
"output": "jorge sanchez"
}
}

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1c0ef13b-6d6a-460e-aa97-934df7bee0ec%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

8

Hey Jorge,

you can have two exact the same inputs. However, those need to have a
different output in order to occur as different suggestions (as those are
important for the data being returned).

--Alex

On Sat, Feb 8, 2014 at 4:02 AM, Jorge Sanchez xsanch@gmail.com wrote:

Hi,

really appreciate that you are looking into this. I got it working with elasticsearch-1.0.0.RC2
release. I don't know what was wrong with 0.90. Now the deletes and updates
are ok anyway I have a question. If there are two documents with "exactly"
the same input - in my case i put two documents with input 'jorge' , is
there a way how to get both of the documents returned for suggest query ? I
only get one back which probably has the highest priority.

Thanks,

Jorge

Dne pátek, 7. února 2014 16:24:30 UTC-6 Binh Ly napsal(a):

Jorge,

I ran your sequence and I got these results which look correct:

{
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"user-suggest" : [ {
"text" : "j",
"offset" : 0,
"length" : 1,
"options" : [ {
"text" : "jorge",
"score" : 1.0, "payload" : {"userid":2,"grouporuser":"user"}
}, {
"text" : "jorge sanchez",
"score" : 1.0, "payload" : {"userid":2,"grouporuser":"user"}
} ]
} ]
}

Then I did this:

DELETE http://localhost:9200/gruppu/user/2?refresh=true

And query again, and I got this:

{
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"user-suggest" : [ {
"text" : "j",
"offset" : 0,
"length" : 1,
"options" : [ {
"text" : "jorge",
"score" : 1.0, "payload" : {"userid":100000001,"
grouporuser":"group"}
} ]
} ]
}

And then

DELETE http://localhost:9200/gruppu/user/100000001?refresh=true

And query again, and I got this:

{
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"user-suggest" : [ {
"text" : "j",
"offset" : 0,
"length" : 1,
"options" :
} ]
}

Keep in mind that when you suggest on "j", this entry will produce 2
suggestions because it matches both "jorge" and "jorge sanchez":

post /gruppu/user/2?refresh=true
{
"name" : "jorge",
"suggest" : {
"input": [ "jorge", "sanchez", "jorge sanchez" ],
"payload" : { "userid" : 2, "grouporuser": "user" }
}
}

If you don't want that behavior, set the output property to a single
string that will uniquely identify a single suggestion (instead of 2
suggestions for the above entry). So for example:

{
"name" : "jorge",
"suggest" : {
"input": [ "jorge", "sanchez", "jorge sanchez" ],
"payload" : { "userid" : 2, "grouporuser": "user" },
"output": "jorge sanchez"
}
}

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1c0ef13b-6d6a-460e-aa97-934df7bee0ec%40googlegroups.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGCwEM_r9mBg_vbij-%3DzkTZoX0Xhxp9gCLA7c22464XBkB%2BZ8A%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

9

Hi,

then I am not sure if the suggester is the right tool to use for my use
case. This is how I want to use ES:

  • I have model where are users, tags and groups. All of them can have the
    same values. The values searched are single word tokens no sentences
  • The search would be done using the twitter typeahead so I need to return
    the token which would represent the searched result. In case of the same
    value (like the group and user with the same name ) the token would be the
    same

Using suggester is then not really helpful, I could do some dirty hacks
like making sure that the output would be always different (appending some
text to it) and using the payload where I would insert all tokens of the
user. This approach doesn't seem right to me and I am sure there must be a
better approach to this regular use case.

Thanks again for looking into this , I really appreciate your time.

Regards,

Jorge

Dne pondělí, 10. února 2014 6:57:49 UTC-6 Alexander Reelsen napsal(a):

Hey Jorge,

you can have two exact the same inputs. However, those need to have a
different output in order to occur as different suggestions (as those are
important for the data being returned).

--Alex

On Sat, Feb 8, 2014 at 4:02 AM, Jorge Sanchez <xsa...@gmail.com<javascript:>

wrote:

Hi,

really appreciate that you are looking into this. I got it working with elasticsearch-1.0.0.RC2
release. I don't know what was wrong with 0.90. Now the deletes and updates
are ok anyway I have a question. If there are two documents with "exactly"
the same input - in my case i put two documents with input 'jorge' , is
there a way how to get both of the documents returned for suggest query ? I
only get one back which probably has the highest priority.

Thanks,

Jorge

Dne pátek, 7. února 2014 16:24:30 UTC-6 Binh Ly napsal(a):

Jorge,

I ran your sequence and I got these results which look correct:

{
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"user-suggest" : [ {
"text" : "j",
"offset" : 0,
"length" : 1,
"options" : [ {
"text" : "jorge",
"score" : 1.0, "payload" : {"userid":2,"grouporuser":"user"}
}, {
"text" : "jorge sanchez",
"score" : 1.0, "payload" : {"userid":2,"grouporuser":"user"}
} ]
} ]
}

Then I did this:

DELETE http://localhost:9200/gruppu/user/2?refresh=true

And query again, and I got this:

{
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"user-suggest" : [ {
"text" : "j",
"offset" : 0,
"length" : 1,
"options" : [ {
"text" : "jorge",
"score" : 1.0, "payload" : {"userid":100000001,"
grouporuser":"group"}
} ]
} ]
}

And then

DELETE http://localhost:9200/gruppu/user/100000001?refresh=true

And query again, and I got this:

{
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"user-suggest" : [ {
"text" : "j",
"offset" : 0,
"length" : 1,
"options" :
} ]
}

Keep in mind that when you suggest on "j", this entry will produce 2
suggestions because it matches both "jorge" and "jorge sanchez":

post /gruppu/user/2?refresh=true
{
"name" : "jorge",
"suggest" : {
"input": [ "jorge", "sanchez", "jorge sanchez" ],
"payload" : { "userid" : 2, "grouporuser": "user" }
}
}

If you don't want that behavior, set the output property to a single
string that will uniquely identify a single suggestion (instead of 2
suggestions for the above entry). So for example:

{
"name" : "jorge",
"suggest" : {
"input": [ "jorge", "sanchez", "jorge sanchez" ],
"payload" : { "userid" : 2, "grouporuser": "user" },
"output": "jorge sanchez"
}
}

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1c0ef13b-6d6a-460e-aa97-934df7bee0ec%40googlegroups.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b3f01564-8325-4fa9-bbb8-2b0760d5b36a%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

10

Hey,

most likely I didnt get your use-case. But something like twitter typeahead
would return a twitter handle as output, which is unique? Or something like
"@spinscale | Alexander Reelsen", which is unique as well. Not sure what
kind of output you want to return which is not unique. If you are searching
in message contents, using the completion suggester might indeed not be the
solution...

--Alex

On Mon, Feb 10, 2014 at 5:30 PM, Jorge Sanchez xsanch@gmail.com wrote:

Hi,

then I am not sure if the suggester is the right tool to use for my use
case. This is how I want to use ES:

  • I have model where are users, tags and groups. All of them can have the
    same values. The values searched are single word tokens no sentences
  • The search would be done using the twitter typeahead so I need to return
    the token which would represent the searched result. In case of the same
    value (like the group and user with the same name ) the token would be the
    same

Using suggester is then not really helpful, I could do some dirty hacks
like making sure that the output would be always different (appending some
text to it) and using the payload where I would insert all tokens of the
user. This approach doesn't seem right to me and I am sure there must be a
better approach to this regular use case.

Thanks again for looking into this , I really appreciate your time.

Regards,

Jorge

Dne pondělí, 10. února 2014 6:57:49 UTC-6 Alexander Reelsen napsal(a):

Hey Jorge,

you can have two exact the same inputs. However, those need to have a
different output in order to occur as different suggestions (as those are
important for the data being returned).

--Alex

On Sat, Feb 8, 2014 at 4:02 AM, Jorge Sanchez xsa...@gmail.com wrote:

Hi,

really appreciate that you are looking into this. I got it working with elasticsearch-1.0.0.RC2
release. I don't know what was wrong with 0.90. Now the deletes and updates
are ok anyway I have a question. If there are two documents with "exactly"
the same input - in my case i put two documents with input 'jorge' , is
there a way how to get both of the documents returned for suggest query ? I
only get one back which probably has the highest priority.

Thanks,

Jorge

Dne pátek, 7. února 2014 16:24:30 UTC-6 Binh Ly napsal(a):

Jorge,

I ran your sequence and I got these results which look correct:

{
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"user-suggest" : [ {
"text" : "j",
"offset" : 0,
"length" : 1,
"options" : [ {
"text" : "jorge",
"score" : 1.0, "payload" : {"userid":2,"grouporuser":"user"}
}, {
"text" : "jorge sanchez",
"score" : 1.0, "payload" : {"userid":2,"grouporuser":"user"}
} ]
} ]
}

Then I did this:

DELETE http://localhost:9200/gruppu/user/2?refresh=true

And query again, and I got this:

{
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"user-suggest" : [ {
"text" : "j",
"offset" : 0,
"length" : 1,
"options" : [ {
"text" : "jorge",
"score" : 1.0, "payload" : {"userid":100000001,"grouporus
er":"group"}
} ]
} ]
}

And then

DELETE http://localhost:9200/gruppu/user/100000001?refresh=true

And query again, and I got this:

{
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"user-suggest" : [ {
"text" : "j",
"offset" : 0,
"length" : 1,
"options" :
} ]
}

Keep in mind that when you suggest on "j", this entry will produce 2
suggestions because it matches both "jorge" and "jorge sanchez":

post /gruppu/user/2?refresh=true
{
"name" : "jorge",
"suggest" : {
"input": [ "jorge", "sanchez", "jorge sanchez" ],
"payload" : { "userid" : 2, "grouporuser": "user" }
}
}

If you don't want that behavior, set the output property to a single
string that will uniquely identify a single suggestion (instead of 2
suggestions for the above entry). So for example:

{
"name" : "jorge",
"suggest" : {
"input": [ "jorge", "sanchez", "jorge sanchez" ],
"payload" : { "userid" : 2, "grouporuser": "user" },
"output": "jorge sanchez"
}
}

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/1c0ef13b-6d6a-460e-aa97-934df7bee0ec%
40googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/b3f01564-8325-4fa9-bbb8-2b0760d5b36a%40googlegroups.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGCwEM9-3aL32eZ92EOZ3G_N9Da2rK6pZy%2BFpB__7Tb42NuYSQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

11

Hi,

twitter typeahead is autocomplete library which is broadly used to
implement such features on websits. It allows to fetch remote suggests via
AJAX which is the way how I use it. The AJAX query has the value the user
searched for , in below case I typed "j" in the search bar. The typeahead
expects back JSON, in the JSON you need to specify the tokens for the
suggest so in my case I am returning tokens "jorge" and "jorge sanchez". If
the user continues to type jorge then autocomplete would use these tokens
to match.

This is the JSON i reply with:
[{"tokens": ["jorge", "jorge sanchez"], "usertype": "user",
"profileImageUrl":
"https://gruppuoutput.s3.amazonaws.com/avatar/jorgeresized100jorge2012-10-13%2014.38.58.jpg?Signature=MLI5yLAZeWJp2S9WxJsg6TReFI0%3D&Expires=1392150840&AWSAccessKeyId=AKIAIXRNLLCMPTZU7ICA",
"name": "jorge sanchez", "value": "jorge sanchez"}]

The tokens in JSON response are taken from ES suggest query, as I don't
specify "output" when indexing I am getting back the matched input from the
document.
Now as I can have multiple users with the same real name or have a group
with the same name, the tokens for individual suggest would be the same.

I am going forward with this solution - I will put all the tokens into
"input" and "payload" of the document, I will set the "output" to be
randmized string of some big length. When I will do
suggest I would be getting all the results as the output would be different
and I would grab tokens from payload.

Regards,

Jorge

Dne úterý, 11. února 2014 7:10:10 UTC-6 Alexander Reelsen napsal(a):

Hey,

most likely I didnt get your use-case. But something like twitter
typeahead would return a twitter handle as output, which is unique? Or
something like "@spinscale | Alexander Reelsen", which is unique as well.
Not sure what kind of output you want to return which is not unique. If you
are searching in message contents, using the completion suggester might
indeed not be the solution...

--Alex

On Mon, Feb 10, 2014 at 5:30 PM, Jorge Sanchez <xsa...@gmail.com<javascript:>

wrote:

Hi,

then I am not sure if the suggester is the right tool to use for my use
case. This is how I want to use ES:

  • I have model where are users, tags and groups. All of them can have the
    same values. The values searched are single word tokens no sentences
  • The search would be done using the twitter typeahead so I need to
    return the token which would represent the searched result. In case of the
    same value (like the group and user with the same name ) the token would be
    the same

Using suggester is then not really helpful, I could do some dirty hacks
like making sure that the output would be always different (appending some
text to it) and using the payload where I would insert all tokens of the
user. This approach doesn't seem right to me and I am sure there must be a
better approach to this regular use case.

Thanks again for looking into this , I really appreciate your time.

Regards,

Jorge

Dne pondělí, 10. února 2014 6:57:49 UTC-6 Alexander Reelsen napsal(a):

Hey Jorge,

you can have two exact the same inputs. However, those need to have a
different output in order to occur as different suggestions (as those are
important for the data being returned).

--Alex

On Sat, Feb 8, 2014 at 4:02 AM, Jorge Sanchez xsa...@gmail.com wrote:

Hi,

really appreciate that you are looking into this. I got it working with elasticsearch-1.0.0.RC2
release. I don't know what was wrong with 0.90. Now the deletes and updates
are ok anyway I have a question. If there are two documents with "exactly"
the same input - in my case i put two documents with input 'jorge' , is
there a way how to get both of the documents returned for suggest query ? I
only get one back which probably has the highest priority.

Thanks,

Jorge

Dne pátek, 7. února 2014 16:24:30 UTC-6 Binh Ly napsal(a):

Jorge,

I ran your sequence and I got these results which look correct:

{
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"user-suggest" : [ {
"text" : "j",
"offset" : 0,
"length" : 1,
"options" : [ {
"text" : "jorge",
"score" : 1.0, "payload" : {"userid":2,"grouporuser":"user"}
}, {
"text" : "jorge sanchez",
"score" : 1.0, "payload" : {"userid":2,"grouporuser":"user"}
} ]
} ]
}

Then I did this:

DELETE http://localhost:9200/gruppu/user/2?refresh=true

And query again, and I got this:

{
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"user-suggest" : [ {
"text" : "j",
"offset" : 0,
"length" : 1,
"options" : [ {
"text" : "jorge",
"score" : 1.0, "payload" : {"userid":100000001,"grouporus
er":"group"}
} ]
} ]
}

And then

DELETE http://localhost:9200/gruppu/user/100000001?refresh=true

And query again, and I got this:

{
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"user-suggest" : [ {
"text" : "j",
"offset" : 0,
"length" : 1,
"options" :
} ]
}

Keep in mind that when you suggest on "j", this entry will produce 2
suggestions because it matches both "jorge" and "jorge sanchez":

post /gruppu/user/2?refresh=true
{
"name" : "jorge",
"suggest" : {
"input": [ "jorge", "sanchez", "jorge sanchez" ],
"payload" : { "userid" : 2, "grouporuser": "user" }
}
}

If you don't want that behavior, set the output property to a single
string that will uniquely identify a single suggestion (instead of 2
suggestions for the above entry). So for example:

{
"name" : "jorge",
"suggest" : {
"input": [ "jorge", "sanchez", "jorge sanchez" ],
"payload" : { "userid" : 2, "grouporuser": "user" },
"output": "jorge sanchez"
}
}

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/1c0ef13b-6d6a-460e-aa97-934df7bee0ec%
40googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/b3f01564-8325-4fa9-bbb8-2b0760d5b36a%40googlegroups.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/acc37746-f0d8-49b5-b83d-9703c4ebaa15%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

12

I ran into same "caching" problem with latest ES 1.3.4.
I updated documents which just hold completion field to use with completion
suggester, even if i deleted docs via id and they can't be found via
_search endpoint anymore i still get suggests for those deleted documents.
I use a cluster with 2 nodes. Any suggestions?!

Am Freitag, 7. Februar 2014 16:04:01 UTC+1 schrieb Jorge Sanchez:

Hello,

I am using completion suggester for user search on my website. I basically
followed the howto from elasticsearch webpage.

Created the mapping:
post /gruppu/user/_mapping
{
"user" : {
"properties" : {
"name" : { "type" : "string" },
"suggest" : { "type" : "completion",
"index_analyzer" : "simple",
"search_analyzer" : "simple",
"payloads" : true
}
}
}
}

And I put something to the ES:
post /gruppu/user/100000001?refresh=true
{
"name" : "jorge",
"suggest" : {
"input": [ "jjorge"],
"payload" : { "userid" : 100000001, "grouporuser": "group" }
}
}'

*The problem is that when I delete or update above document, the
suggester still seems to cache the data. My question is what is the correct
way to update the documents in ES to have the suggester working correctly? *

For example the above document, after updating the document and changing
the "input" , when I search for it I get the latest version:
{"_index":"gruppu","_type":"user","_id":"100000001","_version":4,"exists":true,
"_source" : {
"name" : "jorge",
"suggest" : {
"input": [ "jorge group"],
"payload" : { "userid" : 100000001, "grouporuser": "group" }
}
}'
}

When I run the suggester I get following:
"user-suggest": [
{
"text": "j",
"offset": 0,
"length": 1,
"options": [
{
"text": "jorge group",
"score": 2,
"payload": {
"userid": 100000001,
"grouporuser": "group"
}
},
{
"text": "jjorge",
"score": 1,
"payload": {
"userid": 100000001,
"grouporuser": "group"
}
},

So what is the correct way of updating the ES documents when using the
suggester ?

Thanks,

Jorge Sanchez

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f4bcada2-f0a4-4ea3-a2d7-5de54920fda1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

13

Addendum: And against a 1.3.2 cluster with 2 nodes it works as expected,
but not in 1.3.4 .... ?

Am Montag, 27. Oktober 2014 15:09:10 UTC+1 schrieb Tom:

I ran into same "caching" problem with latest ES 1.3.4.
I updated documents which just hold completion field to use with
completion suggester, even if i deleted docs via id and they can't be found
via _search endpoint anymore i still get suggests for those deleted
documents.
I use a cluster with 2 nodes. Any suggestions?!

Am Freitag, 7. Februar 2014 16:04:01 UTC+1 schrieb Jorge Sanchez:

Hello,

I am using completion suggester for user search on my website. I
basically followed the howto from elasticsearch webpage.

Created the mapping:
post /gruppu/user/_mapping
{
"user" : {
"properties" : {
"name" : { "type" : "string" },
"suggest" : { "type" : "completion",
"index_analyzer" : "simple",
"search_analyzer" : "simple",
"payloads" : true
}
}
}
}

And I put something to the ES:
post /gruppu/user/100000001?refresh=true
{
"name" : "jorge",
"suggest" : {
"input": [ "jjorge"],
"payload" : { "userid" : 100000001, "grouporuser": "group" }
}
}'

*The problem is that when I delete or update above document, the
suggester still seems to cache the data. My question is what is the correct
way to update the documents in ES to have the suggester working correctly? *

For example the above document, after updating the document and changing
the "input" , when I search for it I get the latest version:
{"_index":"gruppu","_type":"user","_id":"100000001","_version":4,"exists":true,
"_source" : {
"name" : "jorge",
"suggest" : {
"input": [ "jorge group"],
"payload" : { "userid" : 100000001, "grouporuser": "group" }
}
}'
}

When I run the suggester I get following:
"user-suggest": [
{
"text": "j",
"offset": 0,
"length": 1,
"options": [
{
"text": "jorge group",
"score": 2,
"payload": {
"userid": 100000001,
"grouporuser": "group"
}
},
{
"text": "jjorge",
"score": 1,
"payload": {
"userid": 100000001,
"grouporuser": "group"
}
},

So what is the correct way of updating the ES documents when using the
suggester ?

Thanks,

Jorge Sanchez

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/54e3a018-c903-47dd-9e4a-4e420ea37964%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

14

Answer: On the failing cluster we had accidently same node names for two
nodes. After fixing that everything is fine.
I am wondering: Shouldn't ES come up with some kind of warning when same
node name is used twice in a cluster or do names haven't to be unique
actually?

Am Montag, 27. Oktober 2014 15:13:30 UTC+1 schrieb Tom:

Addendum: And against a 1.3.2 cluster with 2 nodes it works as expected,
but not in 1.3.4 .... ?

Am Montag, 27. Oktober 2014 15:09:10 UTC+1 schrieb Tom:

I ran into same "caching" problem with latest ES 1.3.4.
I updated documents which just hold completion field to use with
completion suggester, even if i deleted docs via id and they can't be found
via _search endpoint anymore i still get suggests for those deleted
documents.
I use a cluster with 2 nodes. Any suggestions?!

Am Freitag, 7. Februar 2014 16:04:01 UTC+1 schrieb Jorge Sanchez:

Hello,

I am using completion suggester for user search on my website. I
basically followed the howto from elasticsearch webpage.

Created the mapping:
post /gruppu/user/_mapping
{
"user" : {
"properties" : {
"name" : { "type" : "string" },
"suggest" : { "type" : "completion",
"index_analyzer" : "simple",
"search_analyzer" : "simple",
"payloads" : true
}
}
}
}

And I put something to the ES:
post /gruppu/user/100000001?refresh=true
{
"name" : "jorge",
"suggest" : {
"input": [ "jjorge"],
"payload" : { "userid" : 100000001, "grouporuser": "group" }
}
}'

*The problem is that when I delete or update above document, the
suggester still seems to cache the data. My question is what is the correct
way to update the documents in ES to have the suggester working correctly? *

For example the above document, after updating the document and changing
the "input" , when I search for it I get the latest version:
{"_index":"gruppu","_type":"user","_id":"100000001","_version":4,"exists":true,
"_source" : {
"name" : "jorge",
"suggest" : {
"input": [ "jorge group"],
"payload" : { "userid" : 100000001, "grouporuser": "group" }
}
}'
}

When I run the suggester I get following:
"user-suggest": [
{
"text": "j",
"offset": 0,
"length": 1,
"options": [
{
"text": "jorge group",
"score": 2,
"payload": {
"userid": 100000001,
"grouporuser": "group"
}
},
{
"text": "jjorge",
"score": 1,
"payload": {
"userid": 100000001,
"grouporuser": "group"
}
},

So what is the correct way of updating the ES documents when using the
suggester ?

Thanks,

Jorge Sanchez

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0209a805-5150-4564-b53e-88d113b14348%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.