# AutoCompletion Suggester - Duplicate record in suggestion return **URL:** \ **Category:** Elasticsearch\ **Created:** [April 11, 2014, 8:59am UTC](https://discuss.elastic.co/t/autocompletion-suggester-duplicate-record-in-suggestion-return/16950 "2014-04-11T08:59:07Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![zkidkid](https://avatars.discourse-cdn.com/v4/letter/z/fbc32d/32.png) [@zkidkid](https://discuss.elastic.co/u/zkidkid)\ **Post date:** [April 11, 2014, 8:59am UTC](https://discuss.elastic.co/t/autocompletion-suggester-duplicate-record-in-suggestion-return/16950/1 "2014-04-11T08:59:07Z")
Hi All, I have problem with AutoCompletion Suggester. When I index same word in record, AutoCompletion will return duplicate that record, Here is restful: curl -X PUT 10.30.58.105:9200/completion curl -X DELETE 10.30.58.105:9200/completion/song curl -X PUT 10.30.58.105:9200/completion/song/\_mapping -d '{ "song" : { "properties" : { "song-suggest" : { "type" : "completion", "index\_analyzer" : "simple", "search\_analyzer" : "simple", "payloads" : true } } } }' //insert duplicate word "Nothing" curl -X PUT '10.30.58.105:9200/completion/song/1?refresh=true' -d '{"song-suggest":{"input":["Nothing Nothing Else Matter","Nothing Else Matter","Else Matter","Matter"],"payload":{"name":"Nothing Else Matter - Metallica 1"},"weight":12}}' curl -X POST '10.30.58.105:9200/completion/\_suggest?pretty' -d '{ "song-suggest" : { "text" : "Nothing", "completion" : { "field" : "song-suggest" } } }' The return value will: { "\_shards" : { "total" : 2, "successful" : 2, "failed" : 0 }, "song-suggest" : [ { "text" : "Nothing", "offset" : 0, "length" : 7, "options" : [ { "text" : "Nothing Else Matter", "score" : 1.0, "payload" : {"name":"Nothing Else Matter - Metallica 1"} }, { "text" : "Nothing Nothing Else Matter", "score" : 1.0, "payload" : {"name":"Nothing Else Matter - Metallica 1"} } ] } ] } Although I only have one record. Thanks. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/e3c1d118-7b2f-4e03-a682-f901124c287a%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/e3c1d118-7b2f-4e03-a682-f901124c287a%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![zkidkid](https://avatars.discourse-cdn.com/v4/letter/z/fbc32d/32.png) [@zkidkid](https://discuss.elastic.co/u/zkidkid)\ **Post date:** [April 11, 2014, 9:27am UTC](https://discuss.elastic.co/t/autocompletion-suggester-duplicate-record-in-suggestion-return/16950/2 "2014-04-11T09:27:18Z")
Hi, It's really fun because when I added output field it work like a charm. Completion is built in FST, so how could "output" field affect the result ? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/7f98b542-e862-4654-bed7-a31c297d795d%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/7f98b542-e862-4654-bed7-a31c297d795d%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![zkidkid](https://avatars.discourse-cdn.com/v4/letter/z/fbc32d/32.png) [@zkidkid](https://discuss.elastic.co/u/zkidkid)\ **Post date:** [April 13, 2014, 9:53am UTC](https://discuss.elastic.co/t/autocompletion-suggester-duplicate-record-in-suggestion-return/16950/3 "2014-04-13T09:53:21Z")
There are something really strange. I don't know whether anyone have worked with this such feature or it's just not-stable feature. If we do index same input, and different output,payload, then only one result found. Do anyone tell me how could I fix it ? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/f5518627-f29d-4864-9b9f-003675e257ce%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f5518627-f29d-4864-9b9f-003675e257ce%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![zkidkid](https://avatars.discourse-cdn.com/v4/letter/z/fbc32d/32.png) [@zkidkid](https://discuss.elastic.co/u/zkidkid)\ **Post date:** [April 13, 2014, 10:49am UTC](https://discuss.elastic.co/t/autocompletion-suggester-duplicate-record-in-suggestion-return/16950/4 "2014-04-13T10:49:05Z")
I have figure out the problem. The main problem is I have used the same output for all input then ES have been wrong in this case. I still trying to improve the performance. I am just test on 64Gb Ram server (32Gb for ES 1.0.1) 24 core. Have only 2 record but it took me 3ms to suggest. On Sunday, April 13, 2014 4:53:21 PM UTC+7, kidkid wrote: > There are something really strange. > I don't know whether anyone have worked with this such feature or it's > just not-stable feature. > If we do index same input, and different output,payload, then only one > result found. > > Do anyone tell me how could I fix it ? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [April 21, 2014, 12:26pm UTC](https://discuss.elastic.co/t/autocompletion-suggester-duplicate-record-in-suggestion-return/16950/5 "2014-04-21T12:26:13Z")
Hey, the output is used to unify the search results, otherwise the input is used. The payload itself is just meta information. The main reason, why you see the suggestion twice is, that even though a document is deleted and cannot be found anymore, the suggest data structures are only cleaned up during merges/optimizations. Running optimize should fix this. Makes sense? --Alex On Sun, Apr 13, 2014 at 12:49 PM, kidkid [zkidkid@gmail.com](mailto:zkidkid@gmail.com) wrote: > I have figure out the problem. > The main problem is I have used the same output for all input then ES have > been wrong in this case. > > I still trying to improve the performance. I am just test on 64Gb Ram > server (32Gb for ES 1.0.1) 24 core. > Have only 2 record but it took me 3ms to suggest. > > On Sunday, April 13, 2014 4:53:21 PM UTC+7, kidkid wrote: > > > There are something really strange. > > I don't know whether anyone have worked with this such feature or it's > > just not-stable feature. > > If we do index same input, and different output,payload, then only one > > result found. > > > > Do anyone tell me how could I fix it ? > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAGCwEM-yu%3DzB0Zf4Ra%3DAJTz0uUkR4ASGeBDkkdFaDKYmP9VCTQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAGCwEM-yu%3DzB0Zf4Ra%3DAJTz0uUkR4ASGeBDkkdFaDKYmP9VCTQ%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![zkidkid](https://avatars.discourse-cdn.com/v4/letter/z/fbc32d/32.png) [@zkidkid](https://discuss.elastic.co/u/zkidkid)\ **Post date:** [April 22, 2014, 8:51am UTC](https://discuss.elastic.co/t/autocompletion-suggester-duplicate-record-in-suggestion-return/16950/6 "2014-04-22T08:51:59Z")
Hey Alexander, Thanks for your reply. Currently I also manual do optimize by running optimize: host:9200/completion\_index/\_optimize?max\_num\_segments=1 I think it's a work around solution. I would like to make it better. Sometime, I also have problem with update payload, when I change payload, the completion is not change too. On Monday, April 21, 2014 7:26:13 PM UTC+7, Alexander Reelsen wrote: > Hey, > > the output is used to unify the search results, otherwise the input is > used. The payload itself is just meta information. > The main reason, why you see the suggestion twice is, that even though a > document is deleted and cannot be found anymore, the suggest data > structures are only cleaned up during merges/optimizations. Running > optimize should fix this. > > Makes sense? > > --Alex > > On Sun, Apr 13, 2014 at 12:49 PM, kidkid \<[zki...@gmail.com](mailto:zki...@gmail.com) \\>wrote: > > > I have figure out the problem. > > The main problem is I have used the same output for all input then ES > > have been wrong in this case. > > > > I still trying to improve the performance. I am just test on 64Gb Ram > > server (32Gb for ES 1.0.1) 24 core. > > Have only 2 record but it took me 3ms to suggest. > > > > On Sunday, April 13, 2014 4:53:21 PM UTC+7, kidkid wrote: > > > > > There are something really strange. > > > I don't know whether anyone have worked with this such feature or it's > > > just not-stable feature. > > > If we do index same input, and different output,payload, then only one > > > result found. > > > > > > Do anyone tell me how could I fix it ? > > > > > > -- > > > You received this message because you are subscribed to the Google Groups > > > "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send an > > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%40googlegroups.com?utm_medium=email&utm_source=footer) > > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/b3bf74ad-c729-4e78-9ebd-364c6c000819%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/b3bf74ad-c729-4e78-9ebd-364c6c000819%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [April 22, 2014, 10:52pm UTC](https://discuss.elastic.co/t/autocompletion-suggester-duplicate-record-in-suggestion-return/16950/7 "2014-04-22T22:52:16Z")
Hey if you do regular updates on that index, you should not update down to one single segment. Do you have an example, where the updates do not work as expected? --Alex On Tue, Apr 22, 2014 at 10:51 AM, kidkid [zkidkid@gmail.com](mailto:zkidkid@gmail.com) wrote: > Hey Alexander, > Thanks for your reply. > Currently I also manual do optimize by running optimize: > host:9200/completion\_index/\_optimize?max\_num\_segments=1 > I think it's a work around solution. I would like to make it better. > > Sometime, I also have problem with update payload, when I change payload, > the completion is not change too. > > On Monday, April 21, 2014 7:26:13 PM UTC+7, Alexander Reelsen wrote: > > > Hey, > > > > the output is used to unify the search results, otherwise the input is > > used. The payload itself is just meta information. > > The main reason, why you see the suggestion twice is, that even though a > > document is deleted and cannot be found anymore, the suggest data > > structures are only cleaned up during merges/optimizations. Running > > optimize should fix this. > > > > Makes sense? > > > > --Alex > > > > On Sun, Apr 13, 2014 at 12:49 PM, kidkid [zki...@gmail.com](mailto:zki...@gmail.com) wrote: > > > > > I have figure out the problem. > > > The main problem is I have used the same output for all input then ES > > > have been wrong in this case. > > > > > > I still trying to improve the performance. I am just test on 64Gb Ram > > > server (32Gb for ES 1.0.1) 24 core. > > > Have only 2 record but it took me 3ms to suggest. > > > > > > On Sunday, April 13, 2014 4:53:21 PM UTC+7, kidkid wrote: > > > > > > > There are something really strange. > > > > I don't know whether anyone have worked with this such feature or it's > > > > just not-stable feature. > > > > If we do index same input, and different output,payload, then only one > > > > result found. > > > > > > > > Do anyone tell me how could I fix it ? > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > > msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd% > > > > [40googlegroups.com](http://40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > . > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/b3bf74ad-c729-4e78-9ebd-364c6c000819%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/b3bf74ad-c729-4e78-9ebd-364c6c000819%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/b3bf74ad-c729-4e78-9ebd-364c6c000819%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/b3bf74ad-c729-4e78-9ebd-364c6c000819%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAGCwEM8RaaN69kcBQTTbkXfV7RL8fkR0Y8PAuK7jR-Jdma3VGw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAGCwEM8RaaN69kcBQTTbkXfV7RL8fkR0Y8PAuK7jR-Jdma3VGw%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Kaspars\_Sprogis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kaspars_sprogis/32/1609_2.png) [@Kaspars\_Sprogis](https://discuss.elastic.co/u/Kaspars_Sprogis)\ **Post date:** [April 28, 2014, 9:21am UTC](https://discuss.elastic.co/t/autocompletion-suggester-duplicate-record-in-suggestion-return/16950/8 "2014-04-28T09:21:49Z")
Hi, I have exactly same problem. I resolved duplicates and appearance of deleted items by running "\_optimize?only\_expunge\_deletes=true" on daily basis. However i have still problem with updates. Even if data have been updated for items in index, they still show old data when searching. Only solution i found is running: "\_optimize?max\_num\_segments=1'". However i have quite a lot of updates and this get me worried, because of Alexander's note: > if you do regular updates on that index, you should not update down to one > single segment Alexander Reelsen, could you please advise, why we shouldn't do that or does this create some permanent damage for optimization logic or does it affects performance for search later? What should we do instead? Thank you. On Wednesday, April 23, 2014 1:52:16 AM UTC+3, Alexander Reelsen wrote: > Hey > > if you do regular updates on that index, you should not update down to one > single segment. Do you have an example, where the updates do not work as > expected? > > --Alex > > On Tue, Apr 22, 2014 at 10:51 AM, kidkid \<[zki...@gmail.com](mailto:zki...@gmail.com) \\>wrote: > > > Hey Alexander, > > Thanks for your reply. > > Currently I also manual do optimize by running optimize: > > host:9200/completion\_index/\_optimize?max\_num\_segments=1 > > I think it's a work around solution. I would like to make it better. > > > > Sometime, I also have problem with update payload, when I change payload, > > the completion is not change too. > > > > On Monday, April 21, 2014 7:26:13 PM UTC+7, Alexander Reelsen wrote: > > > > > Hey, > > > > > > the output is used to unify the search results, otherwise the input is > > > used. The payload itself is just meta information. > > > The main reason, why you see the suggestion twice is, that even though a > > > document is deleted and cannot be found anymore, the suggest data > > > structures are only cleaned up during merges/optimizations. Running > > > optimize should fix this. > > > > > > Makes sense? > > > > > > --Alex > > > > > > On Sun, Apr 13, 2014 at 12:49 PM, kidkid [zki...@gmail.com](mailto:zki...@gmail.com) wrote: > > > > > > > I have figure out the problem. > > > > The main problem is I have used the same output for all input then ES > > > > have been wrong in this case. > > > > > > > > I still trying to improve the performance. I am just test on 64Gb Ram > > > > server (32Gb for ES 1.0.1) 24 core. > > > > Have only 2 record but it took me 3ms to suggest. > > > > > > > > On Sunday, April 13, 2014 4:53:21 PM UTC+7, kidkid wrote: > > > > > > > > > There are something really strange. > > > > > I don't know whether anyone have worked with this such feature or it's > > > > > just not-stable feature. > > > > > If we do index same input, and different output,payload, then only one > > > > > result found. > > > > > > > > > > Do anyone tell me how could I fix it ? > > > > > > > > > > -- > > > > > You received this message because you are subscribed to the Google > > > > > Groups "elasticsearch" group. > > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > > > msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd% > > > > > [40googlegroups.com](http://40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > > . > > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > -- > > > You received this message because you are subscribed to the Google Groups > > > "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send an > > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/b3bf74ad-c729-4e78-9ebd-364c6c000819%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/b3bf74ad-c729-4e78-9ebd-364c6c000819%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/b3bf74ad-c729-4e78-9ebd-364c6c000819%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/b3bf74ad-c729-4e78-9ebd-364c6c000819%40googlegroups.com?utm_medium=email&utm_source=footer) > > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/3b03ab59-e642-42d8-ab11-86f4589d8bec%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/3b03ab59-e642-42d8-ab11-86f4589d8bec%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![zkidkid](https://avatars.discourse-cdn.com/v4/letter/z/fbc32d/32.png) [@zkidkid](https://discuss.elastic.co/u/zkidkid)\ **Post date:** [April 29, 2014, 2:49am UTC](https://discuss.elastic.co/t/autocompletion-suggester-duplicate-record-in-suggestion-return/16950/9 "2014-04-29T02:49:57Z")
Hi Alexander Reelsen, Just do at my example above. When you do update on payload or do reindex (with same doc id, same input, different output or payload) then the completion will work funny. Sometime it suggest updated document, some time it suggest old document. Currently, I need to use \_optimize?... to refresh FST. I don't know if there have anyway to refresh it better. I do update 5-10 per second, and I think it's fine with "max\_num\_segments=1". I really appreciate your help, thanks in advance. On Wednesday, April 23, 2014 5:52:16 AM UTC+7, Alexander Reelsen wrote: > Hey > > if you do regular updates on that index, you should not update down to one > single segment. Do you have an example, where the updates do not work as > expected? > > --Alex > > On Tue, Apr 22, 2014 at 10:51 AM, kidkid \<[zki...@gmail.com](mailto:zki...@gmail.com) \\>wrote: > > > Hey Alexander, > > Thanks for your reply. > > Currently I also manual do optimize by running optimize: > > host:9200/completion\_index/\_optimize?max\_num\_segments=1 > > I think it's a work around solution. I would like to make it better. > > > > Sometime, I also have problem with update payload, when I change payload, > > the completion is not change too. > > > > On Monday, April 21, 2014 7:26:13 PM UTC+7, Alexander Reelsen wrote: > > > > > Hey, > > > > > > the output is used to unify the search results, otherwise the input is > > > used. The payload itself is just meta information. > > > The main reason, why you see the suggestion twice is, that even though a > > > document is deleted and cannot be found anymore, the suggest data > > > structures are only cleaned up during merges/optimizations. Running > > > optimize should fix this. > > > > > > Makes sense? > > > > > > --Alex > > > > > > On Sun, Apr 13, 2014 at 12:49 PM, kidkid [zki...@gmail.com](mailto:zki...@gmail.com) wrote: > > > > > > > I have figure out the problem. > > > > The main problem is I have used the same output for all input then ES > > > > have been wrong in this case. > > > > > > > > I still trying to improve the performance. I am just test on 64Gb Ram > > > > server (32Gb for ES 1.0.1) 24 core. > > > > Have only 2 record but it took me 3ms to suggest. > > > > > > > > On Sunday, April 13, 2014 4:53:21 PM UTC+7, kidkid wrote: > > > > > > > > > There are something really strange. > > > > > I don't know whether anyone have worked with this such feature or it's > > > > > just not-stable feature. > > > > > If we do index same input, and different output,payload, then only one > > > > > result found. > > > > > > > > > > Do anyone tell me how could I fix it ? > > > > > > > > > > -- > > > > > You received this message because you are subscribed to the Google > > > > > Groups "elasticsearch" group. > > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > > > msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd% > > > > > [40googlegroups.com](http://40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > > . > > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > -- > > > You received this message because you are subscribed to the Google Groups > > > "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send an > > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/b3bf74ad-c729-4e78-9ebd-364c6c000819%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/b3bf74ad-c729-4e78-9ebd-364c6c000819%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/b3bf74ad-c729-4e78-9ebd-364c6c000819%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/b3bf74ad-c729-4e78-9ebd-364c6c000819%40googlegroups.com?utm_medium=email&utm_source=footer) > > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/c26b0711-8422-4741-a613-0c16c0b04c21%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/c26b0711-8422-4741-a613-0c16c0b04c21%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![zkidkid](https://avatars.discourse-cdn.com/v4/letter/z/fbc32d/32.png) [@zkidkid](https://discuss.elastic.co/u/zkidkid)\ **Post date:** [May 20, 2014, 8:24am UTC](https://discuss.elastic.co/t/autocompletion-suggester-duplicate-record-in-suggestion-return/16950/10 "2014-05-20T08:24:47Z")
Do anyone figure out how to fix this error ? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/34f04485-6006-4bdd-b45c-4cef3bea2e06%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/34f04485-6006-4bdd-b45c-4cef3bea2e06%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![alistairj](https://avatars.discourse-cdn.com/v4/letter/a/3ec8ea/32.png) [@alistairj](https://discuss.elastic.co/u/alistairj)\ **Post date:** [August 26, 2014, 11:59am UTC](https://discuss.elastic.co/t/autocompletion-suggester-duplicate-record-in-suggestion-return/16950/11 "2014-08-26T11:59:49Z")
Hi Alexander, If I may, I have a follow-up question to your response here. How does the completion suggester behave with fields such as payload and score when it is unifying the response based on output ?? Are scores increased based on this combination? if payloads are different, which ones are returned? Thanks for you help! Alistair On Monday, April 21, 2014 2:26:13 PM UTC+2, Alexander Reelsen wrote: > Hey, > > the output is used to unify the search results, otherwise the input is > used. The payload itself is just meta information. > The main reason, why you see the suggestion twice is, that even though a > document is deleted and cannot be found anymore, the suggest data > structures are only cleaned up during merges/optimizations. Running > optimize should fix this. > > Makes sense? > > --Alex > > On Sun, Apr 13, 2014 at 12:49 PM, kidkid \<[zki...@gmail.com](mailto:zki...@gmail.com) \\> > wrote: > > > I have figure out the problem. > > The main problem is I have used the same output for all input then ES > > have been wrong in this case. > > > > I still trying to improve the performance. I am just test on 64Gb Ram > > server (32Gb for ES 1.0.1) 24 core. > > Have only 2 record but it took me 3ms to suggest. > > > > On Sunday, April 13, 2014 4:53:21 PM UTC+7, kidkid wrote: > > > > > There are something really strange. > > > I don't know whether anyone have worked with this such feature or it's > > > just not-stable feature. > > > If we do index same input, and different output,payload, then only one > > > result found. > > > > > > Do anyone tell me how could I fix it ? > > > > > > -- > > > You received this message because you are subscribed to the Google Groups > > > "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send an > > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%40googlegroups.com) > > > [https://groups.google.com/d/msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%40googlegroups.com?utm_medium=email&utm_source=footer) > > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/13c35309-a55b-45d7-ba37-bd7bb44e6f20%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/13c35309-a55b-45d7-ba37-bd7bb44e6f20%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Tom\_4](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tom_4/32/1045_2.png) [@Tom\_4](https://discuss.elastic.co/u/Tom_4)\ **Post date:** [December 1, 2014, 10:21am UTC](https://discuss.elastic.co/t/autocompletion-suggester-duplicate-record-in-suggestion-return/16950/12 "2014-12-01T10:21:14Z")
Hi, i still have same problems with completion suggest duplicates of old and updates data using ES 1.4.x Only way that fixed it so far was using \_optimize?max\_num\_segments=1 which has performance and maybe other impacts i assume. Is there another solution than \_optimize?max\_num\_segments=1? Thx Tom Am Montag, 28. April 2014 11:21:49 UTC+2 schrieb Kaspars Sprogis: > Hi, > > I have exactly same problem. > I resolved duplicates and appearance of deleted items by running > "\_optimize?only\_expunge\_deletes=true" on daily basis. > > However i have still problem with updates. Even if data have been updated > for items in index, they still show old data when searching. > Only solution i found is running: "\_optimize?max\_num\_segments=1'". > > However i have quite a lot of updates and this get me worried, because of > Alexander's note: > > > if you do regular updates on that index, you should not update down to > > one single segment > > Alexander Reelsen, could you please advise, why we shouldn't do that or > does this create some permanent damage for optimization logic or does it > affects performance for search later? > What should we do instead? > > Thank you. > > On Wednesday, April 23, 2014 1:52:16 AM UTC+3, Alexander Reelsen wrote: > > > Hey > > > > if you do regular updates on that index, you should not update down to > > one single segment. Do you have an example, where the updates do not work > > as expected? > > > > --Alex > > > > On Tue, Apr 22, 2014 at 10:51 AM, kidkid [zki...@gmail.com](mailto:zki...@gmail.com) wrote: > > > > > Hey Alexander, > > > Thanks for your reply. > > > Currently I also manual do optimize by running optimize: > > > host:9200/completion\_index/\_optimize?max\_num\_segments=1 > > > I think it's a work around solution. I would like to make it better. > > > > > > Sometime, I also have problem with update payload, when I change > > > payload, the completion is not change too. > > > > > > On Monday, April 21, 2014 7:26:13 PM UTC+7, Alexander Reelsen wrote: > > > > > > > Hey, > > > > > > > > the output is used to unify the search results, otherwise the input is > > > > used. The payload itself is just meta information. > > > > The main reason, why you see the suggestion twice is, that even though > > > > a document is deleted and cannot be found anymore, the suggest data > > > > structures are only cleaned up during merges/optimizations. Running > > > > optimize should fix this. > > > > > > > > Makes sense? > > > > > > > > --Alex > > > > > > > > On Sun, Apr 13, 2014 at 12:49 PM, kidkid [zki...@gmail.com](mailto:zki...@gmail.com) wrote: > > > > > > > > > I have figure out the problem. > > > > > The main problem is I have used the same output for all input then ES > > > > > have been wrong in this case. > > > > > > > > > > I still trying to improve the performance. I am just test on 64Gb Ram > > > > > server (32Gb for ES 1.0.1) 24 core. > > > > > Have only 2 record but it took me 3ms to suggest. > > > > > > > > > > On Sunday, April 13, 2014 4:53:21 PM UTC+7, kidkid wrote: > > > > > > > > > > > There are something really strange. > > > > > > I don't know whether anyone have worked with this such feature or > > > > > > it's just not-stable feature. > > > > > > If we do index same input, and different output,payload, then only > > > > > > one result found. > > > > > > > > > > > > Do anyone tell me how could I fix it ? > > > > > > > > > > > > -- > > > > > > You received this message because you are subscribed to the Google > > > > > > Groups "elasticsearch" group. > > > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > > > > msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd% > > > > > > [40googlegroups.com](http://40googlegroups.com) > > > > > > [https://groups.google.com/d/msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > > > . > > > > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > To view this discussion on the web visit > > > > [https://groups.google.com/d/msgid/elasticsearch/b3bf74ad-c729-4e78-9ebd-364c6c000819%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/b3bf74ad-c729-4e78-9ebd-364c6c000819%40googlegroups.com) > > > > [https://groups.google.com/d/msgid/elasticsearch/b3bf74ad-c729-4e78-9ebd-364c6c000819%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/b3bf74ad-c729-4e78-9ebd-364c6c000819%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > . > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/bf7ccf72-d65c-4e1c-a6ec-b3a11c56f834%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/bf7ccf72-d65c-4e1c-a6ec-b3a11c56f834%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Xavier\_TROMP](https://avatars.discourse-cdn.com/v4/letter/x/13edae/32.png) [@Xavier\_TROMP](https://discuss.elastic.co/u/Xavier_TROMP)\ **Post date:** [March 20, 2015, 11:46pm UTC](https://discuss.elastic.co/t/autocompletion-suggester-duplicate-record-in-suggestion-return/16950/13 "2015-03-20T23:46:48Z")
Hello everyone, I have the same problem using ES 1.4.4. Did someone came up with a better solution ? Any help would be greatly appreciated. Thank you, Xavier On Monday, December 1, 2014 at 11:21:14 AM UTC+1, Tom wrote: > Hi, > > i still have same problems with completion suggest duplicates of old and > updates data using ES 1.4.x > Only way that fixed it so far was using \_optimize?max\_num\_segments=1 which > has performance and maybe other impacts i assume. > > Is there another solution than \_optimize?max\_num\_segments=1? > > Thx > Tom > > Am Montag, 28. April 2014 11:21:49 UTC+2 schrieb Kaspars Sprogis: > > > Hi, > > > > I have exactly same problem. > > I resolved duplicates and appearance of deleted items by running > > "\_optimize?only\_expunge\_deletes=true" on daily basis. > > > > However i have still problem with updates. Even if data have been updated > > for items in index, they still show old data when searching. > > Only solution i found is running: "\_optimize?max\_num\_segments=1'". > > > > However i have quite a lot of updates and this get me worried, because of > > Alexander's note: > > > > > if you do regular updates on that index, you should not update down to > > > one single segment > > > > Alexander Reelsen, could you please advise, why we shouldn't do that or > > does this create some permanent damage for optimization logic or does it > > affects performance for search later? > > What should we do instead? > > > > Thank you. > > > > On Wednesday, April 23, 2014 1:52:16 AM UTC+3, Alexander Reelsen wrote: > > > > > Hey > > > > > > if you do regular updates on that index, you should not update down to > > > one single segment. Do you have an example, where the updates do not work > > > as expected? > > > > > > --Alex > > > > > > On Tue, Apr 22, 2014 at 10:51 AM, kidkid [zki...@gmail.com](mailto:zki...@gmail.com) wrote: > > > > > > > Hey Alexander, > > > > Thanks for your reply. > > > > Currently I also manual do optimize by running optimize: > > > > host:9200/completion\_index/\_optimize?max\_num\_segments=1 > > > > I think it's a work around solution. I would like to make it better. > > > > > > > > Sometime, I also have problem with update payload, when I change > > > > payload, the completion is not change too. > > > > > > > > On Monday, April 21, 2014 7:26:13 PM UTC+7, Alexander Reelsen wrote: > > > > > > > > > Hey, > > > > > > > > > > the output is used to unify the search results, otherwise the input is > > > > > used. The payload itself is just meta information. > > > > > The main reason, why you see the suggestion twice is, that even though > > > > > a document is deleted and cannot be found anymore, the suggest data > > > > > structures are only cleaned up during merges/optimizations. Running > > > > > optimize should fix this. > > > > > > > > > > Makes sense? > > > > > > > > > > --Alex > > > > > > > > > > On Sun, Apr 13, 2014 at 12:49 PM, kidkid [zki...@gmail.com](mailto:zki...@gmail.com) wrote: > > > > > > > > > > > I have figure out the problem. > > > > > > The main problem is I have used the same output for all input then ES > > > > > > have been wrong in this case. > > > > > > > > > > > > I still trying to improve the performance. I am just test on 64Gb Ram > > > > > > server (32Gb for ES 1.0.1) 24 core. > > > > > > Have only 2 record but it took me 3ms to suggest. > > > > > > > > > > > > On Sunday, April 13, 2014 4:53:21 PM UTC+7, kidkid wrote: > > > > > > > > > > > > > There are something really strange. > > > > > > > I don't know whether anyone have worked with this such feature or > > > > > > > it's just not-stable feature. > > > > > > > If we do index same input, and different output,payload, then only > > > > > > > one result found. > > > > > > > > > > > > > > Do anyone tell me how could I fix it ? > > > > > > > > > > > > > > -- > > > > > > > You received this message because you are subscribed to the Google > > > > > > > Groups "elasticsearch" group. > > > > > > > To unsubscribe from this group and stop receiving emails from it, > > > > > > > send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > > > > > msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd% > > > > > > > [40googlegroups.com](http://40googlegroups.com) > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > > > > . > > > > > > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > > > -- > > > > > You received this message because you are subscribed to the Google > > > > > Groups "elasticsearch" group. > > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > > To view this discussion on the web visit > > > > > [https://groups.google.com/d/msgid/elasticsearch/b3bf74ad-c729-4e78-9ebd-364c6c000819%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/b3bf74ad-c729-4e78-9ebd-364c6c000819%40googlegroups.com) > > > > > [https://groups.google.com/d/msgid/elasticsearch/b3bf74ad-c729-4e78-9ebd-364c6c000819%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/b3bf74ad-c729-4e78-9ebd-364c6c000819%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > > . > > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/b06125c6-de2f-4d8f-906d-2b66d35207eb%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/b06125c6-de2f-4d8f-906d-2b66d35207eb%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Xavier\_TROMP](https://avatars.discourse-cdn.com/v4/letter/x/13edae/32.png) [@Xavier\_TROMP](https://discuss.elastic.co/u/Xavier_TROMP)\ **Post date:** [March 17, 2016, 11:15am UTC](https://discuss.elastic.co/t/autocompletion-suggester-duplicate-record-in-suggestion-return/16950/14 "2016-03-17T11:15:15Z")
Is this problem solved with the completion suggester v2 (with elasticsearch 2.2) ? If not optimizing an index with approximatly 300 writes/s to 1 segment fine ? I plan to upgrade to 2.2 but an official answer would be nice. Thank you ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 11:07pm UTC](https://discuss.elastic.co/t/autocompletion-suggester-duplicate-record-in-suggestion-return/16950/15 "2017-07-05T23:07:35Z")