AutoCompletion Suggester - Duplicate record in suggestion return


(Sang Dang) #1

Hi All,
I have problem with AutoCompletion Suggester.
When I index same word in record, AutoCompletion will return duplicate that
record,
Here is restful:

curl -X PUT 10.30.58.105:9200/completion
curl -X DELETE 10.30.58.105:9200/completion/song
curl -X PUT 10.30.58.105:9200/completion/song/_mapping -d '{
"song" : {
"properties" : {
"song-suggest" : { "type" : "completion",
"index_analyzer" : "simple",
"search_analyzer" : "simple",
"payloads" : true
}
}
}
}'
//insert duplicate word "Nothing"
curl -X PUT '10.30.58.105:9200/completion/song/1?refresh=true' -d
'{"song-suggest":{"input":["Nothing Nothing Else Matter","Nothing Else
Matter","Else Matter","Matter"],"payload":{"name":"Nothing Else Matter -
Metallica 1"},"weight":12}}'

curl -X POST '10.30.58.105:9200/completion/_suggest?pretty' -d '{
"song-suggest" : {
"text" : "Nothing",
"completion" : {
"field" : "song-suggest"
}
}
}'

The return value will:

{
"_shards" : {
"total" : 2,
"successful" : 2,
"failed" : 0
},
"song-suggest" : [ {
"text" : "Nothing",
"offset" : 0,
"length" : 7,
"options" : [ {
"text" : "Nothing Else Matter",
"score" : 1.0, "payload" : {"name":"Nothing Else Matter - Metallica
1"}
}, {
"text" : "Nothing Nothing Else Matter",
"score" : 1.0, "payload" : {"name":"Nothing Else Matter - Metallica
1"}
} ]
} ]
}

Although I only have one record.

Thanks.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e3c1d118-7b2f-4e03-a682-f901124c287a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Suggestion payload is not updated on document update
(Sang Dang) #2

Hi,
It's really fun because when I added output field it work like a charm.
Completion is built in FST, so how could "output" field affect the result ?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/7f98b542-e862-4654-bed7-a31c297d795d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Sang Dang) #3

There are something really strange.
I don't know whether anyone have worked with this such feature or it's just
not-stable feature.
If we do index same input, and different output,payload, then only one
result found.

Do anyone tell me how could I fix it ?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f5518627-f29d-4864-9b9f-003675e257ce%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Sang Dang) #4

I have figure out the problem.
The main problem is I have used the same output for all input then ES have
been wrong in this case.

I still trying to improve the performance. I am just test on 64Gb Ram
server (32Gb for ES 1.0.1) 24 core.
Have only 2 record but it took me 3ms to suggest.

On Sunday, April 13, 2014 4:53:21 PM UTC+7, kidkid wrote:

There are something really strange.
I don't know whether anyone have worked with this such feature or it's
just not-stable feature.
If we do index same input, and different output,payload, then only one
result found.

Do anyone tell me how could I fix it ?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Alexander Reelsen) #5

Hey,

the output is used to unify the search results, otherwise the input is
used. The payload itself is just meta information.
The main reason, why you see the suggestion twice is, that even though a
document is deleted and cannot be found anymore, the suggest data
structures are only cleaned up during merges/optimizations. Running
optimize should fix this.

Makes sense?

--Alex

On Sun, Apr 13, 2014 at 12:49 PM, kidkid zkidkid@gmail.com wrote:

I have figure out the problem.
The main problem is I have used the same output for all input then ES have
been wrong in this case.

I still trying to improve the performance. I am just test on 64Gb Ram
server (32Gb for ES 1.0.1) 24 core.
Have only 2 record but it took me 3ms to suggest.

On Sunday, April 13, 2014 4:53:21 PM UTC+7, kidkid wrote:

There are something really strange.
I don't know whether anyone have worked with this such feature or it's
just not-stable feature.
If we do index same input, and different output,payload, then only one
result found.

Do anyone tell me how could I fix it ?

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGCwEM-yu%3DzB0Zf4Ra%3DAJTz0uUkR4ASGeBDkkdFaDKYmP9VCTQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Sang Dang) #6

Hey Alexander,
Thanks for your reply.
Currently I also manual do optimize by running optimize:
host:9200/completion_index/_optimize?max_num_segments=1
I think it's a work around solution. I would like to make it better.

Sometime, I also have problem with update payload, when I change payload,
the completion is not change too.

On Monday, April 21, 2014 7:26:13 PM UTC+7, Alexander Reelsen wrote:

Hey,

the output is used to unify the search results, otherwise the input is
used. The payload itself is just meta information.
The main reason, why you see the suggestion twice is, that even though a
document is deleted and cannot be found anymore, the suggest data
structures are only cleaned up during merges/optimizations. Running
optimize should fix this.

Makes sense?

--Alex

On Sun, Apr 13, 2014 at 12:49 PM, kidkid <zki...@gmail.com <javascript:>>wrote:

I have figure out the problem.
The main problem is I have used the same output for all input then ES
have been wrong in this case.

I still trying to improve the performance. I am just test on 64Gb Ram
server (32Gb for ES 1.0.1) 24 core.
Have only 2 record but it took me 3ms to suggest.

On Sunday, April 13, 2014 4:53:21 PM UTC+7, kidkid wrote:

There are something really strange.
I don't know whether anyone have worked with this such feature or it's
just not-stable feature.
If we do index same input, and different output,payload, then only one
result found.

Do anyone tell me how could I fix it ?

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b3bf74ad-c729-4e78-9ebd-364c6c000819%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Alexander Reelsen) #7

Hey

if you do regular updates on that index, you should not update down to one
single segment. Do you have an example, where the updates do not work as
expected?

--Alex

On Tue, Apr 22, 2014 at 10:51 AM, kidkid zkidkid@gmail.com wrote:

Hey Alexander,
Thanks for your reply.
Currently I also manual do optimize by running optimize:
host:9200/completion_index/_optimize?max_num_segments=1
I think it's a work around solution. I would like to make it better.

Sometime, I also have problem with update payload, when I change payload,
the completion is not change too.

On Monday, April 21, 2014 7:26:13 PM UTC+7, Alexander Reelsen wrote:

Hey,

the output is used to unify the search results, otherwise the input is
used. The payload itself is just meta information.
The main reason, why you see the suggestion twice is, that even though a
document is deleted and cannot be found anymore, the suggest data
structures are only cleaned up during merges/optimizations. Running
optimize should fix this.

Makes sense?

--Alex

On Sun, Apr 13, 2014 at 12:49 PM, kidkid zki...@gmail.com wrote:

I have figure out the problem.
The main problem is I have used the same output for all input then ES
have been wrong in this case.

I still trying to improve the performance. I am just test on 64Gb Ram
server (32Gb for ES 1.0.1) 24 core.
Have only 2 record but it took me 3ms to suggest.

On Sunday, April 13, 2014 4:53:21 PM UTC+7, kidkid wrote:

There are something really strange.
I don't know whether anyone have worked with this such feature or it's
just not-stable feature.
If we do index same input, and different output,payload, then only one
result found.

Do anyone tell me how could I fix it ?

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%
40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/b3bf74ad-c729-4e78-9ebd-364c6c000819%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/b3bf74ad-c729-4e78-9ebd-364c6c000819%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGCwEM8RaaN69kcBQTTbkXfV7RL8fkR0Y8PAuK7jR-Jdma3VGw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Kaspars Sprogis) #8

Hi,

I have exactly same problem.
I resolved duplicates and appearance of deleted items by running
"_optimize?only_expunge_deletes=true" on daily basis.

However i have still problem with updates. Even if data have been updated
for items in index, they still show old data when searching.
Only solution i found is running: "_optimize?max_num_segments=1'".

However i have quite a lot of updates and this get me worried, because of
Alexander's note:

if you do regular updates on that index, you should not update down to one
single segment

Alexander Reelsen, could you please advise, why we shouldn't do that or
does this create some permanent damage for optimization logic or does it
affects performance for search later?
What should we do instead?

Thank you.

On Wednesday, April 23, 2014 1:52:16 AM UTC+3, Alexander Reelsen wrote:

Hey

if you do regular updates on that index, you should not update down to one
single segment. Do you have an example, where the updates do not work as
expected?

--Alex

On Tue, Apr 22, 2014 at 10:51 AM, kidkid <zki...@gmail.com <javascript:>>wrote:

Hey Alexander,
Thanks for your reply.
Currently I also manual do optimize by running optimize:
host:9200/completion_index/_optimize?max_num_segments=1
I think it's a work around solution. I would like to make it better.

Sometime, I also have problem with update payload, when I change payload,
the completion is not change too.

On Monday, April 21, 2014 7:26:13 PM UTC+7, Alexander Reelsen wrote:

Hey,

the output is used to unify the search results, otherwise the input is
used. The payload itself is just meta information.
The main reason, why you see the suggestion twice is, that even though a
document is deleted and cannot be found anymore, the suggest data
structures are only cleaned up during merges/optimizations. Running
optimize should fix this.

Makes sense?

--Alex

On Sun, Apr 13, 2014 at 12:49 PM, kidkid zki...@gmail.com wrote:

I have figure out the problem.
The main problem is I have used the same output for all input then ES
have been wrong in this case.

I still trying to improve the performance. I am just test on 64Gb Ram
server (32Gb for ES 1.0.1) 24 core.
Have only 2 record but it took me 3ms to suggest.

On Sunday, April 13, 2014 4:53:21 PM UTC+7, kidkid wrote:

There are something really strange.
I don't know whether anyone have worked with this such feature or it's
just not-stable feature.
If we do index same input, and different output,payload, then only one
result found.

Do anyone tell me how could I fix it ?

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%
40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/b3bf74ad-c729-4e78-9ebd-364c6c000819%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/b3bf74ad-c729-4e78-9ebd-364c6c000819%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/3b03ab59-e642-42d8-ab11-86f4589d8bec%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Sang Dang) #9

Hi Alexander Reelsen,
Just do at my example above. When you do update on payload or do reindex
(with same doc id, same input, different output or payload) then the
completion will work funny.
Sometime it suggest updated document, some time it suggest old document.

Currently, I need to use _optimize?... to refresh FST. I don't know if
there have anyway to refresh it better.

I do update 5-10 per second, and I think it's fine with
"max_num_segments=1".

I really appreciate your help, thanks in advance.

On Wednesday, April 23, 2014 5:52:16 AM UTC+7, Alexander Reelsen wrote:

Hey

if you do regular updates on that index, you should not update down to one
single segment. Do you have an example, where the updates do not work as
expected?

--Alex

On Tue, Apr 22, 2014 at 10:51 AM, kidkid <zki...@gmail.com <javascript:>>wrote:

Hey Alexander,
Thanks for your reply.
Currently I also manual do optimize by running optimize:
host:9200/completion_index/_optimize?max_num_segments=1
I think it's a work around solution. I would like to make it better.

Sometime, I also have problem with update payload, when I change payload,
the completion is not change too.

On Monday, April 21, 2014 7:26:13 PM UTC+7, Alexander Reelsen wrote:

Hey,

the output is used to unify the search results, otherwise the input is
used. The payload itself is just meta information.
The main reason, why you see the suggestion twice is, that even though a
document is deleted and cannot be found anymore, the suggest data
structures are only cleaned up during merges/optimizations. Running
optimize should fix this.

Makes sense?

--Alex

On Sun, Apr 13, 2014 at 12:49 PM, kidkid zki...@gmail.com wrote:

I have figure out the problem.
The main problem is I have used the same output for all input then ES
have been wrong in this case.

I still trying to improve the performance. I am just test on 64Gb Ram
server (32Gb for ES 1.0.1) 24 core.
Have only 2 record but it took me 3ms to suggest.

On Sunday, April 13, 2014 4:53:21 PM UTC+7, kidkid wrote:

There are something really strange.
I don't know whether anyone have worked with this such feature or it's
just not-stable feature.
If we do index same input, and different output,payload, then only one
result found.

Do anyone tell me how could I fix it ?

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%
40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/b3bf74ad-c729-4e78-9ebd-364c6c000819%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/b3bf74ad-c729-4e78-9ebd-364c6c000819%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/c26b0711-8422-4741-a613-0c16c0b04c21%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Sang Dang) #10

Do anyone figure out how to fix this error ?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/34f04485-6006-4bdd-b45c-4cef3bea2e06%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(alistairj) #11

Hi Alexander,

If I may, I have a follow-up question to your response here. How does the
completion suggester behave with fields such as payload and score when it
is unifying the response based on output ?? Are scores increased based on
this combination? if payloads are different, which ones are returned?

Thanks for you help!

Alistair

On Monday, April 21, 2014 2:26:13 PM UTC+2, Alexander Reelsen wrote:

Hey,

the output is used to unify the search results, otherwise the input is
used. The payload itself is just meta information.
The main reason, why you see the suggestion twice is, that even though a
document is deleted and cannot be found anymore, the suggest data
structures are only cleaned up during merges/optimizations. Running
optimize should fix this.

Makes sense?

--Alex

On Sun, Apr 13, 2014 at 12:49 PM, kidkid <zki...@gmail.com <javascript:>>
wrote:

I have figure out the problem.
The main problem is I have used the same output for all input then ES
have been wrong in this case.

I still trying to improve the performance. I am just test on 64Gb Ram
server (32Gb for ES 1.0.1) 24 core.
Have only 2 record but it took me 3ms to suggest.

On Sunday, April 13, 2014 4:53:21 PM UTC+7, kidkid wrote:

There are something really strange.
I don't know whether anyone have worked with this such feature or it's
just not-stable feature.
If we do index same input, and different output,payload, then only one
result found.

Do anyone tell me how could I fix it ?

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/13c35309-a55b-45d7-ba37-bd7bb44e6f20%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Tom-4) #12

Hi,

i still have same problems with completion suggest duplicates of old and
updates data using ES 1.4.x
Only way that fixed it so far was using _optimize?max_num_segments=1 which
has performance and maybe other impacts i assume.

Is there another solution than _optimize?max_num_segments=1?

Thx
Tom

Am Montag, 28. April 2014 11:21:49 UTC+2 schrieb Kaspars Sprogis:

Hi,

I have exactly same problem.
I resolved duplicates and appearance of deleted items by running
"_optimize?only_expunge_deletes=true" on daily basis.

However i have still problem with updates. Even if data have been updated
for items in index, they still show old data when searching.
Only solution i found is running: "_optimize?max_num_segments=1'".

However i have quite a lot of updates and this get me worried, because of
Alexander's note:

if you do regular updates on that index, you should not update down to
one single segment

Alexander Reelsen, could you please advise, why we shouldn't do that or
does this create some permanent damage for optimization logic or does it
affects performance for search later?
What should we do instead?

Thank you.

On Wednesday, April 23, 2014 1:52:16 AM UTC+3, Alexander Reelsen wrote:

Hey

if you do regular updates on that index, you should not update down to
one single segment. Do you have an example, where the updates do not work
as expected?

--Alex

On Tue, Apr 22, 2014 at 10:51 AM, kidkid zki...@gmail.com wrote:

Hey Alexander,
Thanks for your reply.
Currently I also manual do optimize by running optimize:
host:9200/completion_index/_optimize?max_num_segments=1
I think it's a work around solution. I would like to make it better.

Sometime, I also have problem with update payload, when I change
payload, the completion is not change too.

On Monday, April 21, 2014 7:26:13 PM UTC+7, Alexander Reelsen wrote:

Hey,

the output is used to unify the search results, otherwise the input is
used. The payload itself is just meta information.
The main reason, why you see the suggestion twice is, that even though
a document is deleted and cannot be found anymore, the suggest data
structures are only cleaned up during merges/optimizations. Running
optimize should fix this.

Makes sense?

--Alex

On Sun, Apr 13, 2014 at 12:49 PM, kidkid zki...@gmail.com wrote:

I have figure out the problem.
The main problem is I have used the same output for all input then ES
have been wrong in this case.

I still trying to improve the performance. I am just test on 64Gb Ram
server (32Gb for ES 1.0.1) 24 core.
Have only 2 record but it took me 3ms to suggest.

On Sunday, April 13, 2014 4:53:21 PM UTC+7, kidkid wrote:

There are something really strange.
I don't know whether anyone have worked with this such feature or
it's just not-stable feature.
If we do index same input, and different output,payload, then only
one result found.

Do anyone tell me how could I fix it ?

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/b3bf74ad-c729-4e78-9ebd-364c6c000819%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/b3bf74ad-c729-4e78-9ebd-364c6c000819%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/bf7ccf72-d65c-4e1c-a6ec-b3a11c56f834%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Xavier) #13

Hello everyone,

I have the same problem using ES 1.4.4.

Did someone came up with a better solution ?

Any help would be greatly appreciated.

Thank you,
Xavier

On Monday, December 1, 2014 at 11:21:14 AM UTC+1, Tom wrote:

Hi,

i still have same problems with completion suggest duplicates of old and
updates data using ES 1.4.x
Only way that fixed it so far was using _optimize?max_num_segments=1 which
has performance and maybe other impacts i assume.

Is there another solution than _optimize?max_num_segments=1?

Thx
Tom

Am Montag, 28. April 2014 11:21:49 UTC+2 schrieb Kaspars Sprogis:

Hi,

I have exactly same problem.
I resolved duplicates and appearance of deleted items by running
"_optimize?only_expunge_deletes=true" on daily basis.

However i have still problem with updates. Even if data have been updated
for items in index, they still show old data when searching.
Only solution i found is running: "_optimize?max_num_segments=1'".

However i have quite a lot of updates and this get me worried, because of
Alexander's note:

if you do regular updates on that index, you should not update down to
one single segment

Alexander Reelsen, could you please advise, why we shouldn't do that or
does this create some permanent damage for optimization logic or does it
affects performance for search later?
What should we do instead?

Thank you.

On Wednesday, April 23, 2014 1:52:16 AM UTC+3, Alexander Reelsen wrote:

Hey

if you do regular updates on that index, you should not update down to
one single segment. Do you have an example, where the updates do not work
as expected?

--Alex

On Tue, Apr 22, 2014 at 10:51 AM, kidkid zki...@gmail.com wrote:

Hey Alexander,
Thanks for your reply.
Currently I also manual do optimize by running optimize:
host:9200/completion_index/_optimize?max_num_segments=1
I think it's a work around solution. I would like to make it better.

Sometime, I also have problem with update payload, when I change
payload, the completion is not change too.

On Monday, April 21, 2014 7:26:13 PM UTC+7, Alexander Reelsen wrote:

Hey,

the output is used to unify the search results, otherwise the input is
used. The payload itself is just meta information.
The main reason, why you see the suggestion twice is, that even though
a document is deleted and cannot be found anymore, the suggest data
structures are only cleaned up during merges/optimizations. Running
optimize should fix this.

Makes sense?

--Alex

On Sun, Apr 13, 2014 at 12:49 PM, kidkid zki...@gmail.com wrote:

I have figure out the problem.
The main problem is I have used the same output for all input then ES
have been wrong in this case.

I still trying to improve the performance. I am just test on 64Gb Ram
server (32Gb for ES 1.0.1) 24 core.
Have only 2 record but it took me 3ms to suggest.

On Sunday, April 13, 2014 4:53:21 PM UTC+7, kidkid wrote:

There are something really strange.
I don't know whether anyone have worked with this such feature or
it's just not-stable feature.
If we do index same input, and different output,payload, then only
one result found.

Do anyone tell me how could I fix it ?

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/f6547a58-c002-4ff3-80c9-2052e1d14ddd%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/b3bf74ad-c729-4e78-9ebd-364c6c000819%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/b3bf74ad-c729-4e78-9ebd-364c6c000819%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b06125c6-de2f-4d8f-906d-2b66d35207eb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Xavier) #14

Is this problem solved with the completion suggester v2 (with elasticsearch 2.2) ?

If not optimizing an index with approximatly 300 writes/s to 1 segment fine ?

I plan to upgrade to 2.2 but an official answer would be nice.

Thank you


(system) #15