Can ElasticSearch sort data that is stored as a linked list?

Shane_Claggett · May 20, 2014, 6:06pm

Hi all,

Is it possible for ES to sort records that are related to one another as a
linked list? For example, is there a query that would order the following
sample data?

[
{
"id":"572bc3df-f567-432e-8087-718b1517099a",
"previous":"",
"next":"3481ba2f-2bf8-4ea2-a187-311ed33d8f85"
},
{
"id":"3481ba2f-2bf8-4ea2-a187-311ed33d8f85",
"previous":"572bc3df-f567-432e-8087-718b1517099a",
"next":"dc01b22e-1663-4276-9ebe-5ecdb0d976ef"
},
{
"id":"dc01b22e-1663-4276-9ebe-5ecdb0d976ef",
"previous":"3481ba2f-2bf8-4ea2-a187-311ed33d8f85",
"next":"5281764d-7f84-4a8e-910d-124e003aabed"
},
{
"id":"5281764d-7f84-4a8e-910d-124e003aabed",
"previous":"dc01b22e-1663-4276-9ebe-5ecdb0d976ef",
"next":""
}
]

I've read the scoring and sorting documentation as well as googled around,
but haven't found anything that would allow me to say "put this item before
that item if 'this.id'=='that.previous'". Is such a thing possible with ES?

Thanks,
Shane

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8b58eb8b-7c01-456f-b8a8-7798844ce548%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

jpountz · May 21, 2014, 7:28am

Unfortunately, this is not possible.

On Tue, May 20, 2014 at 8:06 PM, Shane Claggett shaneclaggett@gmail.comwrote:

Hi all,

Is it possible for ES to sort records that are related to one another as a
linked list? For example, is there a query that would order the following
sample data?

[
{
"id":"572bc3df-f567-432e-8087-718b1517099a",
"previous":"",
"next":"3481ba2f-2bf8-4ea2-a187-311ed33d8f85"
},
{
"id":"3481ba2f-2bf8-4ea2-a187-311ed33d8f85",
"previous":"572bc3df-f567-432e-8087-718b1517099a",
"next":"dc01b22e-1663-4276-9ebe-5ecdb0d976ef"
},
{
"id":"dc01b22e-1663-4276-9ebe-5ecdb0d976ef",
"previous":"3481ba2f-2bf8-4ea2-a187-311ed33d8f85",
"next":"5281764d-7f84-4a8e-910d-124e003aabed"
},
{
"id":"5281764d-7f84-4a8e-910d-124e003aabed",
"previous":"dc01b22e-1663-4276-9ebe-5ecdb0d976ef",
"next":""
}
]

I've read the scoring and sorting documentation as well as googled around,
but haven't found anything that would allow me to say "put this item before
that item if 'this.id'=='that.previous'". Is such a thing possible with
ES?

Thanks,
Shane

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/8b58eb8b-7c01-456f-b8a8-7798844ce548%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/8b58eb8b-7c01-456f-b8a8-7798844ce548%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Adrien Grand

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j6wo7mqm3JYnUXXrBRv2OSpJJJL-2GQBVh3fRSjXCRYoQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Shane_Claggett · May 21, 2014, 2:44pm

Thanks Adrien, I had the suspicion that was the case, but I appreciate you
confirming it. On to plan B!

On Wed, May 21, 2014 at 12:28 AM, Adrien Grand <
adrien.grand@elasticsearch.com> wrote:

Unfortunately, this is not possible.

On Tue, May 20, 2014 at 8:06 PM, Shane Claggett shaneclaggett@gmail.comwrote:

Hi all,

Is it possible for ES to sort records that are related to one another as
a linked list? For example, is there a query that would order the following
sample data?

[
{
"id":"572bc3df-f567-432e-8087-718b1517099a",
"previous":"",
"next":"3481ba2f-2bf8-4ea2-a187-311ed33d8f85"
},
{
"id":"3481ba2f-2bf8-4ea2-a187-311ed33d8f85",
"previous":"572bc3df-f567-432e-8087-718b1517099a",
"next":"dc01b22e-1663-4276-9ebe-5ecdb0d976ef"
},
{
"id":"dc01b22e-1663-4276-9ebe-5ecdb0d976ef",
"previous":"3481ba2f-2bf8-4ea2-a187-311ed33d8f85",
"next":"5281764d-7f84-4a8e-910d-124e003aabed"
},
{
"id":"5281764d-7f84-4a8e-910d-124e003aabed",
"previous":"dc01b22e-1663-4276-9ebe-5ecdb0d976ef",
"next":""
}
]

I've read the scoring and sorting documentation as well as googled
around, but haven't found anything that would allow me to say "put this
item before that item if 'this.id'=='that.previous'". Is such a thing
possible with ES?

Thanks,
Shane

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/8b58eb8b-7c01-456f-b8a8-7798844ce548%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/8b58eb8b-7c01-456f-b8a8-7798844ce548%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Adrien Grand

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/11fVxjIHg7w/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j6wo7mqm3JYnUXXrBRv2OSpJJJL-2GQBVh3fRSjXCRYoQ%40mail.gmail.com https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j6wo7mqm3JYnUXXrBRv2OSpJJJL-2GQBVh3fRSjXCRYoQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAMyodgYwSA4g%2BKafi4vJEAxadTxCqBvtT3WtMccBdzjE4QbGFg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

dmitry_polushkin · August 21, 2014, 8:28pm

Hi Shane,

How did you solved this problem?

Regards,
Dmitry

Shane_Claggett · August 27, 2014, 5:35pm

Hi Dmitry,

I did indeed come up with a solution. Basically, I added a field to the
schema of type "double", and use that to sort the items. Linked list items
that are added to the end of the growing list are assigned values that are
sequential integers, e.g.:

Item A - 1.0
Item B - 2.0
Item C - 3.0

If an item is inserted into the linked list between A and B, it is assigned
a value midway:

Item A - 1.0
Item D - 1.5
Item B - 2.0
Item C - 3.0

Likewise if something is inserted at the head of the list:

Item E - 0.5
Item A - 1.0
Item D - 1.5
Item B - 2.0
Item C - 3.0

This is in addition to keeping the previous and next item references, so
the index can be regenerated as needed, e.g. if something gets corrupted,
which is entirely likely when writing something like this yourself.

This solution certainly has the feel of a hack, but it has worked well so
far. Double values allow for a fairly high degree of precision, so this
trick can be carried on for the insertion of many items. It's not as good
as native support in the system for a linked list structure, but adequate
for the practical needs of the application I’m working on.

Be aware that two threads inserting an item at the same place and same time
would lead to duplicate index values unless some sort of "fuzziness" is
included when calculating the index.

Cheers,
Shane

On Thu, Aug 21, 2014 at 4:28 PM, dmitry.polushkin <
dmitry.polushkin@gmail.com> wrote:

Hi Shane,

How did you solved this problem?

Regards,
Dmitry

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/Can-ElasticSearch-sort-data-that-is-stored-as-a-linked-list-tp4056166p4062249.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/11fVxjIHg7w/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1408652899659-4062249.post%40n3.nabble.com
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAMyodgbiuLXV827OFkEXsfRzsovSv6MYPCofxFd_jpc%3DgH9izQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.