Sorting by rounded date


(Karol Gwaj) #1

hi,

can elasticsearch sort by rounded date ?
i want to sort by document creation date but rounded to 1 day, something
like that:

"sort": [

      "CreationDate*/1d*"
  
]

solr supports this type of sorting, so i expect elasticsearch is supporting
it too, but i cant find any documentation for that

i tried scripted sorting but it is much slower, for example sorting 30m
documents:

"sort": [
{
"_script" :
{
"script" : "doc['CreationDate'].value",
"type" : "number",
"order": "desc"
}
}
]

vs

"sort": [
{
"CreationDate": {
"order": "desc"
}
}
]

first one (script) is like 50 times slower (i guess because it is loading
CreationDate from _source)

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/cbb2fb50-3ccb-4ed3-8de2-95280666241f%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Alexander Reelsen) #2

Hey,

the date field does not get loaded from the source, but from a memory
structure called fielddata. On certain occasions this fielddata structure
needs to be removed and then recreated. If you fire a query after a removal
and the structure has not yet been recreated, you will get into performance
issues. You can use the warmer API to prevent exactly this behaviour and
ensure that the fielddata structure is loaded for your date field.

In addition you still do have 30m calculations if you sort by the above
script. You would gain far more performance, if you could involve filters
before or maybe take a look at rescoring (not sure if it fits your
use-case, but you should take a look)

Hope this helps..

--Alex

On Fri, Feb 7, 2014 at 7:40 PM, Karol Gwaj karol@gwaj.me wrote:

hi,

can elasticsearch sort by rounded date ?
i want to sort by document creation date but rounded to 1 day, something
like that:

"sort": [

      "CreationDate*/1d*"

]

solr supports this type of sorting, so i expect elasticsearch is
supporting it too, but i cant find any documentation for that

i tried scripted sorting but it is much slower, for example sorting 30m
documents:

"sort": [
{
"_script" :
{
"script" : "doc['CreationDate'].value",
"type" : "number",
"order": "desc"
}
}
]

vs

"sort": [
{
"CreationDate": {
"order": "desc"
}
}
]

first one (script) is like 50 times slower (i guess because it is loading
CreationDate from _source)

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/cbb2fb50-3ccb-4ed3-8de2-95280666241f%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGCwEM-_FDzDM5JP62LTHEv%3D7M%3DZRFEzmOxCY0CZcRWjQev3gA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(system) #3