Scroll with script sorting return endless data

ES 7.4.0
Centos 7

ref github issue

I would like to do a scroll operation for some data with the sorting of the abs value of the diff between publishTime VS. current system time.

In my opinion, the scroll might be:

  1. find out and make a snapshot for the query conditions
  2. cache them by the sort formula
  3. return the data paged by size

but for this query, each time i query for scroll page by scroll id, es result has these 2 results:

  1. data with messed up sequence
  2. return the same data list

I think this issue should be caused by System.currentTimeMillis()? I don't know, maybe ?

{
    "query": {
"some filters"
},
    "sort":[
        {
            "_script":{
                "script":{
                    "inline":"Math.abs(doc['publishTime'] - System.currentTimeMillis())",
                    "lang":"painless"
                },
                "type":"number",
                "order":"desc"
            }
        }
    ]
}

I suspect that the full result set is not cached but rather the offsets into the result set. As you are using a parameter in the script that varies over time this probably gets re-evaluated. As you are returning the full result set, why not either just sort on publishTime or pass the system time in as a parameter defined at the start of the query instead?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.