Index refresh and search&scroll


(Joan Balagueró) #1

Hello,

I have an index with these settings:

   properties: {
     ( . . . )

     "creationTime" : { "type" : "long", "index" : "not_analyzed" },

  settings: {

    ( . . . )
    "number_of_replicas" : "0",
    "number_of_shards" : "1",
    "refresh_interval" : "10000ms",

Now I'm performing a search&scroll against this index using the 'creationTime'. Our search starts at 02-12-2015 18:02:58, takes about 200ms and retrieves 416.000 documents.

Now I'm printing the oldest and newest 'creationTime' from these 416.0000 documents:
[min, max] = [ 02-12-2015 17:47:13, 02-12-2015 18:02:55 ]

So, how is it possible that this search has found a document with creationTime = 18:02:55 if the search started at 18:02:58 and the refresh interval is 10 seconds? Shouldn't it find only documents with creationTime less than 18:02:48 (18:02:58 minus 10 seconds)?

Thanks.


(Magnus Bäck) #2

A 10 second refresh rate means that an index refresh takes place every ten seconds, not that a document is included in a refresh after ten seconds. Consequently, an updated document is included in a refresh after at most ten seconds. On average it'll take half that time.


(system) #3