How to find elasticsearch index store size based on the timestamp

[r.ganeshbabu (ganeshbabu)]

Hi All,

I have an index in elasticsearch and I want to find the each day number of documents and index size using es query.

Below is the query I tried for finding the no of docs for each day,

curl -XPOST "http://localhost:9201/awsflowlog_129/_search?size=0&pretty=true" -H 'Content-Type: application/json' -d' {"aggs":{"time":{"date_histogram":{"field":"timestamp","interval" :"day"}}}}'

below is the response of the query,

{
  "took" : 264,
  "timed_out" : false,
  "_shards" : {
    "total" : 4,
    "successful" : 4,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 8801057,
    "max_score" : 0.0,
    "hits" : [ ]
  },
  "aggregations" : {
    "time" : {
      "buckets" : [
        {
          "key_as_string" : "2018-08-07 00:00:00.000",
          "key" : 1533600000000,
          "doc_count" : 2539192
        },
        {
          "key_as_string" : "2018-08-08 00:00:00.000",
          "key" : 1533686400000,
          "doc_count" : 6261865
        }
      ]
    }
  }
}

Similarly I used the below ES query to get index store size & document count,

curl -XGET "http://localhost:9201/awsflowlog_129/_stats/store,docs?pretty=true"

and below is the response of the query,

{
"_shards" : {
"total" : 4,
"successful" : 4,
"failed" : 0
},
"_all" : {
"primaries" : {
"docs" : {
"count" : 8801057,
"deleted" : 0
},
"store" : {
"size_in_bytes" : 2042580596,
"throttle_time_in_millis" : 0
}
},
"total" : {
"docs" : {
"count" : 8801057,
"deleted" : 0
},
"store" : {
"size_in_bytes" : 2042580596,
"throttle_time_in_millis" : 0
}
}
},
"indices" : {
"awsflowlog_129" : {
"primaries" : {
"docs" : {
"count" : 8801057,
"deleted" : 0
},
"store" : {
"size_in_bytes" : 2042580596,
"throttle_time_in_millis" : 0
}
},
"total" : {
"docs" : {
"count" : 8801057,
"deleted" : 0
},
"store" : {
"size_in_bytes" : 2042580596,
"throttle_time_in_millis" : 0
}
}
}
}
}

I want to combine these two queries and get the result of total index size on each day with number of documents.

Correct me If I am doing anything wrong with the approach to find index size on daily basis,

Please let me know your thoughts and it would be very helpful.

Thanks,
Ganeshbabu R