Cat indices docs.count is way above actual document count

Elastic Stack Elasticsearch
1

Hello.

I would like to get help with number of documents.

I have a indice which contains about 30M docs. cat indices shows as below.

curl -XGET 10.0.1.5:9200/_cat/indices/sonested?v
health status index    uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   sonested 3ZkXcwtjSIOcdlZJWpKgcg   1   0   29421692            0      4.9gb          4.9gb

However, if I execute match_all query , hits.total only returns 11203027.

cat query.json.2
{
 "size": 0,
 "query": {
    "match_all": {}
 }
}
curl -XGET 10.0.1.5:9200/sonested/_search?pretty -d @query.json.2
{
  "took" : 328,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  },
  "hits" : {
    "total" : 11203027,
    "max_score" : 0.0,
    "hits" : [ ]
  }
}

counts api also returns 11M

curl -XGET '10.0.1.5:9200/sonested/_count?pretty'
{
  "count" : 11203027,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  }
}

Why is hits.total different from docs.count of cat api?

Version I am using is 5.3.0

[2017-04-11T16:41:35,777][INFO ][o.e.n.Node ] [uniform-2] version[5.3.0], pid[17602], build[3adb13b/2017-03-23T03:31:50.652Z], OS[Linux/3.10.0-

2

Hey,

is it possible that you are using nested documents? Those are counted as part of the indices.countas they are separate lucene documents.

--Alex

1 Like
3

See also https://www.elastic.co/guide/en/elasticsearch/reference/5.3/cat-indices.html#cat-indices

" We can tell quickly how many shards make up an index, the number of docs at the Lucene level, including hidden docs (e.g., from nested types), deleted docs, primary store size, and total store size (all shards including replicas). All these exposed metrics come directly from Lucene APIs."

4

@spinscale

Your right. I did not notice that . Thanks!

{
  "sonested": {
    "mappings": {
      "question": {
        "dynamic": "strict",
        "properties": {
          "answer_count": {
            "type": "integer"
          },
          "answers": {
            "type": "nested",
            "properties": {
              "date": {
                "type": "date"
              },
              "user": {
                "type": "keyword"
              }
            }
          },
5

@johtani

Thanks Ohtani-san! I missed that line...

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.