Elastic search: Two level nested mapping, estimate total number of documents


(Nishant Kumar) #1

I have mapping as:

"properties": {
  "parents": {
    "type": "nested",
    "properties": {
      "parent_id": {
        "type": "integer",
        "doc_values": false
      },
      "child": {
        "type": "nested",
        "properties": {
          "is_deleted": {
            "type": "boolean",
            "doc_values": false
          },
          "child_id": {
            "type": "integer",
            "doc_values": false
          },
          "timestamp": {
            "type": "long",
            "doc_values": false
          }
        }
      }
    }
  },
  "other_ID": {
    "type": "string",
    "index": "not_analyzed",
    "doc_values": false
  }
}

Each parent will have unique children i.e. there is no common child among parents.

I have assigned 5 shards (My Elastic search cluster has 7 node) to this index but not sure if that will be sufficient.

Each Elasticsearch shard is a Lucene index. There is a maximum number of documents you can have in a single Lucene index. As of LUCENE-5843, the limit is 2,147,483,519 (= Integer.MAX_VALUE - 128) documents. You can monitor shard sizes using the _cat/shards api.

Assuming there will be say x unique _ID and each _IDs will have y parents and each parent will have z child. what will be total document count in my index?

To safeguard against ill-defined mappings the number of nested fields that can be defined per index has been limited to 50

Does this means that i can't have more than 50 parents in a _ID?


(system) #2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.