Is index rolling possible with parent-child modeling?

Elastic Stack Elasticsearch
1

Hello, I am working on parent-child modeling.

We are testing whether time series-based index rolling is possible.

Scenario

  1. Members are stored in index 202404 (assuming they joined in Apr 2024)
  2. Membership access information is stored in index 202405 (assuming access in May 2024)
  3. Check whether the indexes 202404 and 202405 are inquired by a multi-tenancy search.
PUT member-202404
{
  "settings": {
    "refresh_interval": "5s", 
    "number_of_shards": 3,
    "number_of_replicas": 1
  },
  "mappings": {
    "properties": {
      "site_code": {
        "type": "keyword"
      },
      "member_code": {
        "type": "keyword"
      },
      "connected_at": {
        "type": "date"
      },
      "join_fields": {
        "type": "join",
        "relations": {
          "member": "connections"
        }
      }
    }
  }
}

PUT member-202405
{
  "settings": {
    "refresh_interval": "5s", 
    "number_of_shards": 3,
    "number_of_replicas": 1
  },
  "mappings": {
    "properties": {
      "site_code": {
        "type": "keyword"
      },
      "member_code": {
        "type": "keyword"
      },
      "connected_at": {
        "type": "date"
      },
      "join_fields": {
        "type": "join",
        "relations": {
          "member": "connections"
        }
      }
    }
  }
}

PUT member-202404/_doc/member_aa?routing=site_a
{
  "site_code": "site_a",
  "member_code": "member_aa",
  "join_fields": "member"
}


PUT member-202405/_doc/member_aa-2024-05-01?routing=site_a
{
  "site_code": "site_a",
  "member_code": "member_aa",
  "connected_at": "2024-05-01",
   "join_fields": {
    "name": "connections",
    "parent": "member_aa"
  }
}

GET member-2024*/_search
{
  "query": {
    "has_child": {
      "type": "connections",
      "query": {
        "match_all": {}
      }
    }
  }
}

{
  "took" : 13,
  "timed_out" : false,
  "_shards" : {
    "total" : 6,
    "successful" : 6,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 0,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [ ]
  }
}

The search fails even if routing_id is specified.

Second, I can't find it if I try to run it again with 1 shard on the same node.

GET member-2024*/_search_shards


"shards" : [
  [
    {
      "state" : "STARTED",
      "primary" : true,
      "node" : "xY__PrtKS1ap5pGbxGRjXw",
      "relocating_node" : null,
      "shard" : 0,
      "index" : "member-202404",
      "allocation_id" : {
        "id" : "rz-RkYMjQn65b-L0aH0lIw"
      }
    }
  ],
  [
    {
      "state" : "STARTED",
      "primary" : true,
      "node" : "xY__PrtKS1ap5pGbxGRjXw",
      "relocating_node" : null,
      "shard" : 0,
      "index" : "member-202405",
      "allocation_id" : {
        "id" : "wYPv63ATRG6DsHSfyNap1g"
      }
    }
  ]
]

The shard is the same, but only allocation_id is different.
Is it because of this?

I'd like to know why.

Thank you

2

Parent-child relationships require that all related documents reside in the same shard (same shard in the same index), so it is generally not possible to use this with time-based indices where related data may be spread out over time. Your two sample documents are in different indices, which is not supported.

1 Like
3

Thank you. :grinning: