Extract nested documents to new index(non-nestify)

Elastic Stack Elasticsearch
1

Hello!

I'm wondering I can extract nested documents for the new index.

For example,
if there is an index of company which also containing products data as a nested field, i want to make a new product index from the former index.

from

{
  "mappings": {
    "properties": {
      "company_no": {
        "type": "keyword"
      },
      // ...
      "products" : {
        "type" : "nested",
        "properties" : {
          "product_price" : {
            "type" : "double"
          },
          "product_name" : {
            "type" : "keyword"
          }
        }
      }
    }
  }
}

to

{
  "mappings": {
    "properties": {
      "company_no": {
        "type": "keyword"
      },
      "product_price": {
        "type": "double"
      },
      "product_name": {
        "type": "keyword"
      }
    }
  }
}

I've tried to use reindex API with script, but it seems to overwrite when there are multiple nested objects(products in this case).

POST /_reindex
{
  "source": {
    "index": "old_company",
    "_source": ["company_no", "products"],
    "query": {
      "nested": {
      "path": "products",
      "query": {
        "exists": {
          "field": "products"
        }
      }
    }
    }
  },
  "dest": {
    "index": "new_company"
  },
  "script": {
    "source": """
      for (product in ctx._source.products) {
        def new_doc = [:];
        new_doc.company_no = ctx._source.company_no;
        new_doc.product_price = product.product_price;
        new_doc.product_name = product.product_name;
        ctx._id = UUID.randomUUID().toString(); 
        ctx._source = new_doc;
      }
    """
  }
}

The reason why I doing this is I want to visualize the chart for each product per company's price distribution.
And actually I'm not accessable to the origin data from database, so that I want to do something like this in Kibana DevTools.

Is there anything I can do?

  • reindex nested documents without missing in Kibana
  • Or visualize the nested documents in Kibana
  • Or.. any suggestions..?