Copy multiple document

[moranmono (Moran)]

Hi
I am using nest client with my application, I am trying to find a way to create a copy of multiple documents and change their properties. So far I have only got to this solution - Bulk api, but this solution requires me to get all the documents to my client and only then update them, which will take a lot of time and memory since I need to update more than 20,000 records at a time.

Is there a way to perform all the work on the elastic server instead of getting all the documents and sending them back?

[warkolm (Mark Walkom)]

Welcome to our community!

Why are you copying the documents and not updating them?
What sort of updates are you doing?

[moranmono (Moran)]

This is my document

"_source": {
      "category_id": 10000137,
      "aspect_id": 1000119,
      "cms_id": "custom_stores-1490011980432-96306-94",
      "aspect_name": "Packaging",
      "aspect_uuid": "772b6513-c3cf-11e9-b164-c3925ec7a797",
      "category_name": "General",
      "category_uuid": "28413970-c3de-11e9-a2bc-95a493b58504",
      "cms_line": "<cms_hl1>Like</cms_hl1> it is wrapped in the luxury all of us deserve.<revuze_cms_hidden_context>rid=203697490</revuze_cms_hidden_context>",
      "insert_date": 1647043200000,
      "not_aspect": true,
      "quote": "Like it is wrapped in the luxury all of us deserve.",
      "review_id": "203697490",
    }

I have multiple documents that I want to relate to aspect_id. Sure I can change the aspect_id field to an array and have multiple entries, but this will make it difficult for me to filter the results later.
That is why I prefer to create a copy with different property values.

[warkolm (Mark Walkom)]

@moranmono

insert_date

It looks like this is potentially time based data?
If there is a timestamp attached to each of these documents, then I would treat it like other time based data and just add each document as an event stream. That means you could easily create the document in your app rather than having to collect it from Elasticsearch.

[moranmono (Moran)]

Hi, I can't create it in my app because the data is being inserted into elastic from a different source.