Elasticsearch add new field in index by using update by query plugin

Elastic Stack Elasticsearch
1

Hi All,

I want to add a new field (with value) to all documents which is already indexed.

I guess we can possible to do it using bulk update_by_query plugin.

We are using Elasticsearch 1.7.3 version in DEV server and I installed update_by_plugin of version 2.6.0 and added dependency property in pom.xml & setting script.disable_dynamic: false added in elasticsearch yml file

Is the version update_by_plugin 2.6.0 is correct?

Below is the link I followed,

I here provided the sample data,

POST es_item/item/15781272
{
"ITEM_ID": 15781272,
"ITEM_CODE": "15781272",
"ITEM_DSCR": null,
"ITEM_SPECIFICITY_REF_ID": 184,
"ITEM_TYPE": "STANDARD ITEM",
"MOD_CHR_VAL_ID": 44632242,
"MOD_DSCR": "CANADIAN TRANSITION STANDARD MODULE",
"SG_CHR_VAL_ID": 44632240,
"PG_CHR_VAL_ID": 44632241
"RELATIONSHIP": [
{
"REL_TYP_REF_ID": -999,
"REL_TYPE": "NO RELATIONSHIP",
"REL_CTGRY": "NIL"
}
],
"ITEM_MISUSED_GTIN_FLG": "N",
"CRT_DTTM": "2012-04-03 00:00:00",
"UPD_DTTM": "2016-02-03 15:18:44",
"ICV": {
"C21472": 45465553
}
}

I want to add the new field "GLBL_CTGRY_ID":456 in the existing index next to the ICV part.

Is it possible to do in Elasticsearch Update by query plugin?

Please provide your feedback and it would be very helpful.

Thanks,
Ganeshbabu R

2

Since you are changing every document and not just a subset it may make more sense to just reindex into a new index. An update in elasticsearch is essentially a delete and an insert and the deleted documents aren't physically deleted until background housekeeping tasks clear those marked as deleted away.

It might be more efficient to reindex the content into a new index following the guidelines here: https://www.elastic.co/guide/en/elasticsearch/guide/current/reindex.html

3

Hi @Mark_Harwood

Thanks for the clarifications.

As I checked in the ES documentation using logstash we can reindex the data to the new index at that time can we add the new fields to the document with the value?

Is it possible in logstash?

Let us know your suggestions

Thanks,
Ganeshbabu R

4

I'm not a Logstash expert I'm afraid but it sounds like the sort of thing that should be possible. I suggest asking in the Logstash forum.

Cheers
Mark

5

Okay @Mark_Harwood

Hi @magnusbaeck/@dadoonet

Could you please share you thoughts on this request?

It would be very helpful.

Thanks
Ganeshbabu R