Elastic script to update document based on documents present in other index

Elastic Stack Elasticsearch
1

Hi Team,

I want to update document based on other index document.

I am trying something like below but I am getting "illegal_argument_exception" error. I am wondering is it possible or I am missing something?

POST test-transaction/_update/2
{
"scripted_upsert": true,
"script": {
"source": """
def eventDoc = ctx.client.get({
index: 'test-event',
id: '3'
});
def eventStatus = eventDoc['_source']['eventStatus'];

  ctx._source.status = eventStatus;
""",
"lang": "painless"

}
}

I tried below query too but getting same error-

POST test-transaction/_update/2
{
"scripted_upsert": true,
"script": {
"source": """
// Fetch the document from the other index
def txnDoc = ctx._source;
def status = txnDoc['status'];
def evtDoc = ctx._index['test-event'].get('3');

""",
"lang": "painless"

}
}

2

A script only has access to the current document and can not lookup documents in other indices. The only way you may be able to do this would be through an ingest pipeline with an enrich processor.

3

Thank you @Christian_Dahlqvist

I am able to achieve above problem statement via enrich documents.
Just need one more help currently I am having below setup -

Policy
PUT /_enrich/policy/event-policy
{
"match": {
"indices": "test-event",
"match_field": "correlationId",
"enrich_fields": ["eventStatus"],
"query": {
"match": {
"eventStatus": "Delivered"
}
}
}
}

Pipeline:
PUT /_ingest/pipeline/event_lookup
{
"processors" : [
{
"enrich" : {
"description": "Add 'eventStatus' data based on 'correlation'",
"policy_name": "event-policy",
"field" : "correlationId",
"target_field": "status",
"max_matches": "1",
"override": true
}
}
]
}

Incoming document:
POST test-transaction/_doc/11?pipeline=event_lookup
{
"status": "Initiate",
"payload": "data",
"correlationId": "123"
}

And after applying pipeline incoming document will enrich something like below:
"_source" : {
"payload" : "data",
"correlationId" : "123",
"status" : {
"correlationId" : "123",
"eventStatus" : "Delivered"
}
}

But I want something like below:
"_source" : {
"payload" : "data",
"correlationId" : "123",
"status" : "Delivered"
}

Please let me know how I can achieve this?

4

Am I correctly understanding that you want an exect term match on the correlationId but only consider documents that have the eventStatus field set to "Delivered"?

I do not think the API allows this (am not sure though), but you might want to add an alias with a filter clause on top of your indices and see if you can run the enrich policy based on this instead of directly linking to the indices. Have never done this so am not sure it will work.

5

Thanks @Christian_Dahlqvist m able to achieve that, I edited my questions. Please provide your suggestion on that.

6

Never mind, I found the solution via set processor.
Thanks a lot for all your help.