Combine multiple document into one document with limited Fields ( merging of documents )

Elasticsearch
#1

Combine multiple documents into one document

I have a few documents, for example, 3 documents
Document - 1
{
...
test: "a"
...
}

Document - 2
{
...
test: "b"
...
}

Document - 3
{
...
test: "c"
...
}

===============Output I am looking for ==============
Document -
{
...
test: "a", "b","c"
...
}

This is like the merging of multiple documents into a single document ( could be in different index).

#2

You will need to do that in your application. You can not format results like that in Elasticsearch.

#3

Thank you, Christian for the reply. I need some information.
As per my understating,

  1. This can not be achieved using the KQL, DSL directly.
  2. This has to be done using the java rest API
#4

You need to have a custom step to post process the response. It can as far as I know not be done in Kibana.

#5

I got it. Thank you, Christian, for your quick reply.

#6

You can use a transform for it. The group_by must be based on a common criteria, e.g. a common id.

For the combining step, you can use a scripted metric, e.g.:

"all_docs": {
  "scripted_metric": {
    "init_script": "state.docs = []",
    "map_script": "state.docs.add(new HashMap(params['_source']))",
    "combine_script": "return state.docs",
    "reduce_script": "def docs = []; for (s in states) {for (d in s) { docs.add(d);}}return docs"
  }
}

this would create an array with the original docs. If you want to merge them or merge only a specific field, you need to change the example accordingly, the docs contain some more ideas.

Hope that helps.

1 Like
How to join 2 index using 2 common values
#7

Thank you for the information. We are able to achieve using Painless script. is there any other way as well?

#8

I am not aware of another solution and I think the painless solution should perform well. Painless is compiled into java byte code and therefore comparable in terms of speed with a pure java implementation.

The only real downside I see is ease of use.

#9

Thank you, Hendrik_Muhs