Combine multiple document into one document with limited Fields ( merging of documents )

Elastic Stack Elasticsearch
1

Combine multiple documents into one document

I have a few documents, for example, 3 documents
Document - 1
{
...
test: "a"
...
}

Document - 2
{
...
test: "b"
...
}

Document - 3
{
...
test: "c"
...
}

===============Output I am looking for ==============
Document -
{
...
test: "a", "b","c"
...
}

This is like the merging of multiple documents into a single document ( could be in different index).

2

You will need to do that in your application. You can not format results like that in Elasticsearch.

3

Thank you, Christian for the reply. I need some information.
As per my understating,

  1. This can not be achieved using the KQL, DSL directly.
  2. This has to be done using the java rest API
4

You need to have a custom step to post process the response. It can as far as I know not be done in Kibana.

5

I got it. Thank you, Christian, for your quick reply.

6

You can use a transform for it. The group_by must be based on a common criteria, e.g. a common id.

For the combining step, you can use a scripted metric, e.g.:

"all_docs": {
  "scripted_metric": {
    "init_script": "state.docs = []",
    "map_script": "state.docs.add(new HashMap(params['_source']))",
    "combine_script": "return state.docs",
    "reduce_script": "def docs = []; for (s in states) {for (d in s) { docs.add(d);}}return docs"
  }
}

this would create an array with the original docs. If you want to merge them or merge only a specific field, you need to change the example accordingly, the docs contain some more ideas.

Hope that helps.

1 Like
7

Thank you for the information. We are able to achieve using Painless script. is there any other way as well?

8

I am not aware of another solution and I think the painless solution should perform well. Painless is compiled into java byte code and therefore comparable in terms of speed with a pure java implementation.

The only real downside I see is ease of use.

9

Thank you, Hendrik_Muhs

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.