Merging 2 log lines into 1

Elastic Stack Elasticsearch
1

Hi,

I am looking at 2 indices that have 1 field in common and then different fields. Would it be possible to merge them into 1 log line using the similar field?

Picture1

Using the example above, both lines have the same value (24) in the Roll Number field, is it possible to merge them into 1 log line so that all the data comes as 1 and looks something like this:

@timestamp   Roll Number   address  name  age Gender
14:23:00.000     24        Riseley  Lenny  23  Other
2

Generally you do this as the data is being ingested into Elasticsearch. So how are you doing that now?

3

Hi @warkolm

Currently I have 2 separate indices (each line corresponds to a separate indices) and I am seeing if it is possible to get the current data, that is in Elasticsearch, to be combined like the example above.

As you say it is possible to do this whilst the data is being ingested, but is it at all possible to combine them into 1 line when the data has already been ingested?

I have seen you can create an index pattern that can include both of them, but it does not combine similar fields together and keeps the logs separate

4

This would be a join, which Elasticsearch does not support.

5

Thank you for the quick responses

6

There's a new feature you could try if this works fr you.

See lookup runtime fields but as it gives the "first" hit, it might produce weird results as you are searching within the same index.

1 Like
7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.