How to merge two document in logstash

Elastic Stack Logstash
1

Hi, I am using ELK to visualize the data in logs. Is it possible to merge two documents into one? As I know, each line from filebeat will make one doc output to elasticsearch(or with multi-lines input, then multi-lines will make one doc, but my situation is from different files). To be specific, I have one doc with fields A, B ; Another doc with fields A, C ;How to make new doc with fields A, B, C and remove previous two docs?

By the way, all I want to do is to create the visualization of B/C (B divided by C), so if there is any possible way to do this by Kibana/Elasticsearch is also okey.

2

Can you give more details about your use case?

How do the 2 documents from your example relate to each other?

Is there a unique id value for common to both documents and different to any other document pairs that you use for your calculations?

You may use aggregate filter to join documents before indexing, enrich them using the already present data from elasticsearch, configure the output with upsert using or constructing an id when indexing...

There are multiple approaches with their own pros and cons - the more detailed information you add, the better answers you will get :slight_smile:

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.