Is this possible with Aggregate plugin or Ruby? Any ideas?

Elastic Stack Logstash
1

I am trying to combine records from two (or more) sources.

Here is the example:

Source A contains: doc_id, RecA
Source B contains: doc_id, RecB

I want to aggregate them to create

"doc_id": 44342
"RecA": "hello"
"RecB": "I am fine"

Here is my approach:

filter {
aggregate {
    task_id => "%{doc_id}"
    code =>
    "
     event.get['doc_id'] = map('doc_id')
     "
   push_previous_map_as_event => true
   timeout => 3
}
}

The bigger issue with the aggregate function here would be how to define that it has to look for that DOC_ID in the other source. Because as the sources are different, they will not come in one-after-the-other. Any ideas?

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.