We have ELK setup with filebeat, logstash and elasticsearch and kibana. I need aggregate request and response at the logstash.
I have configured pipeline configuration as below. now log aggregation working without any issue if i use single worker for pipeline. if i use multiple workers aggregation is not happening. any solution to use multiple workers and log aggregation?
if [transaction] == "request" {
aggregate {
task_id => "%{id}"
code => "
map['method'] = event.get('method')
map['request'] = event.get('request')
map['user'] = event.get('user')
map['application'] = event.get('application')"
map_action => "create"
}
drop {}#drop the request before persisting, to save indexing space in elasticsearch server
}
if [message] =~ "TRANSACTION:response" {
aggregate {
task_id => "%{id}"
code => "
event.set('method', map['method'])
event.set('response', map['response'])
event.set('user', map['user'])
event.set('application', map['application'])"
map_action => "update"
}
}