Logstash Aggregation Plugin

Elastic Stack Logstash
#1

We are getting multiple events come in ( because of multiple botnet calls, but it is the same SRC and DST) How do you use the aggregation plugin to combine it all into 1 . Timestamps are slightly different

	Time 	src      	dhost        	dst        	requestMethod        	request        	act        	outcome        	reason        	riskScore        	requestContext      
	June 2nd 2019, 1.2.3.4		62.212.33.98	62.212.33.98	CONNECT	62.212.33.98/	Blocked	403	Reputation block outbound request: botnet site	0	None
	June 2nd 2019, 1.2.3.4		62.212.33.98	62.212.33.98	CONNECT	62.212.33.98/	Blocked	403	Reputation block outbound request: botnet site	0	None
	June 2nd 2019, 1.2.3.4	   62.212.33.98	    62.212.33.98	CONNECT	62.212.33.98/	Blocked	403	Reputation block outbound request: botnet site	0	None
#2

What do you want to aggregate?

#3

"src" and "dst" and "request"

#4

Those three fields have the same values in those three messages. It does not make any sense to aggregate them.

#5

Thats the problem, We get multiple events from zscaler for the same thing that happened . Thats why im trying to condense it into 1 document

#6

It sounds like you want to do de-duplication. If you are using an elasticsearch output then this blog has some ideas.

#7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.