Logstash Aggregation Plugin

We are getting multiple events come in ( because of multiple botnet calls, but it is the same SRC and DST) How do you use the aggregation plugin to combine it all into 1 . Timestamps are slightly different

	Time 	src      	dhost        	dst        	requestMethod        	request        	act        	outcome        	reason        	riskScore        	requestContext      
	June 2nd 2019, 1.2.3.4		62.212.33.98	62.212.33.98	CONNECT	62.212.33.98/	Blocked	403	Reputation block outbound request: botnet site	0	None
	June 2nd 2019, 1.2.3.4		62.212.33.98	62.212.33.98	CONNECT	62.212.33.98/	Blocked	403	Reputation block outbound request: botnet site	0	None
	June 2nd 2019, 1.2.3.4	   62.212.33.98	    62.212.33.98	CONNECT	62.212.33.98/	Blocked	403	Reputation block outbound request: botnet site	0	None

What do you want to aggregate?

"src" and "dst" and "request"

Those three fields have the same values in those three messages. It does not make any sense to aggregate them.

Thats the problem, We get multiple events from zscaler for the same thing that happened . Thats why im trying to condense it into 1 document

It sounds like you want to do de-duplication. If you are using an elasticsearch output then this blog has some ideas.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.