i have streaming data coming from kafka , and using spark structured streaming i do some filteration .
I want to send streaming data to elasticsearch. how can i dump this aggregated data to elastic search. i have used output mode complete and update too.
without aggregation my streaming data is dumping to elastic search with output mode append.it works fine .
but i want to send the aggregated data of streaming batch to elastic search .