How to dump streaming aggregate data to elastic search


(sheryl) #1

i have streaming data coming from kafka , and using spark structured streaming i do some filteration .

val df=streamingData.selectExpr("*").groupBy("date","location","name").agg((count(("name"))

I want to send streaming data to elasticsearch. how can i dump this aggregated data to elastic search. i have used output mode complete and update too.
without aggregation my streaming data is dumping to elastic search with output mode append.it works fine .
but i want to send the aggregated data of streaming batch to elastic search .


(system) #2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.