Did some trial and error and Looks like the below fixes the problem.
df
.writeStream()
.outputMode("append")
.trigger(Trigger.ProcessingTime(1000))
.format("org.elasticsearch.spark.sql")
.option("checkpointLocation", "/tmp")
.start("test/hello");