How to send Logs from Apache Spark ( HDInsight Cluster in Azure) to Elastic Search / Kibana


(Arvind Singh) #1

Our logs are at DFS and we can access them using ....
hadoop fs -cat wasbs://yourcontainer@youraccount.blob.core.windows.net/testDir/testFile
test file content

Now we want to configure filebeats here so that we can ship logs / or contents to logstash. How to configure filebeats here..I mean how to give command in filebeats so that it finally reaches to Kibana portal....


(Mark Walkom) #2

Please don't cross post - How to Configure FileBeat for Spark Headnode ( HDInsight Cluster in Azure)


(Mark Walkom) #3