I want to start pulling my Elasticsearch logs with kibana and put them in Elasticsearch.
elasticsearch log -> logstash -> kafka -> logstash -> elasticsearch (and Repeat)
Ok this is a recursive issue, that will assume would crash my cluster. So I figure I would send my Elasticsearch logs from Production to my QA Elastic-search cluster so that I can trouble shoot issues since I am dealing with over 30 nodes
elasticsearch prod logs -> logstash -> qa kafka -> qa logstash -> QA Elasticsearch
So here are my questions:
- Anyone got some good Logstash INPUT and filters they would care to share to parse the logs?
- Any advice on good stuff to make visualizations/dashboards I should use
- Any GOTCHA's that you think I will encounter?
Or is there a better way , I am on 2.4 and about to upgrade to 5.3 so I would take solutions for either platform.