Best practice for Log-Analyzing

Illmatic · November 6, 2018, 8:25am

I need to Analyze the Logs(Exceptions) from a Rest-Application. I have a Log-File with the logged Exceptions and was wondering, what would be best practice to Analyze this Logs and wich Components from the ELK-Stack I should use for that.

Thanks for the Help allready

jakelandis · November 6, 2018, 3:18pm

The first step is to get the data into Elasticsearch.

Take a look at Filebeat [1] (which installs on the same host as your Rest-Application). Pay particular attention to the multi-line support [2] since most exceptions are multi-line.

I would suggest to start with sending data directly to Elasticsearch [3]. If you need any pre-processing of the data prior to indexing, you have a few options [4][5][6].

If you have your own instance of Elasticsearch and Kibana already running, great! If not our cloud offering [7] is really great for getting starting (and production too !). Docker is also great for getting started (and production) [8], and using docker compose with Elasticsearch and Kibana [9] is great for a quick test environment.

Once you have the data in Elasticsearch and Kibana running to view the data, there is a lot available information on this topic via the interwebs. It's a wide topic so specific questions would be needed to properly advise your for your usecase.

Illmatic · November 21, 2018, 8:35am

Thank you for this detailed answer! You helped me alot!

system · December 19, 2018, 8:35am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.