Up to date way to ingest logs from Java application using log4j2

Elastic Stack Logstash
#1

Hi, I've a Spring Boot 2.3.1 application that produces some logs. I'd like to ingest them into my ElasticSearch stack (I'm hosted on Elastic cloud).
What's the better way to do that? I saw the logstash log4j plugin is deprecated.

I saw some alternatives like https://github.com/rfoltyns/log4j2-elasticsearch but I'm not sure is the recomended way. Another approach is to use LogTash: https://medium.com/@d.lopez.j/easily-connecting-your-spring-boot-applications-to-the-elastic-stack-with-log4j2-1809e81c6a2e

BTW, I'm also using APM in my server application.

Thanks

#2

Hey guys, any hint about this? Thanks

#3

GELF might be an option. See this blog.

#4

Thanks for the hint. I was looking something integrated in my application. In fact it's distributed automatically on AWS EC2 and the machine is a pretty standard linux machine. I should see if I can change the machine template and install the software + configuration. It's a bit tricky.

Using a Java library would be much easier.

Thanks!

#5

Hi Danielle,

It's all about the trade-offs.

Logstash is a mature product that offers vast variety of plugins, log processing options, delivery strategies, Elasticsearch API integrations and - most of all - elastic.co support and offloading log delivery to other processes. It's just the safest bet.

Alternatives like log4j2-elasticsearch (thanks for mentioning) offer a simple bulk delivery with a few failover options and just a few API integrations: Index template, ILM policy (coming in 1.5) and - most of all - all the work is done within the same process as your business logic, which in some scenarios might put your application at risk.

But it's very easy to extend, customize and work with. Control of the output is much closer to the application/developer/business and you usually need to just refurbish current log4j2.xml configuration. Also, CPU and memory footprint is very small (especially when using log4j2-elasticsearch-hc module) even at very high workload.

If you have any trouble setting it up just raise an issue.
If you have any features you'd like to have implemented just raise an issue.
If you'd like to contribute, send me an email or just raise a PR.