# Using Logstash to analyse log4j log files

**URL:** https://discuss.elastic.co/t/using-logstash-to-analyse-log4j-log-files/44914
**Category:** Logstash
**Created:** [March 20, 2016, 1:50am UTC](https://discuss.elastic.co/t/using-logstash-to-analyse-log4j-log-files/44914 "2016-03-20T01:50:47Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![Jemli\_Fathi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jemli_fathi/32/10714_2.png) [@Jemli\_Fathi](https://discuss.elastic.co/u/Jemli_Fathi)
#### Post date: [March 20, 2016, 1:50am UTC](https://discuss.elastic.co/t/using-logstash-to-analyse-log4j-log-files/44914/1 "2016-03-20T01:50:47Z")

</div>

Hi,

Can you give me a basic example on how to configure logstash to handle log4j files as input including how to give an entire directory to look into.

Thank you.

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [March 20, 2016, 4:15pm UTC](https://discuss.elastic.co/t/using-logstash-to-analyse-log4j-log-files/44914/2 "2016-03-20T16:15:57Z")

</div>

Log files produced by Log4j can have just about any format so specific advice is next to impossible. Use a file input to read the files and a grok filter to parse them. The Logstash documentation contains examples of how to read httpd log files and the same general principles apply in your case. I suppose log messages can span over multiple lines (like e.g. Java stacktraces) so you'll probably want to use a multiline codec, but you can add that later.

Common pitfalls when dealing with a directory full of (presumably old) files:

- By default recent versions of Logstash ignores files older than 24 hours. Use the file input's `ignore_older` option to change this.
- Make sure you understand sincedb and the file input's `start_position` option if you want Logstash to read existing files from the beginning.

---

<div class="post-metadata">

### Author: ![Jemli\_Fathi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jemli_fathi/32/10714_2.png) [@Jemli\_Fathi](https://discuss.elastic.co/u/Jemli_Fathi)
#### Post date: [March 20, 2016, 7:59pm UTC](https://discuss.elastic.co/t/using-logstash-to-analyse-log4j-log-files/44914/3 "2016-03-20T19:59:08Z")

</div>

Thank you, it seems to be useful.

---

<div class="post-metadata">

### Author: ![CraigFoote](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/craigfoote/32/4571_2.png) [@CraigFoote](https://discuss.elastic.co/u/CraigFoote)
#### Post date: [March 21, 2016, 6:44pm UTC](https://discuss.elastic.co/t/using-logstash-to-analyse-log4j-log-files/44914/4 "2016-03-21T18:44:36Z")

</div>

Another option is to use the log4j input. Define a socket appender in your log4j properties file for your logstash server and an open port. The input automatically creates fields based on log4J.LoggingEvent fields.

---

<div class="post-metadata">

### Author: ![Jemli\_Fathi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jemli_fathi/32/10714_2.png) [@Jemli\_Fathi](https://discuss.elastic.co/u/Jemli_Fathi)
#### Post date: [March 21, 2016, 8:39pm UTC](https://discuss.elastic.co/t/using-logstash-to-analyse-log4j-log-files/44914/5 "2016-03-21T20:39:48Z")

</div>

Hi Craig,

Thank you, but can you give me a simple basic example? It will be very useful.

Thank you in advance.

---

<div class="post-metadata">

### Author: ![CraigFoote](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/craigfoote/32/4571_2.png) [@CraigFoote](https://discuss.elastic.co/u/CraigFoote)
#### Post date: [March 22, 2016, 2:28pm UTC](https://discuss.elastic.co/t/using-logstash-to-analyse-log4j-log-files/44914/6 "2016-03-22T14:28:49Z")

</div>

On the logstash side, the log4j input is pretty simple:

```
input{
    log4j{
        port => ####
    }
}

```

The server was already set up to do log4j logging so it was just a matter of adding a log4J SocketAppender:

```
#add the socketappender by name to the rootlogger
log4j.rootlogger=DEBUG,file,logstash

#configure socketappender
log4j.appender.logstash=org.apache.log4j.net.SocketAppender
log4j.appender.logstash.Port=####
log4j.appender.logstash.RemoteHost=[your logstash server hostname]
log4j.appender.logstash.ReconnectionDelay=10000

```

I believe the log4j.LoggingEvent object is serialized and sent to logstash where it's fields are used to create JSON fields, i.e. message, level, etc. No need for a filter, just output to elasticsearch.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 5:05am UTC](https://discuss.elastic.co/t/using-logstash-to-analyse-log4j-log-files/44914/7 "2017-07-06T05:05:53Z")

</div>


