Log4j input with empty members

Elastic Stack Logstash
1

I'm trying to use Logstash to index my log files created by log4j. I'm using the following versions:

  • Logstash: 2.1.1
  • ES: 2.1.0
  • log4j: 1.2.17

I followed the recommendation found at http://stackoverflow.com/a/26227002 and use the SocketAppender in log4j configuration (and there are many more questions/articles on the net recommending the log4j input in Logstash and the SocketAppender in log4j)

The SocketAppender is defined as following:

<appender name="socket" class="org.apache.log4j.net.SocketAppender">
    <param name="RemoteHost" value="localhost" />
    <param name="Port" value="4560" />
    <param name="LocationInfo" value="true"/>
    <layout class="org.apache.log4j.PatternLayout">
        <param name="ConversionPattern" value="[%d{ISO8601}][%-5p][%-25c] %m%n" />
    </layout>
</appender>

I tried with the PatternLayout as described here https://www.elastic.co/blog/logging-elasticsearch-events-with-logstash-and-elasticsearch (although the pattern will be ignored according to http://stackoverflow.com/questions/11270504/how-to-use-pattern-layout-with-socketappender )

My pipe is defined as following:

input {
    log4j {
        mode => server
        host => "0.0.0.0"
        port => "4560"
        type => "ex"
    }
}
output {
    stdout { codec => rubydebug }
    elasticsearch { index => exlogs }
}

The events are indexed, and the output on stdout looks like the following:

{
        "message" => "dummy error",
       "@version" => "1",
     "@timestamp" => "2015-12-15T09:00:43.978Z",
      "timestamp" => 1450170043978,
           "path" => "com.ex.monitoring.jmxclient.JmxReader",
       "priority" => "ERROR",
    "logger_name" => "com.ex.monitoring.jmxclient.JmxReader",
         "thread" => "hugo",
          "class" => "?",
           "file" => "?:?",
         "method" => "?",
           "host" => "127.0.0.1:51808",
           "type" => "ex"
}

I'm wondering why "class", "file", "method" are empty all the time? Am I doing anything wrong?

2

These information are to be considered debug information because they are costly to produce,
and are only written to the appender when it is configured to do so with

<param name="LocationInfo" value="true" />

On the other hand I think the input should not write "garbage" that correspond to "not available", I have raise an issue about this in https://github.com/logstash-plugins/logstash-input-log4j/issues/25

3

Thanks for the reply.

I have this parameter configured in the appender (see the log4j configuration in my initial post). Any idea what I might be missing?

4

Oups, sorry to have overlooked that.
I used your appender config in a quick test, and the correct info is sent to Logstash. So i'm rather clueless.

Do you have any additional custom configuration for Log4j such as a custom implementation of Logger ? that could be a reason...

Or is there another socket appender configured without the LocationInfo enabled ??

5

Oh, it's b/c I use an async appender and this one is sending to the socket, now I directly send to socket w/o the async appender and I'm getting it.

Thanks for your support.