Elasticsearch AsyncAppender configuration


(Mohamed Amine ABDESSEMED) #1

We've set up elasticsearch logging as described here logging elasticsearch
events with logstash
http://www.elasticsearch.org/blog/logging-elasticsearch-events-with-logstash-and-elasticsearch/ and
we want to use an AsyncAppender instead of using a SocketAppender directly
in order to avoid freeze times while logging but only xml log4j conf file
format supports AsyncAppender.

Is there another way to achieve this since elasticsearch doesn't use xml
conf files?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/095b157e-f2f4-4385-8205-dd9ff41f5f7e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


How can i configure log4j to use AsyncAppender instead SocketAppender?
(Jörg Prante) #2

It's easy, just remove log4j jar and config/logging.yml, then add
log4j2.xml, log4j-api, log4j-core, log4j-slf4j-impl and slf4j-api jars to
the lib folder. Then you can set up async appenders with log4j2, which are
18x more performant than log4j 1.x appenders.

Jörg

On Tue, Feb 17, 2015 at 5:42 PM, Mohamed Amine ABDESSEMED <
amineini@gmail.com> wrote:

We've set up elasticsearch logging as described here logging
elasticsearch events with logstash
http://www.elasticsearch.org/blog/logging-elasticsearch-events-with-logstash-and-elasticsearch/ and
we want to use an AsyncAppender instead of using a SocketAppender directly
in order to avoid freeze times while logging but only xml log4j conf file
format supports AsyncAppender.

Is there another way to achieve this since elasticsearch doesn't use xml
conf files?

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/095b157e-f2f4-4385-8205-dd9ff41f5f7e%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/095b157e-f2f4-4385-8205-dd9ff41f5f7e%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoG2WCNunGdWx5VwytYUv-1TCWNgae%3DQixSwx9014HWpzw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Mohamed Amine ABDESSEMED) #3

I can't see how removing the log4 jar and logging.yml will allow
elasticsearch to init log4j2 using the xml conf file since the allowed
suffixes are 'yml, yaml, json , properties' (
https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/common/logging/log4j/LogConfigurator.java#L48)
?

Amine

On Wednesday, February 18, 2015 at 9:12:15 AM UTC+1, Jörg Prante wrote:

It's easy, just remove log4j jar and config/logging.yml, then add
log4j2.xml, log4j-api, log4j-core, log4j-slf4j-impl and slf4j-api jars to
the lib folder. Then you can set up async appenders with log4j2, which are
18x more performant than log4j 1.x appenders.

Jörg

On Tue, Feb 17, 2015 at 5:42 PM, Mohamed Amine ABDESSEMED <
amin...@gmail.com <javascript:>> wrote:

We've set up elasticsearch logging as described here logging
elasticsearch events with logstash
http://www.elasticsearch.org/blog/logging-elasticsearch-events-with-logstash-and-elasticsearch/ and
we want to use an AsyncAppender instead of using a SocketAppender directly
in order to avoid freeze times while logging but only xml log4j conf file
format supports AsyncAppender.

Is there another way to achieve this since elasticsearch doesn't use xml
conf files?

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/095b157e-f2f4-4385-8205-dd9ff41f5f7e%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/095b157e-f2f4-4385-8205-dd9ff41f5f7e%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/032b5dc8-fd5b-4714-948d-7fe74a63ee65%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Jörg Prante) #4

If you remove the log4j jar, the ES logging is initialized by a logging
facade called SLF4J, and this passes initialization to the underlying
logging framework, e.g. log4j2

https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/common/logging/ESLoggerFactory.java#L40

Jörg

On Wed, Feb 18, 2015 at 4:28 PM, Mohamed Amine ABDESSEMED <
amineini@gmail.com> wrote:

I can't see how removing the log4 jar and logging.yml will allow
elasticsearch to init log4j2 using the xml conf file since the allowed
suffixes are 'yml, yaml, json , properties' (
https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/common/logging/log4j/LogConfigurator.java#L48)
?

Amine

On Wednesday, February 18, 2015 at 9:12:15 AM UTC+1, Jörg Prante wrote:

It's easy, just remove log4j jar and config/logging.yml, then add
log4j2.xml, log4j-api, log4j-core, log4j-slf4j-impl and slf4j-api jars to
the lib folder. Then you can set up async appenders with log4j2, which are
18x more performant than log4j 1.x appenders.

Jörg

On Tue, Feb 17, 2015 at 5:42 PM, Mohamed Amine ABDESSEMED <
amin...@gmail.com> wrote:

We've set up elasticsearch logging as described here logging
elasticsearch events with logstash
http://www.elasticsearch.org/blog/logging-elasticsearch-events-with-logstash-and-elasticsearch/ and
we want to use an AsyncAppender instead of using a SocketAppender directly
in order to avoid freeze times while logging but only xml log4j conf file
format supports AsyncAppender.

Is there another way to achieve this since elasticsearch doesn't use xml
conf files?

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/095b157e-f2f4-4385-8205-dd9ff41f5f7e%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/095b157e-f2f4-4385-8205-dd9ff41f5f7e%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/032b5dc8-fd5b-4714-948d-7fe74a63ee65%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/032b5dc8-fd5b-4714-948d-7fe74a63ee65%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE0M%3DY%2BSYXpJPLBzUKu6GGvpNOj4iOUhPX-TU-1XHS7LA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Cactusbone) #5

Hello ! Sorry about reviving this topic, but i'm trying to use asyncappender too and so i followed the remarks here.
I seem to have log4j2 working, but can't use
${path.logs}/${cluster.name}_${node.name}

I'm not sure how to get those variables, any hint ?


(system) #6