Elasticsearch logging

Elastic Stack Elasticsearch
#1

Hey All,

When reviewing the logs generated by elasticsearch (1.4.x), a single log
message is being split across multiple lines?

Is this configurable within logging.yml? My undstanding is that
elasticsearch is powered by log4j however, I'm not familiar with this
product, nor can I find any decent documentation on the logging.yml file.

Thanks all.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/337ffff1-bb65-41fc-986a-bffc3c473bb7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

#2

For reference here is the configuration we are currently using:

file:

type: rollingFile
file: ${path.logs}/${cluster.name}.log
maxBackupIndex: 1
maxFileSize: "100MB"
layout:
  type: pattern
  conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"

On Friday, January 2, 2015 12:20:32 PM UTC-6, Jesse Redl wrote:

Hey All,

When reviewing the logs generated by elasticsearch (1.4.x), a single log
message is being split across multiple lines?

Is this configurable within logging.yml? My undstanding is that
elasticsearch is powered by log4j however, I'm not familiar with this
product, nor can I find any decent documentation on the logging.yml file.

Thanks all.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e56ec7a0-ab0f-4152-8a67-52cd8c407bb9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

#3

Logging.yml is a funky wrapper around log4j.properties style log4j
configuration so that is why you don't see as much documentation on it.

Do you see log lines smashed together and cut apart randomly? That'd be a
bug.

Its customary for logs to be single lines except for stack traces which
contain multiple lines but all after the first start indented.
Elasticsearch inserts line breaks when it logs large things like thr
cluster state to make scanning the logs visually easier.

You could play with logging.yml to reformat the logs (probably possible to
make them json or something) or you could assume new log events start when
the first part of the new line is the date stamp.

Nik
On Jan 2, 2015 1:20 PM, "Jesse Redl" jredl@vendasta.com wrote:

Hey All,

When reviewing the logs generated by elasticsearch (1.4.x), a single log
message is being split across multiple lines?

Is this configurable within logging.yml? My undstanding is that
elasticsearch is powered by log4j however, I'm not familiar with this
product, nor can I find any decent documentation on the logging.yml file.

Thanks all.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/337ffff1-bb65-41fc-986a-bffc3c473bb7%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/337ffff1-bb65-41fc-986a-bffc3c473bb7%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPmjWd0srkXorAkY2vBf7aLGu-_wZ0dnG5LKL%2Bcwc3thhm85sw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

#4

On Friday, January 02, 2015 at 19:20 CET,
Jesse Redl jredl@vendasta.com wrote:

When reviewing the logs generated by elasticsearch (1.4.x), a single
log message is being split across multiple lines?

Yes, that's normal for Java logs.

Is this configurable within logging.yml? My undstanding is that
elasticsearch is powered by log4j however, I'm not familiar with
this product, nor can I find any decent documentation on the
logging.yml file.

I suspect the reason you're asking is that you're considering
ingesting the Elasticsearch logs with Logstash, in which case
you should look into the JSONEventLayoutV1 Log4j layout
(https://github.com/logstash/log4j-jsonevent-layout) to get the
logs in JSON format. You should be able to drop that jar file
along with its dependencies (net.minidev:json-smart:1.1.1 and
commons-lang:commons-lang:2.6) into the Elasticsearch lib directory
(typically /usr/share/elasticsearch/lib) and adjust logging.yml
to use that layout. See also
https://github.com/elasticsearch/elasticsearch/issues/8786.

--
Magnus Bäck | Software Engineer, Development Tools
magnus.back@sonymobile.com | Sony Mobile Communications

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/20150105095131.GA12719%40seldlx20533.corpusers.net.
For more options, visit https://groups.google.com/d/optout.

#5