Help setting up logging

Elastic Stack Elasticsearch
1

ES is creating the log files upon startup but they are empty? I switched
every log level to DEBUG and it started pouring more log into
elasticsearch.log still, no query or indexing is logged.

-rw-r--r-- 1 elasticsearch elasticsearch 0 Mar 21 00:54
elasticsearch_index_indexing_slowlog.log
-rw-r--r-- 1 elasticsearch elasticsearch 0 Mar 21 00:54
elasticsearch_index_search_slowlog.log
-rw-r--r-- 1 elasticsearch elasticsearch 83910 Mar 21 01:00
elasticsearch.log

Heres my logging.yml

you can override this using by setting a system property, for example

-Des.logger.level=DEBUG
es.logger.level: DEBUG
rootLogger: ${es.logger.level}, console, file
logger:

log action execution errors for easier debugging

action: DEBUG

reduce the logging for aws, too much is logged under the default INFO

com.amazonaws: DEBUG

gateway

gateway: DEBUG
index.gateway: DEBUG

peer shard recovery

indices.recovery: DEBUG

discovery

discovery: DEBUG

index.search.slowlog: DEBUG, index_search_slow_log_file
index.indexing.slowlog: DEBUG, index_indexing_slow_log_file

additivity:
index.search.slowlog: true
index.indexing.slowlog: true
appender:
console:
type: console
layout:
type: consolePattern
conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"

file:
type: dailyRollingFile
file: ${path.logs}/${cluster.name}.log
datePattern: "'.'yyyy-MM-dd"
layout:
type: pattern
conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"

index_search_slow_log_file:
type: dailyRollingFile
file: ${path.logs}/${cluster.name}_index_search_slowlog.log
datePattern: "'.'yyyy-MM-dd"
layout:
type: pattern
conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"

index_indexing_slow_log_file:
type: dailyRollingFile
file: ${path.logs}/${cluster.name}_index_indexing_slowlog.log
datePattern: "'.'yyyy-MM-dd"
layout:
type: pattern
conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/39a035f9-752c-47a5-9a5c-61d4aeb643ee%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

The logging configuration specifies how and what to log, but it does not
specify when or what actually constitutes a slow query/index. Not all
queries/index requests are logged, just the slow ones. You need to define
the threshold in the main elasticsearch.yml config file.

--
Ivan

On Thu, Mar 20, 2014 at 6:05 PM, Raphael Miranda
raphaelmiranda@gmail.comwrote:

ES is creating the log files upon startup but they are empty? I switched
every log level to DEBUG and it started pouring more log into
elasticsearch.log still, no query or indexing is logged.

-rw-r--r-- 1 elasticsearch elasticsearch 0 Mar 21 00:54
elasticsearch_index_indexing_slowlog.log
-rw-r--r-- 1 elasticsearch elasticsearch 0 Mar 21 00:54
elasticsearch_index_search_slowlog.log
-rw-r--r-- 1 elasticsearch elasticsearch 83910 Mar 21 01:00
elasticsearch.log

Heres my logging.yml

you can override this using by setting a system property, for example

-Des.logger.level=DEBUG
es.logger.level: DEBUG
rootLogger: ${es.logger.level}, console, file
logger:

log action execution errors for easier debugging

action: DEBUG

reduce the logging for aws, too much is logged under the default INFO

com.amazonaws: DEBUG

gateway

gateway: DEBUG
index.gateway: DEBUG

peer shard recovery

indices.recovery: DEBUG

discovery

discovery: DEBUG

index.search.slowlog: DEBUG, index_search_slow_log_file
index.indexing.slowlog: DEBUG, index_indexing_slow_log_file

additivity:
index.search.slowlog: true
index.indexing.slowlog: true
appender:
console:
type: console
layout:
type: consolePattern
conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"

file:
type: dailyRollingFile
file: ${path.logs}/${cluster.name}.log
datePattern: "'.'yyyy-MM-dd"
layout:
type: pattern
conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"

index_search_slow_log_file:
type: dailyRollingFile
file: ${path.logs}/${cluster.name}_index_search_slowlog.log
datePattern: "'.'yyyy-MM-dd"
layout:
type: pattern
conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"

index_indexing_slow_log_file:
type: dailyRollingFile
file: ${path.logs}/${cluster.name}_index_indexing_slowlog.log
datePattern: "'.'yyyy-MM-dd"
layout:
type: pattern
conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/39a035f9-752c-47a5-9a5c-61d4aeb643ee%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/39a035f9-752c-47a5-9a5c-61d4aeb643ee%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBwy_yL_wif20GNtk3z_B%3Djt2%2BQCxxjQ-CCPh4WX9mqdw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

Thank you very much.

I figured the name slowlog meant it was verbose and not directives to log
actions that surpass a given threshold. I lowered the config to 1ms and now
I can see the logs.

On Thursday, 20 March 2014 22:56:36 UTC-3, Ivan Brusic wrote:

The logging configuration specifies how and what to log, but it does not
specify when or what actually constitutes a slow query/index. Not all
queries/index requests are logged, just the slow ones. You need to define
the threshold in the main elasticsearch.yml config file.

Elasticsearch Platform — Find real-time answers at scale | Elastic

--
Ivan

On Thu, Mar 20, 2014 at 6:05 PM, Raphael Miranda <raphael...@gmail.com<javascript:>

wrote:

ES is creating the log files upon startup but they are empty? I switched
every log level to DEBUG and it started pouring more log into
elasticsearch.log still, no query or indexing is logged.

-rw-r--r-- 1 elasticsearch elasticsearch 0 Mar 21 00:54
elasticsearch_index_indexing_slowlog.log
-rw-r--r-- 1 elasticsearch elasticsearch 0 Mar 21 00:54
elasticsearch_index_search_slowlog.log
-rw-r--r-- 1 elasticsearch elasticsearch 83910 Mar 21 01:00
elasticsearch.log

Heres my logging.yml

you can override this using by setting a system property, for example

-Des.logger.level=DEBUG
es.logger.level: DEBUG
rootLogger: ${es.logger.level}, console, file
logger:

log action execution errors for easier debugging

action: DEBUG

reduce the logging for aws, too much is logged under the default INFO

com.amazonaws: DEBUG

gateway

gateway: DEBUG
index.gateway: DEBUG

peer shard recovery

indices.recovery: DEBUG

discovery

discovery: DEBUG

index.search.slowlog: DEBUG, index_search_slow_log_file
index.indexing.slowlog: DEBUG, index_indexing_slow_log_file

additivity:
index.search.slowlog: true
index.indexing.slowlog: true
appender:
console:
type: console
layout:
type: consolePattern
conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"

file:
type: dailyRollingFile
file: ${path.logs}/${cluster.name}.log
datePattern: "'.'yyyy-MM-dd"
layout:
type: pattern
conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"

index_search_slow_log_file:
type: dailyRollingFile
file: ${path.logs}/${cluster.name}_index_search_slowlog.log
datePattern: "'.'yyyy-MM-dd"
layout:
type: pattern
conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"

index_indexing_slow_log_file:
type: dailyRollingFile
file: ${path.logs}/${cluster.name}_index_indexing_slowlog.log
datePattern: "'.'yyyy-MM-dd"
layout:
type: pattern
conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/39a035f9-752c-47a5-9a5c-61d4aeb643ee%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/39a035f9-752c-47a5-9a5c-61d4aeb643ee%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/41599209-093b-40d9-890b-90071d4c17e8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.