Increase/improve logging to debug slowdowns?

Elastic Stack Elasticsearch
1

I'm trying to track down delays we're seeing in our application. Our
monitoring indicates the application is sometimes held up for long periods
of time while talking to elasticsearch.

Can someone recommend logging settings that would help me get to the bottom
of what elasticsearch is doing at any given time?

I guess, at a minimum, I should be logging when elasticsearch is merging and
expunging? We currently have the following in logging.yml (which I thought
would do what I want):

rootLogger: INFO, console, file
logger:
  action: DEBUG
  com.amazonaws: WARN

appender:
  console:
    type: console
    layout:
      type: consolePattern
      conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"

  file:
    type: dailyRollingFile
    file: ${path.logs}/${cluster.name}.log
    datePattern: "'.'yyyy-MM-dd"
    layout:
      type: pattern
      conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"

Any suggestions appreciated. Thanks.

Cheers,
David.

2

There isn't logging that one can enabled for merging operations. In master, there are stats on merging happening though.

The two places that you would want to start with checking is long garbage collection and merging. Another one is to make sure elasticsearch has enough memory (check the node stats for field data cache evictions and how much memory it takes, as well as the filter cache.

-shay.banon
On Tuesday, March 8, 2011 at 1:39 AM, David Taylor wrote:

I'm trying to track down delays we're seeing in our application. Our monitoring indicates the application is sometimes held up for long periods of time while talking to elasticsearch.

Can someone recommend logging settings that would help me get to the bottom of what elasticsearch is doing at any given time?

I guess, at a minimum, I should be logging when elasticsearch is merging and expunging? We currently have the following in logging.yml (which I thought would do what I want):

rootLogger: INFO, console, file
logger:
action: DEBUG
com.amazonaws: WARN

appender:
console:
type: console
layout:
type: consolePattern
conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"

file:
type: dailyRollingFile
file: ${path.logs}/${cluster.name}.log
datePattern: "'.'yyyy-MM-dd"
layout:
type: pattern
conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"

Any suggestions appreciated. Thanks.

Cheers,
David.