How to see full logs

[osykora]

Hello,

when I check service logs in Kibana, I can see some errors with a truncated stack trace. I wonder whether there's a place where I could see those errors with full stack trace so that I can fix the issues easily.

For example, there's this record in "service-logs-2019.03.01":
[2019-03-01 14:01:58,147][WARN ][no.found.runner.allocation.elasticsearch.snapshot.Snapshotter] Unexpected http response received during snapshot iteration: [Unable to ensure repository [found-snapshots]: [500 Internal Server Error]]: [HttpResponse(500 Internal Server Error,HttpEntity(application/json; charset=UTF-8,{"error":{"root_cause":[{"type":"repository_verification_exception","reason":"[found-snapshots] path [snapshots/eacde3e30c2c40baab9cf7c7a429ba6a] is not accessible on master node"}],"type":"repository_verification_exception","reason":"[found-snapshots] path [snapshots/eacde3e30c2c40baab9cf7c7a429ba6a] is not accessible on master node","caused_by":{"type":"i_o_exception","reason":"Unable to upload object [snapshots/eacde3e30c2c40baab9cf7c7a429ba6a/tests-EkjxNmiASziJ8gnnMZHUYA/master.dat] using a ...)

ending with "using a ...)". Can I do anything to view the full stack trace (to configure something, catching it in the related docker container, etc.)?

[rockybean]

You can see detailed logs in "cluster-logs-*" filtering dedicated cluster id.

[osykora]

Thanks @rockybean for pointing me to logs that I didn't know about. Unfortunatelly these logs don't contain more information. Actually I don't see the exception there at all.

[rockybean]

You could turn on debug log for your es cluster like below.

PUT _cluster/settings
{
  "transient": {
    "logger._root": "DEBUG"
  }
}

After that, you should see more useful logs.

[osykora]

Cool! This works like a charm. I can see full stack trace now. Thanks @rockybean

[rockybean]

You're welcome~