Log queries from elasticsearch output

[SpaceRaven (Roman)]

Hi, Is there any way to log somehow queries from elasticsearch output plugin for es updates? i mean i want to inspect json queris which will go es cluster for update/delete/insert.

Thanks

[ylasri (Yassine LASRI)]

You can enable that at the level of logging

logger.elasticsearchoutput.name = logstash.outputs.elasticsearch
logger.elasticsearchoutput.level = debug

May be this link can help

[SpaceRaven (Roman)]

First set loglevels:

$ curl -XPUT 'localhost:9600/_node/logging?pretty' -H 'Content-Type: application/json' -d '{"logger.logstash.inputs.jdbc" : "WARN"}'
$ curl -XPUT 'localhost:9600/_node/logging?pretty' -H 'Content-Type: application/json' -d '{"logger.org.logstash" : "TRACE"}'
$ curl -XPUT 'localhost:9600/_node/logging?pretty' -H 'Content-Type: application/json' -d '{"logger.logstash" : "TRACE"}'

Here is actual config for logging:

$ curl 'localhost:9600/_node/logging?pretty'
{
  "host" : "localhost",
  "version" : "6.4.0",
  "http_address" : "127.0.0.1:9600",
  "id" : "id",
  "name" : "name",
  "loggers" : {
    "logstash.agent" : "TRACE",
    "logstash.api.service" : "TRACE",
    "logstash.codecs.plain" : "TRACE",
    "logstash.config.source.local.configpathloader" : "TRACE",
    "logstash.config.source.multilocal" : "TRACE",
    "logstash.config.sourceloader" : "TRACE",
    "logstash.configmanagement.extension" : "TRACE",
    "logstash.filters.json" : "TRACE",
    "logstash.inputs.jdbc" : "WARN",
    "logstash.instrument.periodicpoller.deadletterqueue" : "TRACE",
    "logstash.instrument.periodicpoller.jvm" : "TRACE",
    "logstash.instrument.periodicpoller.os" : "TRACE",
    "logstash.instrument.periodicpoller.persistentqueue" : "TRACE",
    "logstash.modules.scaffold" : "TRACE",
    "logstash.modules.xpackscaffold" : "TRACE",
    "logstash.monitoringextension" : "TRACE",
    "logstash.monitoringextension.pipelineregisterhook" : "TRACE",
    "logstash.outputs.elasticsearch" : "TRACE",
    "logstash.pipeline" : "TRACE",
    "logstash.plugins.registry" : "TRACE",
    "logstash.runner" : "TRACE",
    "org.logstash.FieldReference" : "TRACE",
    "org.logstash.Logstash" : "TRACE",
    "org.logstash.execution.AbstractPipelineExt" : "TRACE",
    "org.logstash.execution.ShutdownWatcherExt" : "TRACE",
    "org.logstash.instrument.metrics.gauge.LazyDelegatingGauge" : "TRACE",
    "org.logstash.plugins.pipeline.PipelineBus" : "TRACE",
    "org.logstash.secret.store.SecretStoreFactory" : "TRACE",
    "org.logstash.secret.store.backend.JavaKeyStore" : "TRACE",
    "slowlog.logstash.codecs.plain" : "INFO",
    "slowlog.logstash.filters.json" : "INFO",
    "slowlog.logstash.inputs.jdbc" : "INFO",
    "slowlog.logstash.outputs.elasticsearch" : "INFO"
  }
}

But there is no anything from elasticsearch output plugin. With debug level of logging es output plugin dump its initialization steps

Thanks

[Badger]

Partial answer here.