Is there any option to log queries?

Hi,
I'm using Spring Data Elasticsearch, and so some of the queries are
generated. I'd like to be able to log all queries that hit Elasticsearch,
simply to be able to better understand what is being generated and run.
I found some old (2010) discussions saying that this isn't possible - but
I'm hoping that it might be possible now. It would make a lot of sense to
be able to see these queries (conditionally of course, with a TRACE log
level for example).
Is this possible in any way now?
Cheers,
Eugen.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/242b5863-caa6-4a11-84e2-648c48e35d10%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Hi Eugen,

You can use the "slowlog.log" file to view queries that were sent: it
prints out the JSON of the query received.
This log is off by default and only prints logs for operations that took
longer than durations specific in the elasticsearch.yml config file.
You can turn it on and set the thresholds to a very low value to view all
queries - just remember to return them to reasonable values when you are
done testing.
Also, note that you will need to restart your node for the changes to take
effect.

Here is an example of how I set my local (testing) node to print out all
queries received to the "slowlog" log:
################################## Slow Log
##################################

Shard level query and fetch threshold logging.

#index.search.slowlog.threshold.query.warn: 10s
index.search.slowlog.threshold.query.info: 1ms
#index.search.slowlog.threshold.query.debug: 2s
#index.search.slowlog.threshold.query.trace: 500ms

#index.search.slowlog.threshold.fetch.warn: 1s
index.search.slowlog.threshold.fetch.info: 1ms
#index.search.slowlog.threshold.fetch.debug: 500ms
#index.search.slowlog.threshold.fetch.trace: 200ms

#index.indexing.slowlog.threshold.index.warn: 10s
#index.indexing.slowlog.threshold.index.info: 5s
#index.indexing.slowlog.threshold.index.debug: 2s
#index.indexing.slowlog.threshold.index.trace: 500ms

################################## GC Logging
################################

Hope this helps!

On Tuesday, November 25, 2014 6:20:35 PM UTC+2, Eugen Paraschiv wrote:

Hi,
I'm using Spring Data Elasticsearch, and so some of the queries are
generated. I'd like to be able to log all queries that hit Elasticsearch,
simply to be able to better understand what is being generated and run.
I found some old (2010) discussions saying that this isn't possible - but
I'm hoping that it might be possible now. It would make a lot of sense to
be able to see these queries (conditionally of course, with a TRACE log
level for example).
Is this possible in any way now?
Cheers,
Eugen.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/6381350e-bca5-4564-be10-84a236b63423%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

I did read about that option, thanks for letting me know that it's the way
to go. I though that there would be another way, but since there's not,
I'll go ahead and do that.
Thank you,
Eugen.

On Tue, Nov 25, 2014 at 6:46 PM, Yosi Haran yosi@my6sense.com wrote:

Hi Eugen,

You can use the "slowlog.log" file to view queries that were sent: it
prints out the JSON of the query received.
This log is off by default and only prints logs for operations that took
longer than durations specific in the elasticsearch.yml config file.
You can turn it on and set the thresholds to a very low value to view all
queries - just remember to return them to reasonable values when you are
done testing.
Also, note that you will need to restart your node for the changes to take
effect.

Here is an example of how I set my local (testing) node to print out all
queries received to the "slowlog" log:
################################## Slow Log
##################################

Shard level query and fetch threshold logging.

#index.search.slowlog.threshold.query.warn: 10s
index.search.slowlog.threshold.query.info: 1ms
#index.search.slowlog.threshold.query.debug: 2s
#index.search.slowlog.threshold.query.trace: 500ms

#index.search.slowlog.threshold.fetch.warn: 1s
index.search.slowlog.threshold.fetch.info: 1ms
#index.search.slowlog.threshold.fetch.debug: 500ms
#index.search.slowlog.threshold.fetch.trace: 200ms

#index.indexing.slowlog.threshold.index.warn: 10s
#index.indexing.slowlog.threshold.index.info: 5s
#index.indexing.slowlog.threshold.index.debug: 2s
#index.indexing.slowlog.threshold.index.trace: 500ms

################################## GC Logging
################################

Hope this helps!

On Tuesday, November 25, 2014 6:20:35 PM UTC+2, Eugen Paraschiv wrote:

Hi,
I'm using Spring Data Elasticsearch, and so some of the queries are
generated. I'd like to be able to log all queries that hit Elasticsearch,
simply to be able to better understand what is being generated and run.
I found some old (2010) discussions saying that this isn't possible - but
I'm hoping that it might be possible now. It would make a lot of sense to
be able to see these queries (conditionally of course, with a TRACE log
level for example).
Is this possible in any way now?
Cheers,
Eugen.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/8d--WJ9xHxk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/6381350e-bca5-4564-be10-84a236b63423%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/6381350e-bca5-4564-be10-84a236b63423%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Eugen Paraschiv
Consultant, Baeldung
Mobile: +40728896170
Blog: www.baeldung.com
Twitter: @baeldung https://twitter.com/baeldung

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CA%2BA8vzEkUaV1vMYKyo%3DdA%2BTbwbSiZpa2bVJs2KC_p13FfPPvrA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.