# Slow Log Duplicates **URL:** \ **Category:** Elasticsearch\ **Created:** [November 23, 2012, 10:51pm UTC](https://discuss.elastic.co/t/slow-log-duplicates/9809 "2012-11-23T22:51:43Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Aaditi\_Sharma](https://avatars.discourse-cdn.com/v4/letter/a/85f322/32.png) [@Aaditi\_Sharma](https://discuss.elastic.co/u/Aaditi_Sharma)\ **Post date:** [November 23, 2012, 10:51pm UTC](https://discuss.elastic.co/t/slow-log-duplicates/9809/1 "2012-11-23T22:51:43Z")
Hi, I've set up elasticsearch and it works as expected. However, I need to log all the queries that get fetched. I set up the configuration in the elasticsearch.yml as follows : index.search.slowlog.level: TRACE index.search.slowlog.threshold.query.trace: 0 and this in the logging.yml additivity: index.search.slowlog: true The logging works, however, I get duplicate entries for each search query. I'm not using any client library, I've written a small client myself to check out elasticsearch. Here is a sample from the slow log : TRACE - [Zynger] [dI][1] took[63.3ms], took\_millis[63], search\_type[QUERY\_THEN\_FETCH], total\_shards[5], source[{"from":0,"size":"25","explain":0,"sort":["\_score"],"min\_score":0.1,"fields":["Gr","Ic","Vert","Title"],"query":{"bool":{"should":[{"query\_string":{"auto\_generate\_phrase\_queries":0,"enable\_position\_increments":0,"fields":["Title.partial^2","DB","Typ","Lang","Are","keys"],"defaultOperator":"AND","query":"_lamp_"}}]}}}], extra\_source[], TRACE - [Zynger] [dI][3] took[67.2ms], took\_millis[67], search\_type[QUERY\_THEN\_FETCH], total\_shards[5], source[{"from":0,"size":"25","explain":0,"sort":["\_score"],"min\_score":0.1,"fields":["Gr","Ic","Vert","Title"],"query":{"bool":{"should":[{"query\_string":{"auto\_generate\_phrase\_queries":0,"enable\_position\_increments":0,"fields":["Title.partial^2","DB","Typ","Lang","Are","keys"],"defaultOperator":"AND","query":"_lamp_"}}]}}}], extra\_source[], TRACE - [Zynger] [dI][2] took[3.8ms], took\_millis[3], search\_type[QUERY\_THEN\_FETCH], total\_shards[5], source[{"from":0,"size":"25","explain":0,"sort":["\_score"],"min\_score":0.1,"fields":["Gr","Ic","Vert","Title"],"query":{"bool":{"should":[{"query\_string":{"auto\_generate\_phrase\_queries":0,"enable\_position\_increments":0,"fields":["Title.partial^2","DB","Typ","Lang","Are","keys"],"defaultOperator":"AND","query":"_lamp_"}}]}}}], extra\_source[], Am I missing something? Thanks -- ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [November 24, 2012, 12:00am UTC](https://discuss.elastic.co/t/slow-log-duplicates/9809/2 "2012-11-24T00:00:32Z")
You are seeing the "fetch" stage from each shard, note it says: TRACE - [Zynger] [dI][1] TRACE - [Zynger] [dI][3] TRACE - [Zynger] [dI][2] On Fri, Nov 23, 2012 at 11:51 PM, Aaditi Sharma [aaditi.p.sharma@gmail.com](mailto:aaditi.p.sharma@gmail.com)wrote: > Hi, > > I've set up elasticsearch and it works as expected. > However, I need to log all the queries that get fetched. I set up the > configuration in the elasticsearch.yml as follows : > > index.search.slowlog.level: TRACE > index.search.slowlog.threshold.query.trace: 0 > > and this in the logging.yml > > additivity: > index.search.slowlog: true > > The logging works, however, I get duplicate entries for each search query. > I'm not using any client library, I've written a small client myself to > check out elasticsearch. > Here is a sample from the slow log : > > TRACE - [Zynger] [dI][1] took[63.3ms], took\_millis[63], > search\_type[QUERY\_THEN\_FETCH], total\_shards[5], > source[{"from":0,"size":"25","explain":0,"sort":["\_score"],"min\_score":0.1,"fields":["Gr","Ic","Vert","Title"],"query":{"bool":{"should":[{"query\_string":{"auto\_generate\_phrase\_queries":0,"enable\_position\_increments":0,"fields":["Title.partial^2","DB","Typ","Lang","Are","keys"],"defaultOperator":"AND","query":"_lamp_"}}]}}}], > extra\_source, > TRACE - [Zynger] [dI][3] took[67.2ms], took\_millis[67], > search\_type[QUERY\_THEN\_FETCH], total\_shards[5], > source[{"from":0,"size":"25","explain":0,"sort":["\_score"],"min\_score":0.1,"fields":["Gr","Ic","Vert","Title"],"query":{"bool":{"should":[{"query\_string":{"auto\_generate\_phrase\_queries":0,"enable\_position\_increments":0,"fields":["Title.partial^2","DB","Typ","Lang","Are","keys"],"defaultOperator":"AND","query":"_lamp_"}}]}}}], > extra\_source, > TRACE - [Zynger] [dI][2] took[3.8ms], took\_millis[3], > search\_type[QUERY\_THEN\_FETCH], total\_shards[5], > source[{"from":0,"size":"25","explain":0,"sort":["\_score"],"min\_score":0.1,"fields":["Gr","Ic","Vert","Title"],"query":{"bool":{"should":[{"query\_string":{"auto\_generate\_phrase\_queries":0,"enable\_position\_increments":0,"fields":["Title.partial^2","DB","Typ","Lang","Are","keys"],"defaultOperator":"AND","query":"_lamp_"}}]}}}], > extra\_source, > > Am I missing something? > Thanks > > -- -- ---
**Author:** ![Aaditi\_Sharma](https://avatars.discourse-cdn.com/v4/letter/a/85f322/32.png) [@Aaditi\_Sharma](https://discuss.elastic.co/u/Aaditi_Sharma)\ **Post date:** [November 24, 2012, 12:44am UTC](https://discuss.elastic.co/t/slow-log-duplicates/9809/3 "2012-11-24T00:44:01Z")
Hi Clinton, I'm not familiar with logging, could you guide me in what do I need to change to log entries after the fetch stage? Thanks. On Sat, Nov 24, 2012 at 5:30 AM, Clinton Gormley [clint@traveljury.com](mailto:clint@traveljury.com)wrote: > You are seeing the "fetch" stage from each shard, note it says: > > TRACE - [Zynger] [dI][1] > TRACE - [Zynger] [dI][3] > TRACE - [Zynger] [dI][2] > > On Fri, Nov 23, 2012 at 11:51 PM, Aaditi Sharma \<[aaditi.p.sharma@gmail.com](mailto:aaditi.p.sharma@gmail.com) > > > wrote: > > > Hi, > > > > I've set up elasticsearch and it works as expected. > > However, I need to log all the queries that get fetched. I set up the > > configuration in the elasticsearch.yml as follows : > > > > index.search.slowlog.level: TRACE > > index.search.slowlog.threshold.query.trace: 0 > > > > and this in the logging.yml > > > > additivity: > > index.search.slowlog: true > > > > The logging works, however, I get duplicate entries for each search query. > > I'm not using any client library, I've written a small client myself to > > check out elasticsearch. > > Here is a sample from the slow log : > > > > TRACE - [Zynger] [dI][1] took[63.3ms], took\_millis[63], > > search\_type[QUERY\_THEN\_FETCH], total\_shards[5], > > source[{"from":0,"size":"25","explain":0,"sort":["\_score"],"min\_score":0.1,"fields":["Gr","Ic","Vert","Title"],"query":{"bool":{"should":[{"query\_string":{"auto\_generate\_phrase\_queries":0,"enable\_position\_increments":0,"fields":["Title.partial^2","DB","Typ","Lang","Are","keys"],"defaultOperator":"AND","query":"_lamp_"}}]}}}], > > extra\_source, > > TRACE - [Zynger] [dI][3] took[67.2ms], took\_millis[67], > > search\_type[QUERY\_THEN\_FETCH], total\_shards[5], > > source[{"from":0,"size":"25","explain":0,"sort":["\_score"],"min\_score":0.1,"fields":["Gr","Ic","Vert","Title"],"query":{"bool":{"should":[{"query\_string":{"auto\_generate\_phrase\_queries":0,"enable\_position\_increments":0,"fields":["Title.partial^2","DB","Typ","Lang","Are","keys"],"defaultOperator":"AND","query":"_lamp_"}}]}}}], > > extra\_source, > > TRACE - [Zynger] [dI][2] took[3.8ms], took\_millis[3], > > search\_type[QUERY\_THEN\_FETCH], total\_shards[5], > > source[{"from":0,"size":"25","explain":0,"sort":["\_score"],"min\_score":0.1,"fields":["Gr","Ic","Vert","Title"],"query":{"bool":{"should":[{"query\_string":{"auto\_generate\_phrase\_queries":0,"enable\_position\_increments":0,"fields":["Title.partial^2","DB","Typ","Lang","Are","keys"],"defaultOperator":"AND","query":"_lamp_"}}]}}}], > > extra\_source, > > > > Am I missing something? > > Thanks > > > > -- > > -- -- ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [November 24, 2012, 4:38am UTC](https://discuss.elastic.co/t/slow-log-duplicates/9809/4 "2012-11-24T04:38:23Z")
Each shard is performing the search. So all calls are logged. You can not change that. You can disable TRACE logs for all classes, for a single class or for a method but AFAIK you can not achieve what you want to do. You have to filter that on your side. I'm wondering: what is your use case? Perhaps you can handle it from your client application and trace the ES call when you send the search request? -- David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 24 nov. 2012 à 01:44, Aaditi Sharma [aaditi.p.sharma@gmail.com](mailto:aaditi.p.sharma@gmail.com) a écrit : Hi Clinton, I'm not familiar with logging, could you guide me in what do I need to change to log entries after the fetch stage? Thanks. On Sat, Nov 24, 2012 at 5:30 AM, Clinton Gormley [clint@traveljury.com](mailto:clint@traveljury.com) wrote: > You are seeing the "fetch" stage from each shard, note it says: > > TRACE - [Zynger] [dI][1] > TRACE - [Zynger] [dI][3] > TRACE - [Zynger] [dI][2] > > On Fri, Nov 23, 2012 at 11:51 PM, Aaditi Sharma [aaditi.p.sharma@gmail.com](mailto:aaditi.p.sharma@gmail.com) wrote: > > > Hi, > > > > I've set up elasticsearch and it works as expected. > > However, I need to log all the queries that get fetched. I set up the configuration in the elasticsearch.yml as follows : > > > > index.search.slowlog.level: TRACE > > index.search.slowlog.threshold.query.trace: 0 > > > > and this in the logging.yml > > > > additivity: > > index.search.slowlog: true > > > > The logging works, however, I get duplicate entries for each search query. > > I'm not using any client library, I've written a small client myself to check out elasticsearch. > > Here is a sample from the slow log : > > > > TRACE - [Zynger] [dI][1] took[63.3ms], took\_millis[63], search\_type[QUERY\_THEN\_FETCH], total\_shards[5], source[{"from":0,"size":"25","explain":0,"sort":["\_score"],"min\_score":0.1,"fields":["Gr","Ic","Vert","Title"],"query":{"bool":{"should":[{"query\_string":{"auto\_generate\_phrase\_queries":0,"enable\_position\_increments":0,"fields":["Title.partial^2","DB","Typ","Lang","Are","keys"],"defaultOperator":"AND","query":"_lamp_"}}]}}}], extra\_source, > > TRACE - [Zynger] [dI][3] took[67.2ms], took\_millis[67], search\_type[QUERY\_THEN\_FETCH], total\_shards[5], source[{"from":0,"size":"25","explain":0,"sort":["\_score"],"min\_score":0.1,"fields":["Gr","Ic","Vert","Title"],"query":{"bool":{"should":[{"query\_string":{"auto\_generate\_phrase\_queries":0,"enable\_position\_increments":0,"fields":["Title.partial^2","DB","Typ","Lang","Are","keys"],"defaultOperator":"AND","query":"_lamp_"}}]}}}], extra\_source, > > TRACE - [Zynger] [dI][2] took[3.8ms], took\_millis[3], search\_type[QUERY\_THEN\_FETCH], total\_shards[5], source[{"from":0,"size":"25","explain":0,"sort":["\_score"],"min\_score":0.1,"fields":["Gr","Ic","Vert","Title"],"query":{"bool":{"should":[{"query\_string":{"auto\_generate\_phrase\_queries":0,"enable\_position\_increments":0,"fields":["Title.partial^2","DB","Typ","Lang","Are","keys"],"defaultOperator":"AND","query":"_lamp_"}}]}}}], extra\_source, > > > > Am I missing something? > > Thanks > > > > -- > > -- -- -- ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:03am UTC](https://discuss.elastic.co/t/slow-log-duplicates/9809/5 "2017-07-06T03:03:04Z")