Avoiding delete by query to write in logs

Elastic Stack Elasticsearch
1

Hi I am using delete by query plugin to delete around 10 million records. However, doing so is eating up my disk space as each document deletion event is recorded in elasticsearch logs file. How can i avoid elasticsearch writing this delete to logs.

P.S. my elasticsearch version is 2.4

Regards,
S N

2

Can you configure the logger to reduce the level for the ES component that's writing the log?

Mike McCandless

3

Did you modify logging level?

It should not log that by default AFAIK.

Are you sure it takes more space in logs or that's because of the normal behavior of elasticsearch when it comes to deletion of documents?
I mean that removing a document takes more space because of Lucene immutable segments.

Is that what you are seeing or is it really about logs?

4

Sorry for delayed response. I'm trying to figure out as how the logger can be configured. It's helpful if you can guide me.

Thanks

5

Can you share one of theses logs you mentioned at first?

6

Here is what its getting filled with in the log. I also forgot to mention that i'm using searchgaurd.

[2017-01-09 08:11:54,815][DEBUG][com.floragunn.searchguard.configuration.PrivilegesEvaluator] indicesOptions IndicesOptions[id=48, ignore_unavailable=false, allow_no_indices=false, expand_wildcards_open=false, expand_wildcards_closed=false, allow_alisases_to_multiple_indices=false, forbid_closed_indices=true]
[2017-01-09 08:11:54,815][DEBUG][com.floragunn.searchguard.configuration.PrivilegesEvaluator] raw indices [index_name]
[2017-01-09 08:11:54,815][DEBUG][com.floragunn.searchguard.configuration.PrivilegesEvaluator] Resolved [index_name] to [index_name]

7

So you are using an open source plugin which does that logging.
Have a look at their documentation.

Here may be change config/logging.yml and change the log level for package com.floragunn.searchguard to INFO or WARN.

But it's not caused by elasticsearch code.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.