How to partially delete an index

Elastic Stack Elasticsearch
1

How can I delete only the records that meet the criteria of the index registered in elasticsearch?

I don't want to delete the entire index.
In other words, not the following instruction.

curl -XDELETE localhost:9200/index_name?pretty=true
2

In addition, please let me know if there is an option to run a test run (like dry-run, which doesn't actually delete the file, but checks the execution).

3

Have a look at:

Do a _search instead with the same exact query.

4

Thank you for answering my question.

Does it support the curl command?
I seem to get an error with the following command.

# curl -XPOST 'localhost:9200/api-2021.04.06/_search' -d '{ "query": { "match": { "log.file.path":"/var/log/api/api-2021-02" } } }'
{"error":"Content-Type header [application/x-www-form-urlencoded] is not supported","status":406}
5

First of all, I decided to do as you said and send a GET request from Kibana's developer tool.

The following request works.

GET /api-2021.04.06/_search
{
  "query": {
    "bool": {
      "must": {
        "match": {
          "log.file.path":"/var/log/api/api-2021-02-*.log"
        }
      }
    }
  }
}

However, the all-important delete request does not work.

POST /api-2021.04.06/_delete_by_query
{
  "query": {
    "bool": {
      "must": {
        "match": {
          "log.file.path":"/var/log/api/api-2021-02-*.log"
        }
      }
    }
  }
}

{"statusCode":502,"error":"Bad Gateway","message":"Client request timeout"}

It means a timeout, but I don't believe it is that big a log. What could be the cause? Also, is there any way to extend the timeout period?

6

If I run it from elasticsearch-head, I get another error.

"failures": [
{
... snip ...
"cause": {
"type": "cluster_block_exception",
"reason": "index [api-2021.04.06] blocked by: [TOO_MANY_REQUESTS/12/disk usage exceeded flood-stage watermark, index has read-only-allow-delete block];"
},
"status": 429
}

I saw this same error in Kibana when I had a lot of logs fetched by filebeat and the disk was getting tight.

However, I am trying to delete the log due to this error confirmed by Kibana.
If this operation is also blocked, how can I reduce the disk space?

7

DELETE a document does not remove the doc immediately. It creates more data on disk to Mark the document as deleted.

Then it eventually removes it when a merge happens.

Because you did not tell initially, may be describe what is the pro lyon want to fix by deleting some documents?

8

I am sorry that I did not explain it well enough.

Because you did not tell initially, may be describe what is the pro lyon want to fix by deleting some documents?

What do you mean by this?

Does it mean that it is not practical to request a delete query from elasticsearch in this situation?

Please let me know if you have any good ideas to solve [TOO_MANY_REQUESTS/12/disk usage exceeded flood-stage watermark, index has read-only-allow-delete block].

9

Then it eventually removes it when a merge happens.
Perhaps you may have found yourself in the situation described here.

The following error has occurred.

"failures": [
{
... snip ...
"cause": {
"type": "version_conflict_engine_exception",
"reason": "[3NYwpngBPtRZ_HR4DA5t]: version conflict, required seqNo [11709405], primary term [1]. but no document was found",
"index_uuid": "4FRCHYUyTzWKrYs1fmr2Aw",
"shard": "0",
"index": "api-2021.04.06"
},
"status": 409
}

Is there any way to resolve the conflict?

10

What is the current status of your cluster?

What is the output of:

GET /
GET /_cat/nodes?v
GET /_cat/health?v
GET /_cat/indices?v

If some outputs are too big, please share them on gist.github.com and link them here.

1 Like
11

Thank you for your answer.

I'm very sorry, but I couldn't wait for your answer, so I deleted the corresponding index, and now I can't reproduce it.

I think all the current statuses are in a good state.
I will share the current status.

If the status is not healthy, what does this command tell you?

12

I have deleted the index and can no longer check it. Is the following explanation correct?

GET /api-2021.04.06/_search
{
  "query": {
    "bool": {
      "must": {
        "match": {
          "log.file.path":"/var/log/api/api-2021-02-*.log"
        }
      }
    }
  }
}

However, the all-important delete request does not work.

POST /api-2021.04.06/_delete_by_query
{
  "query": {
    "bool": {
      "must": {
        "match": {
          "log.file.path":"/var/log/api/api-2021-02-*.log"
        }
      }
    }
  }
}

I would like to know this because in the future I may do something like partially deleting a document in the index.

13

It just gives me an overview of your cluster to have a better understanding before trying to dig in things.

Why?

If you are running out of disk space, I can understand that but otherwise I don't see why it would not work.

14

If you are running out of disk space, I can understand that but otherwise I don't see why it would not work.

I'm sorry for the confusion.
Your advice turned out to be correct.

[TOO_MANY_REQUESTS/12/disk usage exceeded flood-stage watermark, index has read-only-allow-delete block].

The cause seems to be that elasticsearch is blocking index due to exhausted disk space.

We have secured enough disk space and changed the destination of the index in elasticsearch.
After collecting the logs again and confirming that there were no errors, I ran the above command and it worked.

15

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.