Delete data by beat.name

[faulander]

Hi Guys,
Can you point me in the right direction when i want to delete data from a given hostname or beatname?

i tried:

POST metricbeat-6.4.0-2018.10.10/_delete_by_query
{
  "query": { 
    "match": {
      "term" : { "beat.name" : "Test"
      }
    }
  }
}

but it doesn't work. So simple question: How do i delete data by beat.name?

[dadoonet (David Pilato)]

What gives the following?

GET metricbeat-6.4.0-2018.10.10/_search
{
  "query": { 
    "match": {
      "term" : { "beat.name" : "Test"
      }
    }
  }
}

May be try with "beat.name" : "test" BTW. I guess it depends on the mapping for that field as you are using a Term query.

[faulander]

@dadoonet

GET metricbeat-6.4.0-2018.10.10/_search { "query": { "match": { "term" : { "beat.name" : "Test" } } } }

Thank you David, GET doesn't find the data either.

What would be the preferred way to delete data shipped by MetricBeat? The Shippers all have unique names and (of course) hostnames.

[dadoonet (David Pilato)]

Depends on your mapping and your actual data. I guess Test was an example.

[faulander]

yes, here is the exact data i am talking about:

[dadoonet (David Pilato)]

It does not tell me anything about the mapping.
But here is a guess then. Try this:

GET metricbeat-6.4.0-2018.10.10/_search
{
  "query": { 
    "match": { "beat.name" : "Test" }
  }
}

[faulander]

@dadoonet

GET metricbeat-6.4.0-2018.10.10/_search { "query": { "match": { "beat.name" : "Test" } } }

That works. Deleting still doesn't work though:
POST metricbeat-6.4.0-2018.10.10/_delete_by_query
{
"query": {
"match": { "beat.name" : "Test - Daimler - SAML" }
}
}

Output:
{
"took": 1,
"timed_out": false,
"total": 0,
"deleted": 0,
"batches": 0,
"version_conflicts": 0,
"noops": 0,
"retries": {
"bulk": 0,
"search": 0
},
"throttled_millis": 0,
"requests_per_second": -1,
"throttled_until_millis": 0,
"failures":
}

[dadoonet (David Pilato)]

In one case you searched for Test but then you deleted something else?

[faulander]

Test was just an Example. Of course i tried the GET also with the „correct“ term - the same i tried with the Delete Statement

[dadoonet (David Pilato)]

If search gives back results, delete by query should work as well.

If not, please share the full commands you're passing to the system and both results.
And the mapping for your index.

[faulander]

Search:
GET metricbeat-6.4.0-2018.10.10/_search
{
"query": {
"match": { "beat.name" : "Test - Daimler - SAML" }
}
}
Response:
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 0,
"max_score": null,
"hits": []
}
}
Delete:
POST metricbeat-6.4.0-2018.10.10/_delete_by_query
{
"query": {
"match": { "beat.name" : "Test - Daimler - SAML" }
}
}
Response:
{
"took": 0,
"timed_out": false,
"total": 0,
"deleted": 0,
"batches": 0,
"version_conflicts": 0,
"noops": 0,
"retries": {
"bulk": 0,
"search": 0
},
"throttled_millis": 0,
"requests_per_second": -1,
"throttled_until_millis": 0,
"failures": []
}

[dadoonet (David Pilato)]

so the search gave back 0 result.
Why would delete by query give another result?

[faulander]

David, i am an idiot. Sorry, topic can be closed.