Search, then remove data

Elastic Stack Elasticsearch
1

I have tried to search for a particular string, and then delete the data that matches. For example, I have an index called 'logstash-2015.04* for all indexes in April. I am trying to search for any part of the message that matches "Error in the RPC receive". I am at a loss at how to accomplish both of these steps. An example I have tried is as such in Postman using POST.

{
"match_phrase" : {
"message" : "Error in the RPC receive"
}
}

But it doesn't respond as I have expect.

Can someone please give me an idea of what would be the best approach to do this?

2

I have tried to search for a particular string, and then delete the data that matches.

Are you actually interested in obtaining the search results or do you just want to delete all data that matches a query? In the latter case you can just use the delete by query API.

But it doesn't respond as I have expect.

Well, how does it respond? Could you include a complete example of your query instead of just a snippet?

3

I haven't gotten any real results to share with what I've done. I'm still learning API calls. But what I was using as a search parameter was this: curl -XPOST 'http://192.168.1.72:9200/_search?1=tag:message'

Here is an example of the raw json message:

{"message":"[ warning] [vmusr:vmusr] Error in the RPC receive loop: RpcIn: Unable to send.\n","@version":"1","@timestamp":"2015-05-06T05:22:45.000Z","host":"192.168.1.38:64173","type":"windowsEventLog","logType":"windowsEventLog","EventTime":"2015-05-06 00:22:45","Hostname":"server.local","Keywords":36028797018963970,"EventType":"WARNING","SeverityValue":3,"Severity":"WARNING","EventID":1000,"SourceName":"VMware Tools","Task":0,"RecordNumber":10516558,"ProcessID":0,"ThreadID":0,"Channel":"Application","Domain":"REALTRUCK","AccountName":"User","UserID":"User","AccountType":"User","Opcode":"Info","EventReceivedTime":1430889766,"SourceModuleName":"eventlog","SourceModuleType":"im_msvistalog","receivedAt":"2015-05-06 05:22:46 UTC"}

My intended goal is to search the message for the string "Error in the RPC receive loop", and if the entry contains this text string, delete the entry.

4

Yes, but are you actually interested in the search results? Or do you just want to delete all matching documents within seeing them one last time?

5

Just to test, I would like to reveal the results to make sure it is matching the data properly. Then pass a 'curl -XDELETE' to get rid of the data.

6

Okay, so do a normal query first and then post the same query as a delete by query request. You said previously that you ran

curl -XPOST 'http://192.168.1.72:9200/_search?1=tag:message

but I don't get what the "1=" part came from. This should work for you:

curl -XPOST 'http://192.168.1.72:9200/_search?q=message:"Error%20in%20the%20RPC%20receive"'

See the URI search documentation. You could of course use the query DSL instead. Once you've verified that you'd be deleting the right documents, change POST to DELETE.