Search, then remove data for the results

Elastic Stack Elasticsearch
1

I am looking for assistance.

I have an elasticsearch cluster that has an index called 'foo'. Within
foo, I have a ton of information that contains the string "Error in the RPC
receive". I need to do a search for any message that contains that string,
then delete anything.

I have seen several examples, but am not able to flush what I need. I am
using the Chrome extension Postman.

I have tried simple queries such as:

{
"match_phrase" : {
"message" : "Error in the RPC receive"
}
}

but to no avail.

Can I please request some help?

--
Please update your bookmarks! We moved to https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/716fda8d-5cd8-410a-9554-05469a45c7ac%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

Do you want something like this

Allan

On 7 May 2015 at 22:15, dpich@realtruck.com wrote:

I am looking for assistance.

I have an elasticsearch cluster that has an index called 'foo'. Within
foo, I have a ton of information that contains the string "Error in the RPC
receive". I need to do a search for any message that contains that string,
then delete anything.

I have seen several examples, but am not able to flush what I need. I am
using the Chrome extension Postman.

I have tried simple queries such as:

{
"match_phrase" : {
"message" : "Error in the RPC receive"
}
}

but to no avail.

Can I please request some help?

--
Please update your bookmarks! We moved to https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/716fda8d-5cd8-410a-9554-05469a45c7ac%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/716fda8d-5cd8-410a-9554-05469a45c7ac%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Please update your bookmarks! We moved to https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAECdJzBiZgwZZX%3DfO%2BCiseGuXMfQj2xFjZno6P7S-q0mcF__Zg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

I haven't gotten any real results to share with what I've done. I'm still
learning API calls. But what I was using as a search parameter was this:
curl -XPOST 'http://192.168.1.72:9200/_search?1=tag:message'

Here is an example of the raw json message:

{"message":"[ warning] [vmusr:vmusr] Error in the RPC receive loop:
RpcIn: Unable to
send.\n","@version":"1","@timestamp":"2015-05-06T05:22:45.000Z","host":"192.168.1.38:64173","type":"windowsEventLog","logType":"windowsEventLog","EventTime":"2015-05-06
00:22:45","Hostname":"server.local","Keywords":36028797018963970,"EventType":"WARNING","SeverityValue":3,"Severity":"WARNING","EventID":1000,"SourceName":"VMware
Tools","Task":0,"RecordNumber":10516558,"ProcessID":0,"ThreadID":0,"Channel":"Application","Domain":"REALTRUCK","AccountName":"User","UserID":"User","AccountType":"User","Opcode":"Info","EventReceivedTime":1430889766,"SourceModuleName":"eventlog","SourceModuleType":"im_msvistalog","receivedAt":"2015-05-06
05:22:46 UTC"}

My intended goal is to search the message for the string "Error in the RPC
receive loop", and if the entry contains this text string, delete the
entry.

--
Please update your bookmarks! We moved to https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/422e1224-20fb-4dca-b891-e482e08e6de0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

4

Hi

Then it definitely sounds like delete_by_query is your friend here. Follow
the link I sent earlier. If it not what you want then come back and tell
us why.

Allan

On 8 May 2015 at 15:15, dpich@realtruck.com wrote:

I haven't gotten any real results to share with what I've done. I'm still
learning API calls. But what I was using as a search parameter was this:
curl -XPOST 'http://192.168.1.72:9200/_search?1=tag:message'

Here is an example of the raw json message:

{"message":"[ warning] [vmusr:vmusr] Error in the RPC receive loop:
RpcIn: Unable to
send.\n","@version":"1","@timestamp":"2015-05-06T05:22:45.000Z","host":"
192.168.1.38:64173","type":"windowsEventLog","logType":"windowsEventLog","EventTime":"2015-05-06
00:22:45","Hostname":"server.local","Keywords":36028797018963970,"EventType":"WARNING","SeverityValue":3,"Severity":"WARNING","EventID":1000,"SourceName":"VMware
Tools","Task":0,"RecordNumber":10516558,"ProcessID":0,"ThreadID":0,"Channel":"Application","Domain":"REALTRUCK","AccountName":"User","UserID":"User","AccountType":"User","Opcode":"Info","EventReceivedTime":1430889766,"SourceModuleName":"eventlog","SourceModuleType":"im_msvistalog","receivedAt":"2015-05-06
05:22:46 UTC"}

My intended goal is to search the message for the string "Error in the RPC
receive loop", and if the entry contains this text string, delete the
entry.

--
Please update your bookmarks! We moved to https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/422e1224-20fb-4dca-b891-e482e08e6de0%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/422e1224-20fb-4dca-b891-e482e08e6de0%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Please update your bookmarks! We moved to https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAECdJzAWWZXJ1hrrFm9%3DFdd1SMKg_Ft2gf%2BawWBx%2BqJxEKQKBQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

5

Hi Alan,

Thanks for your help.

Here is the query that finally got me some results:

curl XGET
'httpd://192.168.1.72:9200/logstash-2015.05.01/_search?q=message:'Error in
the RPC receive''

This gives me the information I wanted to confirm before I deleted.

{
"took": 1990,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 123503,
"max_score": 2.8327804,
"hits": [
{
...foo...

When I switch this to this command (curl XDELETE
'httpd://192.168.1.72:9200/logstash-2015.05.01/_search?q=message:'Error in
the RPC receive''), The other node in the cluster has an issue:

{

"error": "RemoteTransportException[[es-logstash-n2][inet[/192.168.1.80:9301]][indices:admin/mapping/delete]]; nested: TypeMissingException[[_all] type[[_search]] missing: No index has the type.]; ",

"status": 404

}

On Friday, May 8, 2015 at 9:38:31 AM UTC-5, Allan Mitchell wrote:

Hi

Then it definitely sounds like delete_by_query is your friend here.
Follow the link I sent earlier. If it not what you want then come back and
tell us why.

Allan

On 8 May 2015 at 15:15, <dp...@realtruck.com <javascript:>> wrote:

I haven't gotten any real results to share with what I've done. I'm
still learning API calls. But what I was using as a search parameter was
this: curl -XPOST 'http://192.168.1.72:9200/_search?1=tag:message'

Here is an example of the raw json message:

{"message":"[ warning] [vmusr:vmusr] Error in the RPC receive loop:
RpcIn: Unable to
send.\n","@version":"1","@timestamp":"2015-05-06T05:22:45.000Z","host":"
192.168.1.38:64173","type":"windowsEventLog","logType":"windowsEventLog","EventTime":"2015-05-06
00:22:45","Hostname":"server.local","Keywords":36028797018963970,"EventType":"WARNING","SeverityValue":3,"Severity":"WARNING","EventID":1000,"SourceName":"VMware
Tools","Task":0,"RecordNumber":10516558,"ProcessID":0,"ThreadID":0,"Channel":"Application","Domain":"REALTRUCK","AccountName":"User","UserID":"User","AccountType":"User","Opcode":"Info","EventReceivedTime":1430889766,"SourceModuleName":"eventlog","SourceModuleType":"im_msvistalog","receivedAt":"2015-05-06
05:22:46 UTC"}

My intended goal is to search the message for the string "Error in the
RPC receive loop", and if the entry contains this text string, delete the
entry.

--
Please update your bookmarks! We moved to https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/422e1224-20fb-4dca-b891-e482e08e6de0%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/422e1224-20fb-4dca-b891-e482e08e6de0%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Please update your bookmarks! We moved to https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/490f6957-9ac9-4570-bbf7-54377457a3ad%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

6

Hi

Wouldn't you want the _query endpoint not the _search endpoint.

from the documentation

curl -XDELETE 'http://localhost:9200/twitter/tweet/_query?q=user:kimchy'

On 8 May 2015 at 16:01, dpich@realtruck.com wrote:

Hi Alan,

Thanks for your help.

Here is the query that finally got me some results:

curl XGET 'httpd://
192.168.1.72:9200/logstash-2015.05.01/_search?q=message:'Error in the RPC
receive''

This gives me the information I wanted to confirm before I deleted.

{
"took": 1990,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 123503,
"max_score": 2.8327804,
"hits": [
{
...foo...

When I switch this to this command (curl XDELETE 'httpd://
192.168.1.72:9200/logstash-2015.05.01/_search?q=message:'Error in the RPC
receive''), The other node in the cluster has an issue:

{

"error": "RemoteTransportException[[es-logstash-n2][inet[/192.168.1.80:9301]][indices:admin/mapping/delete]]; nested: TypeMissingException[[_all] type[[_search]] missing: No index has the type.]; ",

"status": 404

}

On Friday, May 8, 2015 at 9:38:31 AM UTC-5, Allan Mitchell wrote:

Hi

Then it definitely sounds like delete_by_query is your friend here.
Follow the link I sent earlier. If it not what you want then come back and
tell us why.

Allan

On 8 May 2015 at 15:15, dp...@realtruck.com wrote:

I haven't gotten any real results to share with what I've done. I'm
still learning API calls. But what I was using as a search parameter was
this: curl -XPOST 'http://192.168.1.72:9200/_search?1=tag:message'

Here is an example of the raw json message:

{"message":"[ warning] [vmusr:vmusr] Error in the RPC receive loop:
RpcIn: Unable to
send.\n","@version":"1","@timestamp":"2015-05-06T05:22:45.000Z","host":"
192.168.1.38:64173","type":"windowsEventLog","logType":"windowsEventLog","EventTime":"2015-05-06
00:22:45","Hostname":"server.local","Keywords":36028797018963970,"EventType":"WARNING","SeverityValue":3,"Severity":"WARNING","EventID":1000,"SourceName":"VMware
Tools","Task":0,"RecordNumber":10516558,"ProcessID":0,"ThreadID":0,"Channel":"Application","Domain":"REALTRUCK","AccountName":"User","UserID":"User","AccountType":"User","Opcode":"Info","EventReceivedTime":1430889766,"SourceModuleName":"eventlog","SourceModuleType":"im_msvistalog","receivedAt":"2015-05-06
05:22:46 UTC"}

My intended goal is to search the message for the string "Error in the
RPC receive loop", and if the entry contains this text string, delete the
entry.

--
Please update your bookmarks! We moved to https://discuss.elastic.co/

You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/422e1224-20fb-4dca-b891-e482e08e6de0%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/422e1224-20fb-4dca-b891-e482e08e6de0%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Please update your bookmarks! We moved to https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/490f6957-9ac9-4570-bbf7-54377457a3ad%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/490f6957-9ac9-4570-bbf7-54377457a3ad%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Please update your bookmarks! We moved to https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAECdJzDP%3DNiP%2BOoiL_1xEfvs4fXRE%3DpG11YwZB%2BxuoB5mWWHCQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

7

Thanks Alan,

That did the trick! I appreciate the assistance.

On Friday, May 8, 2015 at 10:12:35 AM UTC-5, Allan Mitchell wrote:

Hi

Wouldn't you want the _query endpoint not the _search endpoint.

from the documentation

curl -XDELETE 'http://localhost:9200/twitter/tweet/_query?q=user:kimchy'

On 8 May 2015 at 16:01, <dp...@realtruck.com <javascript:>> wrote:

Hi Alan,

Thanks for your help.

Here is the query that finally got me some results:

curl XGET 'httpd://
192.168.1.72:9200/logstash-2015.05.01/_search?q=message:'Error in the
RPC receive''

This gives me the information I wanted to confirm before I deleted.

{
"took": 1990,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 123503,
"max_score": 2.8327804,
"hits": [
{
...foo...

When I switch this to this command (curl XDELETE 'httpd://
192.168.1.72:9200/logstash-2015.05.01/_search?q=message:'Error in the
RPC receive''), The other node in the cluster has an issue:

{

"error": "RemoteTransportException[[es-logstash-n2][inet[/192.168.1.80:9301]][indices:admin/mapping/delete]]; nested: TypeMissingException[[_all] type[[_search]] missing: No index has the type.]; ",

"status": 404

}

On Friday, May 8, 2015 at 9:38:31 AM UTC-5, Allan Mitchell wrote:

Hi

Then it definitely sounds like delete_by_query is your friend here.
Follow the link I sent earlier. If it not what you want then come back and
tell us why.

Allan

On 8 May 2015 at 15:15, dp...@realtruck.com wrote:

I haven't gotten any real results to share with what I've done. I'm
still learning API calls. But what I was using as a search parameter was
this: curl -XPOST 'http://192.168.1.72:9200/_search?1=tag:message'

Here is an example of the raw json message:

{"message":"[ warning] [vmusr:vmusr] Error in the RPC receive loop:
RpcIn: Unable to
send.\n","@version":"1","@timestamp":"2015-05-06T05:22:45.000Z","host":"
192.168.1.38:64173","type":"windowsEventLog","logType":"windowsEventLog","EventTime":"2015-05-06
00:22:45","Hostname":"server.local","Keywords":36028797018963970,"EventType":"WARNING","SeverityValue":3,"Severity":"WARNING","EventID":1000,"SourceName":"VMware
Tools","Task":0,"RecordNumber":10516558,"ProcessID":0,"ThreadID":0,"Channel":"Application","Domain":"REALTRUCK","AccountName":"User","UserID":"User","AccountType":"User","Opcode":"Info","EventReceivedTime":1430889766,"SourceModuleName":"eventlog","SourceModuleType":"im_msvistalog","receivedAt":"2015-05-06
05:22:46 UTC"}

My intended goal is to search the message for the string "Error in the
RPC receive loop", and if the entry contains this text string, delete the
entry.

--
Please update your bookmarks! We moved to https://discuss.elastic.co/

You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/422e1224-20fb-4dca-b891-e482e08e6de0%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/422e1224-20fb-4dca-b891-e482e08e6de0%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Please update your bookmarks! We moved to https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/490f6957-9ac9-4570-bbf7-54377457a3ad%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/490f6957-9ac9-4570-bbf7-54377457a3ad%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Please update your bookmarks! We moved to https://discuss.elastic.co/

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d3e4229e-952e-4a08-bf34-08793b1dcdbd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.