# Ignore non-existent docs in mget **URL:** \ **Category:** Elasticsearch\ **Created:** [May 21, 2013, 2:05am UTC](https://discuss.elastic.co/t/ignore-non-existent-docs-in-mget/12044 "2013-05-21T02:05:49Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Allan\_Johns](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/allan_johns/32/2387_2.png) [@Allan\_Johns](https://discuss.elastic.co/u/Allan_Johns)\ **Post date:** [May 21, 2013, 2:05am UTC](https://discuss.elastic.co/t/ignore-non-existent-docs-in-mget/12044/1 "2013-05-21T02:05:49Z")
Hi, ES noob here, bear with me. I need to filter out the results from an mget so that those with exists=false are not returned. How can I do this? The docs state: "All multi indices API support the ignore\_indices option. Setting it to missing will cause indices that do not exists to be ignored from the execution..." But I can't seem to get this to work. I've tried something like this (and various combinations), to no avail: ]$ curl -XGET '[http://localhost:9200/idx/type/\_mget?ignore\_indices=missing](http://localhost:9200/idx/type/_mget?ignore_indices=missing)' -d '{"ids": ["xxx"]}' Thanks, Allan -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![drewr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/drewr/32/7803_2.png) [@drewr](https://discuss.elastic.co/u/drewr)\ **Post date:** [May 21, 2013, 3:40pm UTC](https://discuss.elastic.co/t/ignore-non-existent-docs-in-mget/12044/2 "2013-05-21T15:40:22Z")
Allan Johns wrote: > I need to filter out the results from an mget so that those with > exists=false are not returned. How can I do this? [...] > But I can't seem to get this to work. I've tried something like > this (and various combinations), to no avail: > > ]$ curl -XGET > '[http://localhost:9200/idx/type/\_mget?ignore\_indices=missing](http://localhost:9200/idx/type/_mget?ignore_indices=missing)' > -d '{"ids": ["xxx"]}' ignore\_indices is only useful when you're using an index wildcard. For example: curl -s 'localhost:9200/foo\*/\_search?ignore\_indices=missing&size=1' If no index matches the foo\* pattern, you won't get an error. This would have the same effect as searching the entire cluster. In your case, you're not using a wildcard for the index name, but more importantly, you're using mget which is only useful when you know exactly the index/type/id triplet that will reference the docs you want. Back to your original need, what do you mean by needing "those with exists=false" to be omitted? You have a field called "exists" and want to search where it has a true value? Drew -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Allan\_Johns](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/allan_johns/32/2387_2.png) [@Allan\_Johns](https://discuss.elastic.co/u/Allan_Johns)\ **Post date:** [May 29, 2013, 2:19am UTC](https://discuss.elastic.co/t/ignore-non-existent-docs-in-mget/12044/3 "2013-05-29T02:19:36Z")
I want those nonexistent documents not to be returned in the result set at all. So: I know the index, type and set of IDs, however some IDs will not exist in the database, and I don't know this ahead of time. I simply want those nonexistent documents to be silently removed from the result set. Is this possible? "Back to your original need, what do you mean by needing "those with exists=false" to be omitted? You have a field called "exists" and want to search where it has a true value?" Consider: $ curl -XGET '[http://localhost:9200/idx/type/SOME\_BAD\_ID](http://localhost:9200/idx/type/SOME_BAD_ID)' {"\_index":"fury","\_type":"products","\_id":"ID0","exists":false} I want a result set that does not include these exists=false docs at all. thanks Allan On Wed, May 22, 2013 at 1:40 AM, Drew Raines [aaraines@gmail.com](mailto:aaraines@gmail.com) wrote: > Allan Johns wrote: > > I need to filter out the results from an mget so that those with > > > exists=false are not returned. How can I do this? > > [...] > > But I can't seem to get this to work. I've tried something like this (and > > > various combinations), to no avail: > > > > ]$ curl -XGET '[http://localhost:9200/idx/\*\*type/\_mget?ignore\_indices=](http://localhost:9200/idx/**type/_mget?ignore_indices=)\*\* > > missing [http://localhost:9200/idx/type/\_mget?ignore\_indices=missing](http://localhost:9200/idx/type/_mget?ignore_indices=missing)' > > -d '{"ids": ["xxx"]}' > > ignore\_indices is only useful when you're using an index wildcard. For > example: > > curl -s 'localhost:9200/foo\*/\_search?\*\*ignore\_indices=missing&size=1' > > If no index matches the foo\* pattern, you won't get an error. This would > have the same effect as searching the entire cluster. > > In your case, you're not using a wildcard for the index name, but more > importantly, you're using mget which is only useful when you know exactly > the index/type/id triplet that will reference the docs you want. > > Back to your original need, what do you mean by needing "those with > exists=false" to be omitted? You have a field called "exists" and want to > search where it has a true value? > > Drew > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to elasticsearch+unsubscribe@\*\*[googlegroups.com](http://googlegroups.com)[elasticsearch%2Bunsubscribe@googlegroups.com](mailto:elasticsearch%2Bunsubscribe@googlegroups.com) > . > For more options, visit [https://groups.google.com/\*\*groups/opt\_out](https://groups.google.com/**groups/opt_out)[https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out) > . -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Allan\_Johns](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/allan_johns/32/2387_2.png) [@Allan\_Johns](https://discuss.elastic.co/u/Allan_Johns)\ **Post date:** [May 29, 2013, 2:22am UTC](https://discuss.elastic.co/t/ignore-non-existent-docs-in-mget/12044/4 "2013-05-29T02:22:57Z")
Sorry I meant to show that example using mget: ]$ curl -XGET '[http://localhost:9200/fury/products/\_mget](http://localhost:9200/fury/products/_mget)' -d '{"ids": ["BAD\_ID","GOOD\_ID"]}' {"docs":[{"\_index":"fury","\_type":"products","\_id":"BAD\_ID","exists":false},{"\_index":"fury","\_type":"products","\_id":"GOOD\_ID","foo":"bah","etc":"etcetc"}]} What I want is this behaviour: ]$ curl -XGET '[http://localhost:9200/fury/products/\_mget](http://localhost:9200/fury/products/_mget)' -d '{"ids": ["BAD\_ID", "GOOD\_ID"]}' {"docs":[{"\_index":"fury","\_type":"products","\_id":"GOOD\_ID","foo":"bah","etc":"etcetc"}]} On Wed, May 29, 2013 at 12:19 PM, Allan Johns [nerdvegas@gmail.com](mailto:nerdvegas@gmail.com) wrote: > I want those nonexistent documents not to be returned in the result set at > all. So: I know the index, type and set of IDs, however some IDs will not > exist in the database, and I don't know this ahead of time. I simply want > those nonexistent documents to be silently removed from the result set. Is > this possible? > > "Back to your original need, what do you mean by needing "those with > exists=false" to be omitted? You have a field called "exists" and want to > search where it has a true value?" > > Consider: > > $ curl -XGET '[http://localhost:9200/idx/type/SOME\_BAD\_ID](http://localhost:9200/idx/type/SOME_BAD_ID)' > {"\_index":"fury","\_type":"products","\_id":"ID0","exists":false} > > I want a result set that does not include these exists=false docs at all. > > thanks > Allan > > On Wed, May 22, 2013 at 1:40 AM, Drew Raines [aaraines@gmail.com](mailto:aaraines@gmail.com) wrote: > > > Allan Johns wrote: > > > > I need to filter out the results from an mget so that those with > > > > > exists=false are not returned. How can I do this? > > > > [...] > > > > But I can't seem to get this to work. I've tried something like this > > > > > (and various combinations), to no avail: > > > > > > ]$ curl -XGET '[http://localhost:9200/idx/\*\*type/\_mget?ignore\_indices=](http://localhost:9200/idx/**type/_mget?ignore_indices=)\*\* > > > missing [http://localhost:9200/idx/type/\_mget?ignore\_indices=missing](http://localhost:9200/idx/type/_mget?ignore_indices=missing)' > > > -d '{"ids": ["xxx"]}' > > > > ignore\_indices is only useful when you're using an index wildcard. For > > example: > > > > curl -s 'localhost:9200/foo\*/\_search?\*\*ignore\_indices=missing&size=1' > > > > If no index matches the foo\* pattern, you won't get an error. This would > > have the same effect as searching the entire cluster. > > > > In your case, you're not using a wildcard for the index name, but more > > importantly, you're using mget which is only useful when you know exactly > > the index/type/id triplet that will reference the docs you want. > > > > Back to your original need, what do you mean by needing "those with > > exists=false" to be omitted? You have a field called "exists" and want to > > search where it has a true value? > > > > Drew > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to elasticsearch+unsubscribe@\*\*[googlegroups.com](http://googlegroups.com)[elasticsearch%2Bunsubscribe@googlegroups.com](mailto:elasticsearch%2Bunsubscribe@googlegroups.com) > > . > > For more options, visit [https://groups.google.com/\*\*groups/opt\_out](https://groups.google.com/**groups/opt_out)[https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out) > > . -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [May 29, 2013, 7:39am UTC](https://discuss.elastic.co/t/ignore-non-existent-docs-in-mget/12044/5 "2013-05-29T07:39:07Z")
You can easily filter that on client side. Is your concern about network usage? May be you should open an issue to have an 'ignore\_missing' option for \_mget API (default to false)? -- David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 29 mai 2013 à 04:22, Allan Johns [nerdvegas@gmail.com](mailto:nerdvegas@gmail.com) a écrit : > Sorry I meant to show that example using mget: > > ]$ curl -XGET '[http://localhost:9200/fury/products/\_mget](http://localhost:9200/fury/products/_mget)' -d '{"ids": ["BAD\_ID","GOOD\_ID"]}' > {"docs":[{"\_index":"fury","\_type":"products","\_id":"BAD\_ID","exists":false},{"\_index":"fury","\_type":"products","\_id":"GOOD\_ID","foo":"bah","etc":"etcetc"}]} > > What I want is this behaviour: > > ]$ curl -XGET '[http://localhost:9200/fury/products/\_mget](http://localhost:9200/fury/products/_mget)' -d '{"ids": ["BAD\_ID", "GOOD\_ID"]}' > {"docs":[{"\_index":"fury","\_type":"products","\_id":"GOOD\_ID","foo":"bah","etc":"etcetc"}]} > > On Wed, May 29, 2013 at 12:19 PM, Allan Johns [nerdvegas@gmail.com](mailto:nerdvegas@gmail.com) wrote: > > > I want those nonexistent documents not to be returned in the result set at all. So: I know the index, type and set of IDs, however some IDs will not exist in the database, and I don't know this ahead of time. I simply want those nonexistent documents to be silently removed from the result set. Is this possible? > > > > "Back to your original need, what do you mean by needing "those with exists=false" to be omitted? You have a field called "exists" and want to search where it has a true value?" > > > > Consider: > > > > $ curl -XGET '[http://localhost:9200/idx/type/SOME\_BAD\_ID](http://localhost:9200/idx/type/SOME_BAD_ID)' > > {"\_index":"fury","\_type":"products","\_id":"ID0","exists":false} > > > > I want a result set that does not include these exists=false docs at all. > > > > thanks > > Allan > > > > On Wed, May 22, 2013 at 1:40 AM, Drew Raines [aaraines@gmail.com](mailto:aaraines@gmail.com) wrote: > > > > > Allan Johns wrote: > > > > > > > I need to filter out the results from an mget so that those with exists=false are not returned. How can I do this? > > > > > > [...] > > > > > > > But I can't seem to get this to work. I've tried something like this (and various combinations), to no avail: > > > > > > > > ]$ curl -XGET '[http://localhost:9200/idx/type/\_mget?ignore\_indices=missing](http://localhost:9200/idx/type/_mget?ignore_indices=missing)' > > > > -d '{"ids": ["xxx"]}' > > > > > > ignore\_indices is only useful when you're using an index wildcard. For example: > > > > > > curl -s 'localhost:9200/foo\*/\_search?ignore\_indices=missing&size=1' > > > > > > If no index matches the foo\* pattern, you won't get an error. This would have the same effect as searching the entire cluster. > > > > > > In your case, you're not using a wildcard for the index name, but more importantly, you're using mget which is only useful when you know exactly the index/type/id triplet that will reference the docs you want. > > > > > > Back to your original need, what do you mean by needing "those with exists=false" to be omitted? You have a field called "exists" and want to search where it has a true value? > > > > > > Drew > > > > > > -- > > > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Allan\_Johns](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/allan_johns/32/2387_2.png) [@Allan\_Johns](https://discuss.elastic.co/u/Allan_Johns)\ **Post date:** [May 29, 2013, 8:12am UTC](https://discuss.elastic.co/t/ignore-non-existent-docs-in-mget/12044/6 "2013-05-29T08:12:04Z")
My concern isn't network usage, it's paging. Sometimes my result set is quite large, and I want to only page into the results when necessary. If missing docs are included in the response, I'm forced to fetch the whole lot, in order to filter out missing docs client-side, so that I'm reporting the correct document count to the client app. If I can't do this currently and others agree this is a reasonable thing to want to do, I'll open an issue. thx A On Wed, May 29, 2013 at 5:39 PM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > You can easily filter that on client side. > Is your concern about network usage? > > May be you should open an issue to have an 'ignore\_missing' option for > \_mget API (default to false)? > > -- > David 😉 > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > Le 29 mai 2013 à 04:22, Allan Johns [nerdvegas@gmail.com](mailto:nerdvegas@gmail.com) a écrit : > > Sorry I meant to show that example using mget: > > ]$ curl -XGET '[http://localhost:9200/fury/products/\_mget](http://localhost:9200/fury/products/_mget)' -d '{"ids": > ["BAD\_ID","GOOD\_ID"]}' > > {"docs":[{"\_index":"fury","\_type":"products","\_id":"BAD\_ID","exists":false},{"\_index":"fury","\_type":"products","\_id":"GOOD\_ID","foo":"bah","etc":"etcetc"}]} > > What I want is this behaviour: > > ]$ curl -XGET '[http://localhost:9200/fury/products/\_mget](http://localhost:9200/fury/products/_mget)' -d '{"ids": > ["BAD\_ID", "GOOD\_ID"]}' > > {"docs":[{"\_index":"fury","\_type":"products","\_id":"GOOD\_ID","foo":"bah","etc":"etcetc"}]} > > On Wed, May 29, 2013 at 12:19 PM, Allan Johns [nerdvegas@gmail.com](mailto:nerdvegas@gmail.com) wrote: > > > I want those nonexistent documents not to be returned in the result set > > at all. So: I know the index, type and set of IDs, however some IDs will > > not exist in the database, and I don't know this ahead of time. I simply > > want those nonexistent documents to be silently removed from the result > > set. Is this possible? > > > > "Back to your original need, what do you mean by needing "those with > > exists=false" to be omitted? You have a field called "exists" and want to > > search where it has a true value?" > > > > Consider: > > > > $ curl -XGET '[http://localhost:9200/idx/type/SOME\_BAD\_ID](http://localhost:9200/idx/type/SOME_BAD_ID)' > > {"\_index":"fury","\_type":"products","\_id":"ID0","exists":false} > > > > I want a result set that does not include these exists=false docs at all. > > > > thanks > > Allan > > > > On Wed, May 22, 2013 at 1:40 AM, Drew Raines [aaraines@gmail.com](mailto:aaraines@gmail.com) wrote: > > > > > Allan Johns wrote: > > > > > > I need to filter out the results from an mget so that those with > > > > > > > exists=false are not returned. How can I do this? > > > > > > [...] > > > > > > But I can't seem to get this to work. I've tried something like this > > > > > > > (and various combinations), to no avail: > > > > > > > > ]$ curl -XGET '[http://localhost:9200/idx/\*\*type/\_mget?ignore\_indices=](http://localhost:9200/idx/**type/_mget?ignore_indices=)\*\* > > > > missing [http://localhost:9200/idx/type/\_mget?ignore\_indices=missing](http://localhost:9200/idx/type/_mget?ignore_indices=missing)' > > > > -d '{"ids": ["xxx"]}' > > > > > > ignore\_indices is only useful when you're using an index wildcard. For > > > example: > > > > > > curl -s 'localhost:9200/foo\*/\_search?\*\*ignore\_indices=missing&size=1' > > > > > > If no index matches the foo\* pattern, you won't get an error. This would > > > have the same effect as searching the entire cluster. > > > > > > In your case, you're not using a wildcard for the index name, but more > > > importantly, you're using mget which is only useful when you know exactly > > > the index/type/id triplet that will reference the docs you want. > > > > > > Back to your original need, what do you mean by needing "those with > > > exists=false" to be omitted? You have a field called "exists" and want to > > > search where it has a true value? > > > > > > Drew > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to elasticsearch+unsubscribe@\*\*[googlegroups.com](http://googlegroups.com)[elasticsearch%2Bunsubscribe@googlegroups.com](mailto:elasticsearch%2Bunsubscribe@googlegroups.com) > > > . > > > For more options, visit [https://groups.google.com/\*\*groups/opt\_out](https://groups.google.com/**groups/opt_out)[https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out) > > > . > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:34am UTC](https://discuss.elastic.co/t/ignore-non-existent-docs-in-mget/12044/7 "2017-07-06T02:34:10Z")