# How to use \_mget in elasticserach-hadoop

**URL:** https://discuss.elastic.co/t/how-to-use--mget-in-elasticserach-hadoop/54481
**Category:** Elasticsearch
**Tags:** es-hadoop
**Created:** [June 30, 2016, 11:44pm UTC](https://discuss.elastic.co/t/how-to-use--mget-in-elasticserach-hadoop/54481 "2016-06-30T23:44:18Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![jspooner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jspooner/32/12984_2.png) [@jspooner](https://discuss.elastic.co/u/jspooner)
#### Post date: [June 30, 2016, 11:44pm UTC](https://discuss.elastic.co/t/how-to-use--mget-in-elasticserach-hadoop/54481/1 "2016-06-30T23:44:18Z")

</div>

Is it possible to use the **[\_mget](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-multi-get.html)** function from the [EsSpark](https://github.com/elastic/elasticsearch-hadoop/blob/master/spark/core/main/scala/org/elasticsearch/spark/rdd/EsSpark.scala) class?

I need to update a large amount of documents in ElasticSearch and I'm thinking the best method is collect every document with \_mget. Example

```
POST /devices-v1/device/_mget
{
    "ids" : ["abc, 
    "zzz",
    "ffff3"
    ]
}
```

---

<div class="post-metadata">

### Author: ![james.baiera](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/james.baiera/32/10209_2.png) [@james.baiera](https://discuss.elastic.co/u/james.baiera)
#### Post date: [July 1, 2016, 2:38pm UTC](https://discuss.elastic.co/t/how-to-use--mget-in-elasticserach-hadoop/54481/2 "2016-07-01T14:38:51Z")

</div>

Hello!

ES-Hadoop (and EsSpark) focus primarily on the scroll and bulk api's (amidst node api's for service discovery). It does not support the `_mget` endpoint at this time. If you have a large number of updates you need to perform, I propose setting the `es.write.operation` to `update` in combination to specifying which field is your `id` field with the `es.mapping.id` property.

For more info please see [the documentation pages about connector configuration](https://www.elastic.co/guide/en/elasticsearch/hadoop/current/configuration.html).

Hope this helps!

---

<div class="post-metadata">

### Author: ![jspooner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jspooner/32/12984_2.png) [@jspooner](https://discuss.elastic.co/u/jspooner)
#### Post date: [July 1, 2016, 8:04pm UTC](https://discuss.elastic.co/t/how-to-use--mget-in-elasticserach-hadoop/54481/3 "2016-07-01T20:04:52Z")

</div>

The problem is that my document has an array with objects. If I have a job that adds a new object it will clobber the previous objects in that carry. I was thinking I could get all of the original objects add my new object and resubmit that via the update api.

My other idea is to have a groovy script manage this array

> <https://gist.github.com/jspooner/aca369e357ac824c5832b347f98c005a>

My other option is to use the parent child relationship in ES but I may have too many documents for a fast search.

---

<div class="post-metadata">

### Author: ![james.baiera](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/james.baiera/32/10209_2.png) [@james.baiera](https://discuss.elastic.co/u/james.baiera)
#### Post date: [July 1, 2016, 8:43pm UTC](https://discuss.elastic.co/t/how-to-use--mget-in-elasticserach-hadoop/54481/4 "2016-07-01T20:43:50Z")

</div>

It seems like you're attempting to emulate a map-style object within your document. While a parent child relationship between documents is the easiest way to rationalize this sort of write and update pattern, you are indeed correct that search query performance is impacted negatively with that modeling approach. Using a Parent-Child Relationship is more for situations where child documents outnumber their parent documents by a very large margin.

Groovy may be the best route for conserving your search performance here. My recommendations for this is to parameterize the script so that it can be compiled and cached between indexing calls (which it seems is the way you have outlined in your gist, well done).

Another recommendation, if you have not yet done it already, is to map your demographics field as a nested field. This is going to depend on what types of queries you expect to execute. If you're looking for all "worstgolfers" reported to be age `26` by some source `"ESPN 8, the Ocho"`, you may find that documents erroneously match this query depending on their mixture of data entries in the demographics field. [More on that here](https://www.elastic.co/guide/en/elasticsearch/guide/current/nested-objects.html).

Hope this helps!

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 1:24pm UTC](https://discuss.elastic.co/t/how-to-use--mget-in-elasticserach-hadoop/54481/5 "2017-07-06T13:24:05Z")

</div>


