# Upsert large amount of documents based on the property value
**URL:** https://discuss.elastic.co/t/upsert-large-amount-of-documents-based-on-the-property-value/362494
**Category:** Elasticsearch
**Created:** [July 3, 2024, 9:07pm UTC](https://discuss.elastic.co/t/upsert-large-amount-of-documents-based-on-the-property-value/362494 "2024-07-03T21:07:26Z")
**Posts on this page:** 11
**Page:** 1
### Author:  [@CTO\_Servefast](https://discuss.elastic.co/u/CTO_Servefast)
#### Post date: [July 3, 2024, 9:07pm UTC](https://discuss.elastic.co/t/upsert-large-amount-of-documents-based-on-the-property-value/362494/1 "2024-07-03T21:07:26Z")
Is it possible to use Bulk API to upsert large number of documents 100, 1000 ... but i want them to be inserted based on a query for example based on my userId property
Use case for this is that we need to handle thousands of updates of files that we dont know ids to without reeding them all and putting it in redis or something similar
---
### Author:  [@CTO\_Servefast](https://discuss.elastic.co/u/CTO_Servefast)
#### Post date: [July 24, 2024, 2:46pm UTC](https://discuss.elastic.co/t/upsert-large-amount-of-documents-based-on-the-property-value/362494/2 "2024-07-24T14:46:02Z")
Is this something that can not be done ? We really need to do it this way to ensure scalability and performace
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [July 24, 2024, 3:20pm UTC](https://discuss.elastic.co/t/upsert-large-amount-of-documents-based-on-the-property-value/362494/3 "2024-07-24T15:20:20Z")
No, that is not possible. When using the bulk API you indeed need to specify document IDs and can not use queries.
The [update by query API](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-update-by-query.html) seems to be what you will need to use, and while you may be able to group updates through some clever scripted updates you will need to run this multiple times. If you run it in parallel you likely need to ensure the queries do not affect the same documents as that would lead to version conflicts.
---
### Author:  [@CTO\_Servefast](https://discuss.elastic.co/u/CTO_Servefast)
#### Post date: [July 24, 2024, 4:19pm UTC](https://discuss.elastic.co/t/upsert-large-amount-of-documents-based-on-the-property-value/362494/4 "2024-07-24T16:19:12Z")
thank you very much
EDIT: never the less i think this should be a common requirement for a lot of elastic users worth looking into it
---
### Author:  [@CTO\_Servefast](https://discuss.elastic.co/u/CTO_Servefast)
#### Post date: [July 24, 2024, 4:26pm UTC](https://discuss.elastic.co/t/upsert-large-amount-of-documents-based-on-the-property-value/362494/5 "2024-07-24T16:26:27Z")
I have tried updateByQuery but i can only update one file at the time can you provide me with an example off what you were thinking off?
my example:
```auto
const updateResponse = await elasticSearchClient.update({
index: index,
id: userId,
doc: {
user_id: userId,
provider: provider,
},
doc_as_upsert: true,
});
```
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [July 24, 2024, 4:35pm UTC](https://discuss.elastic.co/t/upsert-large-amount-of-documents-based-on-the-property-value/362494/6 "2024-07-24T16:35:54Z")
> [@CTO\_Servefast](#):
>
> can you provide me with an example off what you were thinking off?
No, not really. It would depend on how you are updating and how many matches/updates there are per query. It is quite possible that you may need to run one for each.
---
### Author:  [@CTO\_Servefast](https://discuss.elastic.co/u/CTO_Servefast)
#### Post date: [July 24, 2024, 4:44pm UTC](https://discuss.elastic.co/t/upsert-large-amount-of-documents-based-on-the-property-value/362494/7 "2024-07-24T16:44:22Z")
We have thousands of users doing thousands of updates that would kill our servers i can imagine we are not the first with this type of question are there any other similar posts maybe i can search trough to find something usefull updating one by one is a no go for us unfortunately
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [July 24, 2024, 4:56pm UTC](https://discuss.elastic.co/t/upsert-large-amount-of-documents-based-on-the-property-value/362494/8 "2024-07-24T16:56:18Z")
Do you have any combination of fields that can not be updated and would be unique to each document so you could use this (or maybe a hash of these fields) as a document ID?
---
### Author:  [@CTO\_Servefast](https://discuss.elastic.co/u/CTO_Servefast)
#### Post date: [July 24, 2024, 5:11pm UTC](https://discuss.elastic.co/t/upsert-large-amount-of-documents-based-on-the-property-value/362494/9 "2024-07-24T17:11:22Z")
I have a property that is unique and is stored in the document and that property is never changeing
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [July 24, 2024, 5:21pm UTC](https://discuss.elastic.co/t/upsert-large-amount-of-documents-based-on-the-property-value/362494/10 "2024-07-24T17:21:09Z")
If that is known or retrievable when you update it would be a good candidate for document ID.
---
### Author:  [@CTO\_Servefast](https://discuss.elastic.co/u/CTO_Servefast)
#### Post date: [July 24, 2024, 7:52pm UTC](https://discuss.elastic.co/t/upsert-large-amount-of-documents-based-on-the-property-value/362494/11 "2024-07-24T19:52:19Z")
not something i can guarantee because we have multiple sources of data and theoretically they could be the same.
But hash with provider field might be a good solution