# How to perform bulk unique inserts into an elasticsearch table using the .Net API? **URL:** \ **Category:** Elasticsearch\ **Created:** [November 4, 2013, 2:56pm UTC](https://discuss.elastic.co/t/how-to-perform-bulk-unique-inserts-into-an-elasticsearch-table-using-the-net-api/14244 "2013-11-04T14:56:51Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![bdb](https://avatars.discourse-cdn.com/v4/letter/b/b77776/32.png) [@bdb](https://discuss.elastic.co/u/bdb)\ **Post date:** [November 4, 2013, 2:56pm UTC](https://discuss.elastic.co/t/how-to-perform-bulk-unique-inserts-into-an-elasticsearch-table-using-the-net-api/14244/1 "2013-11-04T14:56:51Z")
We are currently researching elasticsearch as a replacement for our current system that uses TVP's[http://msdn.microsoft.com/en-us/library/bb675163(v=vs.110).aspx](http://msdn.microsoft.com/en-us/library/bb675163%28v=vs.110%29.aspx) on SQL Server 2008R2. The system ensures that the records about to be inserted are unique in the table (based on a composite key). The two data fields to be checked are UserID and ContentTitle. Instead of checking if both exist, a hash can be created and stored in a single field. When a record is to be added, the hashed valued of the incoming records can be checked against the existing table's hash values. How would this be accomplished in elasticsearch using the .Net API? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [November 14, 2013, 8:56am UTC](https://discuss.elastic.co/t/how-to-perform-bulk-unique-inserts-into-an-elasticsearch-table-using-the-net-api/14244/2 "2013-11-14T08:56:10Z")
Hey, two possible solutions (among a couple of others) 1. If you want to create a hash, use that hash as ID, when indexing your data. The tuple of id and type are unique in an index. So if you reindex a document with that hash, the data gets simple overwritten. Just make sure, you have a good hash function in order to not overwrite data. You could also configure the mapping of the index, to use the content of the hash field to be used as ID. See [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-id-field.html) 2. Instead of creating a hash, it might be sufficient to simply concatenate userId and contentTitle into a single ID (123\_456 for example), and use this ID when indexing. This would save a couple of CPU cycles as you dont need to hash, but it might not be unique, depending on the format of these ids. Sorry, cannot tell you anything about the .NET API, but this should give you a first hint I hope. --Alex On Mon, Nov 4, 2013 at 3:56 PM, bdb [baden0x1@gmail.com](mailto:baden0x1@gmail.com) wrote: > We are currently researching elasticsearch as a replacement for our > current system that uses TVP's[http://msdn.microsoft.com/en-us/library/bb675163(v=vs.110).aspx](http://msdn.microsoft.com/en-us/library/bb675163%28v=vs.110%29.aspx) on > SQL Server 2008R2. > > The system ensures that the records about to be inserted are unique in the > table (based on a composite key). > > The two data fields to be checked are UserID and ContentTitle. Instead of > checking if both exist, a hash can be created and stored in a single field. > When a record is to be added, the hashed valued of the incoming records can > be checked against the existing table's hash values. > > How would this be accomplished in elasticsearch using the .Net API? > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:07am UTC](https://discuss.elastic.co/t/how-to-perform-bulk-unique-inserts-into-an-elasticsearch-table-using-the-net-api/14244/3 "2017-07-06T02:07:14Z")
Closed on July 6, 2017, 2:07am UTC