# Is this the correct impl' for customizing \`\_id\` meta data field
**URL:** https://discuss.elastic.co/t/is-this-the-correct-impl-for-customizing-id-meta-data-field/349059
**Category:** Elasticsearch
**Tags:** docker, language-clients
**Created:** [December 11, 2023, 3:22pm UTC](https://discuss.elastic.co/t/is-this-the-correct-impl-for-customizing-id-meta-data-field/349059 "2023-12-11T15:22:19Z")
**Posts on this page:** 19
**Page:** 1
### Author:  [@iby\_dev](https://discuss.elastic.co/u/iby_dev)
#### Post date: [December 11, 2023, 3:22pm UTC](https://discuss.elastic.co/t/is-this-the-correct-impl-for-customizing-id-meta-data-field/349059/1 "2023-12-11T15:22:19Z")
To get Elasticsearch docker image version:
```
image: docker.elastic.co/elasticsearch/elasticsearch:7.16.2
```
To accept a client side customized UUID on the `_id` field i had to change the: `dynamic: strict` property to `true`.
All this time, we have been using `strict` and auto generated `UUIDs` for the `_id` field.
When inspecting, the newly inserted document, i noticed a new field is added: `id` whose contents are matched exactly onto the `_id` metadata field.
Is this the expected behaviour, is this right way of achieving what i set out to do, which is to generate UUIDs, client side and then push to ELASTIC via bulk insert method.
I am using c# and the nest nuget package in dotnet 7.
Thanks for reading.
---
### Author:  [@iby\_dev](https://discuss.elastic.co/u/iby_dev)
#### Post date: [December 12, 2023, 1:55pm UTC](https://discuss.elastic.co/t/is-this-the-correct-impl-for-customizing-id-meta-data-field/349059/2 "2023-12-12T13:55:14Z")
I changed my client side implementation to have a property named "\_Id".
In the error logs i see, it is indeed mapped correctly.
But i get bad request 400 errors on bulk insert.
When i try a single insert using the 1 json doc payload, i get a 201 created response.
Elastic ignores the client side generated UUId.
---
### Author:  [@iby\_dev](https://discuss.elastic.co/u/iby_dev)
#### Post date: [December 12, 2023, 3:52pm UTC](https://discuss.elastic.co/t/is-this-the-correct-impl-for-customizing-id-meta-data-field/349059/3 "2023-12-12T15:52:09Z")
When going back to the dynamic mode: "strict", there is no winning with Elastic Search API. I get confusing and conflicting error messages:
`"Field [_id] is a metadata field and cannot be added inside a document. Use the index API request parameters.""`
Or
` "type" : "strict_dynamic_mapping_exception", "reason" : "mapping set to strict, dynamic introduction of [id] within [_doc] is not allowed",`
I do not want to add a new field/property to the document, i just want to customize the metadata field called: `_id`.
Why is this so hard?
---
### Author:  [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [December 12, 2023, 4:10pm UTC](https://discuss.elastic.co/t/is-this-the-correct-impl-for-customizing-id-meta-data-field/349059/4 "2023-12-12T16:10:50Z")
I am confused ... this is not an Elasticsearch REST API Endpoint.
So, it is unclear how you are accessing Elasticsearch. Do you have a proxy or app in between?
> **[Index API | Elasticsearch Guide \[8.11\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-index_.html)**
> **[Bulk API | Elasticsearch Guide \[8.11\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html)**
---
### Author:  [@iby\_dev](https://discuss.elastic.co/u/iby_dev)
#### Post date: [December 12, 2023, 4:15pm UTC](https://discuss.elastic.co/t/is-this-the-correct-impl-for-customizing-id-meta-data-field/349059/5 "2023-12-12T16:15:19Z")
Hi Stephen, i am using the nest chsarp library like so:
single doc:
many docs:
The NEST sdk confers the c# logic above to a low level client which in the end talks to the ES API. Does this clarify my usage?
---
### Author:  [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [December 12, 2023, 4:17pm UTC](https://discuss.elastic.co/t/is-this-the-correct-impl-for-customizing-id-meta-data-field/349059/6 "2023-12-12T16:17:12Z")
The `_id` is the "resource" id and goes on the end of the URL
```auto
POST my_index/_doc/12345678-999
{
"foo" : "bar"
}
GET my_index/_search
# POST my_index/_doc/12345678-999 201 Created
{
"_index": "my_index",
"_id": "12345678-999",
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 0,
"_primary_term": 1
}
# GET my_index/_search 200 OK
{
"took": 0,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 1,
"relation": "eq"
},
"max_score": 1,
"hits": [
{
"_index": "my_index",
"_id": "12345678-999",
"_score": 1,
"_source": {
"foo": "bar"
}
}
]
}
}
```
---
### Author:  [@iby\_dev](https://discuss.elastic.co/u/iby_dev)
#### Post date: [December 12, 2023, 4:19pm UTC](https://discuss.elastic.co/t/is-this-the-correct-impl-for-customizing-id-meta-data-field/349059/7 "2023-12-12T16:19:30Z")
Shouldn't the nest c# library do this for me? What purpose do this extension method solve then?
` .Id(document._id)`
or do i have to manually construct the http api call myself ?
---
### Author:  [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [December 12, 2023, 4:19pm UTC](https://discuss.elastic.co/t/is-this-the-correct-impl-for-customizing-id-meta-data-field/349059/8 "2023-12-12T16:19:40Z")
Please do not past screen shots of text... really hard to work with
---
### Author:  [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [December 12, 2023, 4:20pm UTC](https://discuss.elastic.co/t/is-this-the-correct-impl-for-customizing-id-meta-data-field/349059/9 "2023-12-12T16:20:47Z")
I am not the C# expert.... I will have to take a look but I think it is important that you understand the underlying calls... you do not set `_id` as a normal property that is the error message you are getting.
---
### Author:  [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [December 12, 2023, 4:21pm UTC](https://discuss.elastic.co/t/is-this-the-correct-impl-for-customizing-id-meta-data-field/349059/10 "2023-12-12T16:21:18Z")
with the `_bulk` endpoint you can do that as part of the payload.
---
### Author:  [@iby\_dev](https://discuss.elastic.co/u/iby_dev)
#### Post date: [December 12, 2023, 4:23pm UTC](https://discuss.elastic.co/t/is-this-the-correct-impl-for-customizing-id-meta-data-field/349059/11 "2023-12-12T16:23:16Z")
It would be cool if you could refer my Q to a c# expert at ES. That would be greatly appreciated.
I have tried the `_bulk` endpoint and the i can see the mapped `_id` properties when the error messages are logged and handled. But every document gets rejected. Let me try again.
---
### Author:  [@iby\_dev](https://discuss.elastic.co/u/iby_dev)
#### Post date: [December 12, 2023, 4:27pm UTC](https://discuss.elastic.co/t/is-this-the-correct-impl-for-customizing-id-meta-data-field/349059/12 "2023-12-12T16:27:34Z")
@stephenb - will this work if the mapping schema document is set to: dynamic: strict ?
---
### Author:  [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [December 12, 2023, 4:29pm UTC](https://discuss.elastic.co/t/is-this-the-correct-impl-for-customizing-id-meta-data-field/349059/13 "2023-12-12T16:29:23Z")
I pinged internally...
I do not think `dynamic: strict ` is a factor
---
### Author:  [@iby\_dev](https://discuss.elastic.co/u/iby_dev)
#### Post date: [December 12, 2023, 4:30pm UTC](https://discuss.elastic.co/t/is-this-the-correct-impl-for-customizing-id-meta-data-field/349059/14 "2023-12-12T16:30:21Z")
Ok thanks for claryfing, my tech lead advises we dont want to change to:
`dynamic: true`.
---
### Author:  [@iby\_dev](https://discuss.elastic.co/u/iby_dev)
#### Post date: [December 12, 2023, 4:52pm UTC](https://discuss.elastic.co/t/is-this-the-correct-impl-for-customizing-id-meta-data-field/349059/15 "2023-12-12T16:52:29Z")
I tried the `.BulkAll()` method with a list of documents passed to it. Each document has a pregenerated `UUID` assigned to a `_id` property on each document.
The `DroppedDocumentCallback()` callback function is triggered revealing no errors why the document was dropped. Looking closely at the logs, i see:
```auto
2023-12-12 16:32:26 2023-12-12 16:32:26.332 +00:00 [My Worker] [my_scan_1] [] [ERR] [1deab1ef21cc] [Development] [1.0.0.0] Dropped doc - on index: cds_scan_1-doc with _id: hWLfXowBADcS-dXjXrm2
2023-12-12 16:32:26 Api Response: Status: 400 Result:
2023-12-12 16:32:26 Error Response: {"headers":{},"root_cause":null}
2023-12-12 16:32:26 Failed doc: {"_id":"Ba/FyFrAslLo9b82OurV","file_type":"csv","file_name":"20121001 Setembro.csv","ingest":"2023-12-12T16:32:26.0732641+00:00" .... }.
```
> You can see two different ids have been generated, one by me client side (_Ba/FyFrAslLo9b82OurV_) and another (_hWLfXowBADcS-dXjXrm2_) by, ES API - this is revealed by inspecting the: `BulkResponseItemBase.Id` property.
---
### Author:  [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [December 12, 2023, 4:55pm UTC](https://discuss.elastic.co/t/is-this-the-correct-impl-for-customizing-id-meta-data-field/349059/16 "2023-12-12T16:55:32Z")
The .NET lead is out today, hopefully he can take a look tomorrow..
---
### Author:  [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [December 12, 2023, 4:57pm UTC](https://discuss.elastic.co/t/is-this-the-correct-impl-for-customizing-id-meta-data-field/349059/17 "2023-12-12T16:57:36Z")
Sorry, I'm not really following all the "snippets."
Documents can be dropped for several reasons...
For tomorrow I would think you would want to show a sample of the CSV, your template, and your C# code...
---
### Author:  [@iby\_dev](https://discuss.elastic.co/u/iby_dev)
#### Post date: [December 13, 2023, 10:48am UTC](https://discuss.elastic.co/t/is-this-the-correct-impl-for-customizing-id-meta-data-field/349059/18 "2023-12-13T10:48:33Z")
I got this working, now, i will post an update shortly.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 10, 2024, 10:49am UTC](https://discuss.elastic.co/t/is-this-the-correct-impl-for-customizing-id-meta-data-field/349059/19 "2024-01-10T10:49:09Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.