# Using external document\_id **URL:** \ **Category:** Logstash\ **Created:** [February 27, 2020, 9:54am UTC](https://discuss.elastic.co/t/using-external-document-id/221200 "2020-02-27T09:54:44Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![jwinth](https://avatars.discourse-cdn.com/v4/letter/j/f0a364/32.png) [@jwinth](https://discuss.elastic.co/u/jwinth)\ **Post date:** [February 27, 2020, 9:54am UTC](https://discuss.elastic.co/t/using-external-document-id/221200/1 "2020-02-27T09:54:44Z")
Hi, Whenever i try to assign the document\_id with some attribute from the documents, it creates just one new document on ElasticSearch with the document\_id as the name of the attribute i'm trying to use. What i want is the actual amount of documents retrieved, with the document\_id set as the primary key from the source database. output { elasticsearch { hosts =\> ["placeholder"] index =\> "item" document\_id =\> "%{i.RecordId}" doc\_as\_upsert =\> true user =\> "placeholder" password =\> "placeholder" } } specifically, creates a new document that looks like this; "\_index" : "item", "\_type" : "\_doc", "\_id" : "%{i.RecordId}", "\_score" : 1.0, "\_source" : { I've tried a few different approaches, like copying the recordid into a metadata field, but nothing has worked so far. Thanks for the help in advance ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [February 27, 2020, 3:33pm UTC](https://discuss.elastic.co/t/using-external-document-id/221200/2 "2020-02-27T15:33:47Z")
If you add ``` output { stdout { codec => rubydebug } } ``` then what does an event look like? ---
**Author:** ![jwinth](https://avatars.discourse-cdn.com/v4/letter/j/f0a364/32.png) [@jwinth](https://discuss.elastic.co/u/jwinth)\ **Post date:** [March 3, 2020, 8:12am UTC](https://discuss.elastic.co/t/using-external-document-id/221200/3 "2020-03-03T08:12:28Z")
"recordid" =\> 5259, "itemnumber2" =\> nil, "netprice" =\> 66.48, "isstandard" =\> false, "ean" =\> "4008321515285", "@version" =\> "1", "fk\_unit" =\> 6, "isactive" =\> true, "itemonstock" =\> 0.0, "priority" =\> 1, "@timestamp" =\> 2020-03-03T08:11:00.437Z, "itemalias" =\> nil, "wholesalername" =\> "placeholder", "discountamount" =\> 16.62, "unitname" =\> "Styk", "dateupdated" =\> nil, "fk\_catalog" =\> 2, "name" =\> "placeholder", "itemnumbernumerical" =\> 5651001372, "pricematrixitemid" =\> 2899, "datecreated" =\> 2018-03-21T16:43:17.550Z, "fk\_catalogitemgroup" =\> 145, "sellingprice" =\> 132.96, "wholesalerid" =\> 6, "itemnumber" =\> "5651001372", "issellingpricecalculated" =\> true, "markedfordelete" =\> nil, "allowancecharge" =\> nil, "altdescription" =\> nil, "discountpercent" =\> 20.0, "description" =\> "placeholder", "grossprice" =\> 83.1 } ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [March 3, 2020, 3:12pm UTC](https://discuss.elastic.co/t/using-external-document-id/221200/4 "2020-03-03T15:12:23Z")
It looks like you have a field called [recordid]. You do not have a field called [i.RecordId]. ---
**Author:** ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Post date:** [March 3, 2020, 4:01pm UTC](https://discuss.elastic.co/t/using-external-document-id/221200/5 "2020-03-03T16:01:02Z")
everytime you connect to sqlserver/oracle/mysql and combine multiple table. record gets presented to elk from tablename.fieldname to fieldname (only) and all in lowercase ---
**Author:** ![jwinth](https://avatars.discourse-cdn.com/v4/letter/j/f0a364/32.png) [@jwinth](https://discuss.elastic.co/u/jwinth)\ **Post date:** [March 4, 2020, 1:33pm UTC](https://discuss.elastic.co/t/using-external-document-id/221200/6 "2020-03-04T13:33:50Z")
The problem is, however, that no matter how i format it, it ends up using the name of the attribute as the document\_id, all the ways that have been suggested to me have not worked so far. Creating a new metadata field ends up adding "recordid" as the document id as well, for an example. I can't help but wonder if i simply just write it wrong - i'm fairly new to Logstash, so this is completely possible though filter { mutate { add\_field =\> {"[@metadata][id]" =\> "%[recdordid]" } } } output { elasticsearch { hosts =\> ["placeholder"] index =\> "item" doc\_as\_upsert =\> true document\_id =\> "%{[@metadata][id]}" user =\> "placeholder" password =\> "placeholder" } } as an example ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [March 4, 2020, 5:57pm UTC](https://discuss.elastic.co/t/using-external-document-id/221200/7 "2020-03-04T17:57:10Z")
> [@jwinth](#): > > "%[recdordid]" The field appears to be called recordid, not recdordid. There is no need to use [@metadata], you can reference the field directly. ---
**Author:** ![jwinth](https://avatars.discourse-cdn.com/v4/letter/j/f0a364/32.png) [@jwinth](https://discuss.elastic.co/u/jwinth)\ **Post date:** [March 5, 2020, 9:01am UTC](https://discuss.elastic.co/t/using-external-document-id/221200/8 "2020-03-05T09:01:39Z")
"\_index" : "item", "\_type" : "\_doc", "\_id" : "%[recordid]", "\_score" : 1.0, "\_source" : { Is the output i get when just trying to reference the field. Again, i'm new to logstash and i don't know if this fits the syntax, but it's what i have been suggested to write, and it doesn't seem to work the way i want it to ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [March 5, 2020, 2:16pm UTC](https://discuss.elastic.co/t/using-external-document-id/221200/9 "2020-03-05T14:16:17Z")
You need braces in a sprintf reference, so it should be %{[recordid]}. (In this case the brackets are optional.) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 2, 2020, 2:16pm UTC](https://discuss.elastic.co/t/using-external-document-id/221200/10 "2020-04-02T14:16:29Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.