# Document\_id not set in ElasticSearch, even though it is configured inside logstash.conf

**URL:** <https://discuss.elastic.co/t/document-id-not-set-in-elasticsearch-even-though-it-is-configured-inside-logstash-conf/176016>\
**Category:** Logstash\
**Created:** [April 9, 2019, 11:50am UTC](https://discuss.elastic.co/t/document-id-not-set-in-elasticsearch-even-though-it-is-configured-inside-logstash-conf/176016 "2019-04-09T11:50:44Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Miao](https://avatars.discourse-cdn.com/v4/letter/m/8baadc/32.png) [@Miao](https://discuss.elastic.co/u/Miao)\
**Post date:** [April 9, 2019, 11:50am UTC](https://discuss.elastic.co/t/document-id-not-set-in-elasticsearch-even-though-it-is-configured-inside-logstash-conf/176016/1 "2019-04-09T11:50:45Z")

</div>

Here are the contents inside my `logstash.conf`:

```
input {
	http {
		host => "127.0.0.1"
		port => 31311 
	}
}

filter {
	mutate {
		split => { "[headers][request_path]" => "/"}
		add_field => { "index_id" => "%{[headers][request_path][1]}" }
	}
	
	ruby { 
		code => "event.set('request_path_length', event.get('[headers][request_path]').length)" 
	}
	
	if [request_path_length == 3] {
		mutate {
			add_field => { "document_id" => "%{[headers][request_path][2]}" }
		}
	}
}
	
output {
	stdout {
		codec => "rubydebug"
	}
  
	if [request_path_length == 3] {
		elasticsearch {
			hosts => "http://localhost:9200"
			index => "%{index_id}"
			document_id => "%{[headers][request_path][2]}"
		}
	}
	else {
		elasticsearch {
			hosts => "http://localhost:9200"
			index => "%{index_id}"
		}
	}
}

```

As a test, I ran the PowerShell command

`C:\Users\BolverkXR\Downloads\curl-7.64.1-win64-mingw\bin> .\curl.exe -XPUT 'http://127.0.0.1:31311/twitter_new/7'`

I see the following output on my Logstash terminal:

```
{
                "message" => "",
               "@version" => "1",
                   "host" => "127.0.0.1",
             "@timestamp" => 2019-04-09T11:35:22.458Z,
    "request_path_length" => 3,
                "headers" => {
              "http_host" => "127.0.0.1:31311",
         "content_length" => "0",
           "request_path" => [
            [0] "",
            [1] "twitter_new",
            [2] "7"
        ],
            "http_accept" => "*/*",
           "http_version" => "HTTP/1.1",
        "http_user_agent" => "curl/7.64.1",
         "request_method" => "PUT"
    },
               "index_id" => "twitter_new"
}

```

As you can see, `document_id` is not set to `7`, even though that is what I would expect.

How can I fix this?

---

<div class="post-metadata">

**Author:** ![A\_B](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/a_b/32/17104_2.png) [@A\_B](https://discuss.elastic.co/u/A_B)\
**Post date:** [April 9, 2019, 12:15pm UTC](https://discuss.elastic.co/t/document-id-not-set-in-elasticsearch-even-though-it-is-configured-inside-logstash-conf/176016/2 "2019-04-09T12:15:56Z")

</div>

> [@Miao](#):
>
> if [request\_path\_length == 3] {

Try with this

```
if [request_path_length] == 3 {

```

---

<div class="post-metadata">

**Author:** ![Miao](https://avatars.discourse-cdn.com/v4/letter/m/8baadc/32.png) [@Miao](https://discuss.elastic.co/u/Miao)\
**Post date:** [April 9, 2019, 12:21pm UTC](https://discuss.elastic.co/t/document-id-not-set-in-elasticsearch-even-though-it-is-configured-inside-logstash-conf/176016/3 "2019-04-09T12:21:56Z")

</div>

Thanks for your reply. After changing my `logstash.conf` file as you suggested, I made another `PUT` request to `/twitter_new/8`. I then made a `GET` request to retrieve all entries, and this was the entry corresponding to the latest `PUT` request I made:

```
{
	"_index": "twitter_new",
	"_type": "doc",
	"_id": "O5AIAmoBCWsefMj-o7Fw",
	"_score": 1,
	"_source": {
		"message": "",
		"document_id": "8",
		"@version": "1",
		"@timestamp": "2019-04-09T12:18:00.665Z",
		"index_id": "twitter_new",
		"request_path_length": 3,
		"headers": {
			"request_path": [
				"",
				"twitter_new",
				"8"
			],
			"http_accept": "*/*",
			"http_version": "HTTP/1.1",
			"content_length": "0",
			"request_method": "PUT",
			"http_user_agent": "curl/7.64.1",
			"http_host": "127.0.0.1:31311"
		},
		"host": "127.0.0.1"
	}
}

```

As you can see, inside `source`, `document_id` is indeed set to `8`, but `_id` still a randomly generated string. I would expect `_id` to be `8` as well, just like how `_index` is `twitter_new`. Am I misunderstanding something?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 9, 2019, 12:36pm UTC](https://discuss.elastic.co/t/document-id-not-set-in-elasticsearch-even-though-it-is-configured-inside-logstash-conf/176016/4 "2019-04-09T12:36:04Z")

</div>

> [@Miao](#):
>
> if [request\_path\_length == 3] {

Did you change that if in the output section as well as the filter section?

---

<div class="post-metadata">

**Author:** ![Miao](https://avatars.discourse-cdn.com/v4/letter/m/8baadc/32.png) [@Miao](https://discuss.elastic.co/u/Miao)\
**Post date:** [April 9, 2019, 12:37pm UTC](https://discuss.elastic.co/t/document-id-not-set-in-elasticsearch-even-though-it-is-configured-inside-logstash-conf/176016/5 "2019-04-09T12:37:17Z")

</div>

Ah, of course! Silly me.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 7, 2019, 12:37pm UTC](https://discuss.elastic.co/t/document-id-not-set-in-elasticsearch-even-though-it-is-configured-inside-logstash-conf/176016/6 "2019-05-07T12:37:26Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.