# Issue with output plugin type elasticsearch in logstash

**URL:** <https://discuss.elastic.co/t/issue-with-output-plugin-type-elasticsearch-in-logstash/314967>\
**Category:** Logstash\
**Created:** [September 22, 2022, 6:30pm UTC](https://discuss.elastic.co/t/issue-with-output-plugin-type-elasticsearch-in-logstash/314967 "2022-09-22T18:30:52Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![sbk-elk](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sbk-elk/32/73414_2.png) [@sbk-elk](https://discuss.elastic.co/u/sbk-elk)\
**Post date:** [September 22, 2022, 6:30pm UTC](https://discuss.elastic.co/t/issue-with-output-plugin-type-elasticsearch-in-logstash/314967/1 "2022-09-22T18:30:52Z")

</div>

Hello All,

Below is my logstash configuration. I have a few questions with respect to how i added the fields and how they carry over in the stages of my pipeline.

1. Do the fields set in hash in my input plugin are added as metadata to the other plugins in my pipeline?
2. Also with this configuration instead of getting the index name as set in the filter i am getting the name as below in Elasticsearch kibana with the output plugin PFA, instead of **logging-my-apps-dev-us-west1-hello**

`"_index. %{[@metadata][target_index]}"`

```auto
input {
  unix {
    path => "/var/run/logstash.sock"
    codec => json
    tags => ["json", "cloud"]
    add_field => {
      "type" => "${LOGSTASH_TYPE:undefined}"
      "env" => "${LOGSTASH_ENV:undefined}"
      "platform" => "${LOGSTASH_PLATFORM:undefined}"
      "project_id" => "${LOGSTASH_PROJECT_ID:undefined}"
      "cloud_region" => "${LOGSTASH_CLOUD_REGION:undefined}"
    }
  }
}

filter {

    if ([input][type] == "unix") {
        json {
          source => "message"
    }
    if ([type] == "hello") {
            mutate { add_field => { "[@metadata][target_index]" => "logging-%{[platform]}-%{[env]}-%{[cloud_region]}-hello" } }
    }
    else {
            mutate { add_field => { "[@metadata][target_index]" => "logging-%{[platform]}-%{[env]}-%{[cloud_region]}-services" } }
    }

    }

}

output {
    elasticsearch {
        cloud_id => "${LOGSTAH_ELASTIC_ID}"
        user => "${LOGSTASH_ELASTIC_USER}"
        password => "${LOGSTASH_ELASTIC_PASSWORD}"
        index => "%{[@metadata][target_index]}"
        retry_initial_interval => 6
        sniffing => false
        manage_template => false
    }
}

```

 ![Screen Shot 2022-09-22 at 2.30.16 PM](https://us1.discourse-cdn.com/elastic/original/3X/9/6/966a796ad5897e8a55c177e38e7a7df25be13555.png)

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [September 22, 2022, 6:36pm UTC](https://discuss.elastic.co/t/issue-with-output-plugin-type-elasticsearch-in-logstash/314967/2 "2022-09-22T18:36:44Z")

</div>

An index name as `%{[@metadata][target_index]}` means that this field does not exist in your documents.

The fields added in your input will be added at the root of the document, so you will have `type`, not `[input][type]` that you are using in your filter.

Since your conditional is using a non-existent field, the field `[@metadata][target_index]` is not being populate, which explains your index name.

Try to change your conditional to use `[type]` instead of `[input][type]`.

---

<div class="post-metadata">

**Author:** ![sbk-elk](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sbk-elk/32/73414_2.png) [@sbk-elk](https://discuss.elastic.co/u/sbk-elk)\
**Post date:** [September 22, 2022, 8:18pm UTC](https://discuss.elastic.co/t/issue-with-output-plugin-type-elasticsearch-in-logstash/314967/3 "2022-09-22T20:18:27Z")

</div>

Appreciate the quick response but I updated the filter as below and retried it. No luck!

```auto
filter {

    if ([type] == "unix") {
        json {
          source => "message"
    }
    if ([type] == "hello") {
            mutate { add_field => { "[@metadata][target_index]" => "logging-%{[platform]}-%{[env]}-%{[cloud_region]}-hello" } }
    }
    else {
            mutate { add_field => { "[@metadata][target_index]" => "logging-%{[platform]}-%{[env]}-%{[cloud_region]}-services" } }
    }

    }

}

```

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [September 22, 2022, 8:37pm UTC](https://discuss.elastic.co/t/issue-with-output-plugin-type-elasticsearch-in-logstash/314967/4 "2022-09-22T20:37:29Z")

</div>

What is your output from Logstash?

I would suggest that you change your output to stdout to see what is the output.

Add this in your output block:

```auto
stdout { codec => rubydebug { metadata => true } }

```

---

<div class="post-metadata">

**Author:** ![sbk-elk](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sbk-elk/32/73414_2.png) [@sbk-elk](https://discuss.elastic.co/u/sbk-elk)\
**Post date:** [September 22, 2022, 8:39pm UTC](https://discuss.elastic.co/t/issue-with-output-plugin-type-elasticsearch-in-logstash/314967/5 "2022-09-22T20:39:39Z")

</div>

Hey @leandrojmp , thanks for your input. The below configuration worked, the condition before json type in filter plugin isn't right.

```auto
input {
  unix {
    path => "/var/run/logstash.sock"
    codec => json
    tags => ["json", "cloud"]
    add_field => {
      "type" => "${LOGSTASH_TYPE:undefined}"
      "env" => "${LOGSTASH_ENV:undefined}"
      "platform" => "${LOGSTASH_PLATFORM:undefined}"
      "project_id" => "${LOGSTASH_PROJECT_ID:undefined}"
      "cloud_region" => "${LOGSTASH_CLOUD_REGION:undefined}"
    }
  }
}

filter {
    json {
          source => "message"
    }
    if ([type] == "hello") {
            mutate { add_field => { "[@metadata][target_index]" => "logging-%{[platform]}-%{[env]}-%{[cloud_region]}-hello" } }
    }
    else {
            mutate { add_field => { "[@metadata][target_index]" => "logging-%{[platform]}-%{[env]}-%{[cloud_region]}-services" } }
    }
}

output {
    elasticsearch {
        cloud_id => "${LOGSTASH_ELASTIC_ID}"
        user => "${LOGSTASH_ELASTIC_USER}"
        password => "${LOGSTASH_ELASTIC_PASSWORD}"
        index => "%{[@metadata][target_index]}"
        retry_initial_interval => 6
        sniffing => false
        manage_template => false
    }
}

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 20, 2022, 8:40pm UTC](https://discuss.elastic.co/t/issue-with-output-plugin-type-elasticsearch-in-logstash/314967/6 "2022-10-20T20:40:07Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.