# Environment filter is not working

**URL:** <https://discuss.elastic.co/t/environment-filter-is-not-working/42040>\
**Category:** Logstash\
**Created:** [February 17, 2016, 3:43pm UTC](https://discuss.elastic.co/t/environment-filter-is-not-working/42040 "2016-02-17T15:43:56Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![gaurav1424](https://avatars.discourse-cdn.com/v4/letter/g/97f17d/32.png) [@gaurav1424](https://discuss.elastic.co/u/gaurav1424)\
**Post date:** [February 17, 2016, 3:43pm UTC](https://discuss.elastic.co/t/environment-filter-is-not-working/42040/1 "2016-02-17T15:43:57Z")

</div>

here is example:

My input file :  
filter {  
environment {  
add\_metadata\_from\_env =\> {"region" =\> "REGION\_ENV"}  
add\_field =\> ["my\_env", "%{[@metadata][region]}"]  
}  
}

bash-3.2$ echo $REGION\_ENV  
amer  
bash-3.2$

My output :  
{  
"message" =\> "Hello world\r",  
"@version" =\> "1",  
"@timestamp" =\> "2016-02-17T00:04:13.540Z",  
"host" =\> "0:0:0:0:0:0:0:1",  
"port" =\> 61248,  
"my\_env" =\> "%{[@metadata][region]}"  
}

Anything I am missing here ?

Thanks,  
Gaurav

---

<div class="post-metadata">

**Author:** ![gaurav1424](https://avatars.discourse-cdn.com/v4/letter/g/97f17d/32.png) [@gaurav1424](https://discuss.elastic.co/u/gaurav1424)\
**Post date:** [February 17, 2016, 3:44pm UTC](https://discuss.elastic.co/t/environment-filter-is-not-working/42040/2 "2016-02-17T15:44:36Z")

</div>

I tried this as well as mentioned on other thread.

Logstash Input :  
filter {  
environment {  
add\_metadata\_from\_env =\> {"region" =\> "REGION\_ENV"}  
add\_field =\> ["my\_env", "%{[@metadata][region]}"]  
}  
}

output {  
stdout  
{ codec =\> rubydebug {  
metadata =\> true  
}  
}  
}

Logstash Output:  
{  
"message" =\> "hello world\r",  
"@version" =\> "1",  
"@timestamp" =\> "2016-02-17T00:42:35.846Z",  
"host" =\> "0:0:0:0:0:0:0:1",  
"port" =\> 61853,  
"my\_env" =\> "%{[@metadata][region]}",  
"@metadata" =\> {  
"region" =\> nil  
}  
}

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [February 17, 2016, 4:02pm UTC](https://discuss.elastic.co/t/environment-filter-is-not-working/42040/3 "2016-02-17T16:02:37Z")

</div>

Works for me. Are you setting the REGION\_ENV variable in the same shell from which you're invoking Logstash?

```nohighlight
$ cat test.config
input { stdin { } }
output { stdout { codec => rubydebug { metadata => true } } }
filter {
  environment {
    add_metadata_from_env => {"region" => "REGION_ENV"}
    add_field => ["my_env", "%{[@metadata][region]}"]
  }
}
$ echo 'hello' | REGION_ENV=foo /opt/logstash/bin/logstash -f test.config
Settings: Default pipeline workers: 8
Logstash startup completed
{
       "message" => "hello",
      "@version" => "1",
    "@timestamp" => "2016-02-17T16:01:09.963Z",
          "host" => "lnxolofon",
        "my_env" => "foo",
     "@metadata" => {
        "region" => "foo"
    }
}
Logstash shutdown completed

```

---

<div class="post-metadata">

**Author:** ![gaurav1424](https://avatars.discourse-cdn.com/v4/letter/g/97f17d/32.png) [@gaurav1424](https://discuss.elastic.co/u/gaurav1424)\
**Post date:** [February 17, 2016, 7:18pm UTC](https://discuss.elastic.co/t/environment-filter-is-not-working/42040/4 "2016-02-17T19:18:46Z")

</div>

Thanks a lot @magnusbaek. Appreciate your quick help. I finally got it working. I was starting logstash as sudo but setting environment variable in normal shell.

anyways, How do I remove metadata fields in the parsing key value pairs ?

I get something like this :

"region" =\> "amer",  
"metro" =\> "sj",  
"ibx" =\> "sv3",  
"@metadata" =\> {  
"region" =\> "amer",  
"metro" =\> "sj",  
"ibx" =\> "sv3"

I dont want this to be part of my output :

"@metadata" =\> {  
"region" =\> "amer",  
"metro" =\> "sj",  
"ibx" =\> "sv3"

I did this but its not working :

filter {  
environment {  
add\_metadata\_from\_env =\> {"region" =\> "REGION\_ENV"}  
add\_metadata\_from\_env =\> {"metro" =\> "METRO\_ENV"}  
add\_metadata\_from\_env =\> {"ibx" =\> "IBX\_ENV"}  
add\_field =\> ["region", "%{[@metadata][region]}"]  
add\_field =\> ["metro", "%{[@metadata][metro]}"]  
add\_field =\> ["ibx", "%{[@metadata][ibx]}"]  
remove\_field =\> ["@metadata"]  
}  
}

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [February 18, 2016, 7:01am UTC](https://discuss.elastic.co/t/environment-filter-is-not-working/42040/5 "2016-02-18T07:01:35Z")

</div>

The contents of the `@metadata` field aren't included by outputs. One exception is if you use the rubydebug codec and explicitly enable inclusion of those fields as I did in my example above.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:10am UTC](https://discuss.elastic.co/t/environment-filter-is-not-working/42040/6 "2017-07-06T05:10:56Z")

</div>