# Mutate in CSV filter not working with stdin

**URL:** <https://discuss.elastic.co/t/mutate-in-csv-filter-not-working-with-stdin/124987>\
**Category:** Logstash\
**Created:** [March 21, 2018, 12:46pm UTC](https://discuss.elastic.co/t/mutate-in-csv-filter-not-working-with-stdin/124987 "2018-03-21T12:46:17Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![kevkos236](https://avatars.discourse-cdn.com/v4/letter/k/90ced4/32.png) [@kevkos236](https://discuss.elastic.co/u/kevkos236)\
**Post date:** [March 21, 2018, 12:46pm UTC](https://discuss.elastic.co/t/mutate-in-csv-filter-not-working-with-stdin/124987/1 "2018-03-21T12:46:18Z")

</div>

Hello there,

we are using logstash to import data from csv files into elasticsearch. The logstash config worked so far but I was now trying to input the CSV files with `stdin` instead of `file`. The `stdin` input works but I'm performing some `mutate`, `add_field` and `update` on several columns and those actions do not work anymore with the stdin pipe.

Here's my example config:

```
input {
  stdin {
    id => "${LS_FILE}"
  }   
}
filter {
    mutate {
        add_field => { "kvasy_type" => "${KVASY_TYPE}" }
    }
    
    if "${LS_FILE}" == "xyz.csv" {
      csv {
          separator => ";"
          columns =>
          [
            "name1",
            "name2",
            "name3",
            "OID"           
          ]
      }
	    mutate {
          add_field => {
              "searchfield" => "%{name1} %{name2} %{name3}"
			}
		}
		if [OID]{
		mutate {
			update => { "OID" => "n-%{OID}" }
		}
		}
    }
}
output {                                                
  stdout {codec=>"json_lines"}
  file{
    path => "/home/kevin/output.csv"
    codec => "json_lines"
  }
}

```

That's just a snippet of the the complete config, I pipe several csv into logstash and set environment variables before to set the specifics column filters.

I start logstash with`./logstash -f cfg.config < xyz.csv`. The output contains only the columns of the original csv, any mutate action seems not to be processed. I also tried to auto detect the columns with no change.

Does anyboy know why this doesn't work as desired?

Regards KK

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [March 22, 2018, 9:05pm UTC](https://discuss.elastic.co/t/mutate-in-csv-filter-not-working-with-stdin/124987/2 "2018-03-22T21:05:24Z")

</div>

Is the LS\_FILE variable set to xyz.csv in this case?

---

<div class="post-metadata">

**Author:** ![kevkos236](https://avatars.discourse-cdn.com/v4/letter/k/90ced4/32.png) [@kevkos236](https://discuss.elastic.co/u/kevkos236)\
**Post date:** [March 23, 2018, 8:01am UTC](https://discuss.elastic.co/t/mutate-in-csv-filter-not-working-with-stdin/124987/3 "2018-03-23T08:01:16Z")

</div>

Yes I set it with `export Ls_FILE=xyz.csv`. The problem however was that logstash is not capable of using environment variables in conditional clauses. I bypassed this by writing the variable into a metadata field and then using this field i the if-clause.

Like this:

```
mutate {
        add_field => { "[@metadata][LS_FILE]" => "${LS_FILE}"}
    }
    if [@metadata][LS_FILE] == "xyz.csv" {
      csv {
          .......
      }
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 20, 2018, 8:01am UTC](https://discuss.elastic.co/t/mutate-in-csv-filter-not-working-with-stdin/124987/4 "2018-04-20T08:01:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.