# File beat --\> logtash --\> conditional output

**URL:** <https://discuss.elastic.co/t/file-beat-logtash-conditional-output/155792>\
**Category:** Logstash\
**Created:** [November 7, 2018, 8:46pm UTC](https://discuss.elastic.co/t/file-beat-logtash-conditional-output/155792 "2018-11-07T20:46:27Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![rajsolanki](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rajsolanki/32/86859_2.png) [@rajsolanki](https://discuss.elastic.co/u/rajsolanki)\
**Post date:** [November 7, 2018, 8:46pm UTC](https://discuss.elastic.co/t/file-beat-logtash-conditional-output/155792/1 "2018-11-07T20:46:27Z")

</div>

Hi

I did research in this forum and documentation but couldn't find any solution so posting here for help.

OS : windows 10  
Filebeat : 6.4.2 sending 2 CSV files to Logstash  
File name London.csv , Australia.csv

Logstash config has following for output.

output {  
elasticsearch {  
if ([source] =~ \London.csv)  
{  
hosts =\> "[http://myESSErver](http://myESSErver):port"  
manage\_template =\> true  
index =\> "London-%{+YYYY.MM.dd}"

```
 }

```

if ([source] =~ \Australia.csv)  
{  
hosts =\> "[http://myESSErver](http://myESSErver):port"  
manage\_template =\> true  
index =\> "Australia-%{+YYYY.MM.dd}"

```
 }  

```

}

When i start my logstash I get error like

[2018-11-07T13:56:16,779][ERROR][logstash.agent] Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, :exception=\>"LogStash::ConfigurationError", :message=\>"Expected one of #, =\> at line 15, column 7 (byte 2230) after output {\n elasticsearch {\n if "

So am i doing something which i m not supposed to ?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [November 7, 2018, 8:47pm UTC](https://discuss.elastic.co/t/file-beat-logtash-conditional-output/155792/2 "2018-11-07T20:47:58Z")

</div>

> [@rajsolanki](#):
>
> if ([source] =~ \London.csv)

> [@rajsolanki](#):
>
> if ([source] =~ \Australia.csv)

You need a closing backslash for all regular expressions.

---

<div class="post-metadata">

**Author:** ![rajsolanki](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rajsolanki/32/86859_2.png) [@rajsolanki](https://discuss.elastic.co/u/rajsolanki)\
**Post date:** [November 7, 2018, 9:16pm UTC](https://discuss.elastic.co/t/file-beat-logtash-conditional-output/155792/3 "2018-11-07T21:16:59Z")

</div>

so i have tried

if ([source] =~ "\London.csv\")

if ([source] =~ \London.csv\)

and both time it failed.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [November 7, 2018, 9:30pm UTC](https://discuss.elastic.co/t/file-beat-logtash-conditional-output/155792/4 "2018-11-07T21:30:38Z")

</div>

Can you show us what the event and the source field look like?

---

<div class="post-metadata">

**Author:** ![rajsolanki](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rajsolanki/32/86859_2.png) [@rajsolanki](https://discuss.elastic.co/u/rajsolanki)\
**Post date:** [November 7, 2018, 9:40pm UTC](https://discuss.elastic.co/t/file-beat-logtash-conditional-output/155792/5 "2018-11-07T21:40:18Z")

</div>

sure give me 5 mins.

---

<div class="post-metadata">

**Author:** ![rajsolanki](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rajsolanki/32/86859_2.png) [@rajsolanki](https://discuss.elastic.co/u/rajsolanki)\
**Post date:** [November 7, 2018, 9:53pm UTC](https://discuss.elastic.co/t/file-beat-logtash-conditional-output/155792/6 "2018-11-07T21:53:52Z")

</div>

Not diverting topic but just wanted to share this with you first.

So if i try following

input {  
beats {  
port =\> "5045"  
}  
}  
filter {  
csv {  
separator =\> ","  
columns =\> ["Filed1" , "Filed2" , "Filed3"]  
}

}  
output {  
elasticsearch {

```
        hosts => "http://myesserver:port"
        manage_template => true
        index => "London-%{+YYYY.MM.dd}"

```

}  
stdout {}  
}

my logstash agent starts up fine.

So i stop the logstash and add conditional statement

input {  
beats {  
port =\> "5045"  
}  
}  
filter {  
csv {  
separator =\> ","  
columns =\> ["Filed1" , "Filed2" , "Filed3"]  
}

if ([source] =~ \London.csv\)  
{  
add\_field =\> ["Location", "London"]  
}

}  
output {  
elasticsearch {

```
        hosts => "http://myesserver:port"
        manage_template => true
        index => "London-%{+YYYY.MM.dd}"

```

}  
stdout {}  
}

Error

[2018-11-07T15:53:13,144][ERROR][logstash.agent] Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, :exception=\>"LogStash::ConfigurationError", :message=\>"Expected one of #, ", ', / at line 12, column 19 (byte 167) after filter {\n csv {\n separator =\> ","\n columns =\> ["Filed1" , "Filed2" , "Filed3"]\n }\n \n if ([source] =~ ",

---

<div class="post-metadata">

**Author:** ![rajsolanki](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rajsolanki/32/86859_2.png) [@rajsolanki](https://discuss.elastic.co/u/rajsolanki)\
**Post date:** [November 7, 2018, 10:14pm UTC](https://discuss.elastic.co/t/file-beat-logtash-conditional-output/155792/7 "2018-11-07T22:14:33Z")

</div>

another update.

first my IF had wrong syntax i think.

so when do this

input {  
beats {  
port =\> "5045"  
}  
}  
filter {  
csv {  
separator =\> ","  
columns =\> ["Filed1" , "Filed2" , "Filed3"]  
}

if [source] == "C:\mycsv\London.csv"  
{

```
   }  

```

}  
output {  
elasticsearch {

```
        hosts => "http://127.0.0.1:9200"
        manage_template => true
        index => "London-%{+YYYY.MM.dd}"

```

}  
stdout {}  
}

My logstash starts u fine.

But if i add this

if [source] == "C:\mycsv\London.csv"  
{  
add\_field =\> ["Location", "London"]  
}

it fails.

[2018-11-07T16:05:39,470][ERROR][logstash.agent] Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, :exception=\>"LogStash::ConfigurationError", :message=\>"Expected one of #, { at line 14, column 24 (byte 271) after filter {\n csv {\n separator =\> ","\n columns =\> ["Filed1" , "Filed2" , "Filed3"]\n }\n \n if [source] == "C:\mycsv\London.csv"\n {\n add\_field ",

---

<div class="post-metadata">

**Author:** ![rajsolanki](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rajsolanki/32/86859_2.png) [@rajsolanki](https://discuss.elastic.co/u/rajsolanki)\
**Post date:** [November 7, 2018, 10:23pm UTC](https://discuss.elastic.co/t/file-beat-logtash-conditional-output/155792/8 "2018-11-07T22:23:12Z")

</div>

Sorry for posting too many updates 🙂

add\_field needs to be in mutate so my issue is fixed. Will update if ingestion of data works without any issues.

thanks for all the help

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 5, 2018, 10:25pm UTC](https://discuss.elastic.co/t/file-beat-logtash-conditional-output/155792/9 "2018-12-05T22:25:55Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.