# Issues with Logstash Conditionals inside filter

**URL:** <https://discuss.elastic.co/t/issues-with-logstash-conditionals-inside-filter/176322>\
**Category:** Logstash\
**Created:** [April 11, 2019, 5:47am UTC](https://discuss.elastic.co/t/issues-with-logstash-conditionals-inside-filter/176322 "2019-04-11T05:47:32Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Dilruwan\_Madubashi10](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dilruwan_madubashi10/32/43903_2.png) [@Dilruwan\_Madubashi10](https://discuss.elastic.co/u/Dilruwan_Madubashi10)\
**Post date:** [April 11, 2019, 5:47am UTC](https://discuss.elastic.co/t/issues-with-logstash-conditionals-inside-filter/176322/1 "2019-04-11T05:47:32Z")

</div>

I have been trying to get a CSV file parsed into logstash and it works fine. How ever I need to add either a new tag or a new field based on certain conditions.

I have validated that in the logs I have the text that matches the conditions (Below). but it doesn't add any tags nor fields based on my condition. Any help will be much appreciated. (Both the commented section of conditions and uncommitted simpler one doesn't seem to make a difference)

input  
{  
file {  
path =\> "C:/ELK/Data\_Landing/\*.csv"  
start\_position =\> "beginning"  
sincedb\_path =\> "NUL"  
}  
}

filter  
{  
csv{  
separator =\> ","  
skip\_header=\> true  
skip\_empty\_columns =\> true  
autogenerate\_column\_names =\>false  
#31-01-2019 10:13  
columns =\> ["Severity" , "AlertReceived" , "Node" , "Application" , "MessageGroup" , "Object" , "TemplateName" , "ConditionMatched" , "MessageText" , "OpsAck"]  
}

```
            #if [Node] =~ /^"tm"*/ or [Node] =~ /^"tq"*/ or [Node] =~ /^"bp"*/ or [Node] =~ /^"ob"*/ or [Node] =~ /^"le"*/ 
            #{
            # mutate { add_field => "ProductType" => "EXXXXX"}
            #}
            #else if [Node] =~ /^dv*/ AND [Application] != "XXXX"
            #{
            # mutate { add_field => "ProductType" => "SXXX"}
            #}
            # else 
            # {
            # mutate { add_tag => "Undefined"}
            # }
            
            if [Node] == "X-XXXX-XXXX.XX.XXXX.XXXX.local"
            { 
                            mutate { add_field => "ProductType" => "SXXX"}
            }
            
            mutate{
                            gsub => ["AlertReceived", "/", "-"]
                            #gsub => ["AlertReceived", " ", ";"]
            }
            
            date {
                            match => ["AlertReceived", "dd-MM-yyyy HH:mm:ss"]
                            target => "@timestamp"
            }

```

}

output  
{  
elasticsearch  
{  
hosts =\> "localhost:9200"  
index =\> "ito"  
document\_type =\> "csv"  
}  
stdout { codec =\> rubydebug }  
}

Sample Messages

---

<div class="post-metadata">

**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\
**Post date:** [April 11, 2019, 1:13pm UTC](https://discuss.elastic.co/t/issues-with-logstash-conditionals-inside-filter/176322/2 "2019-04-11T13:13:18Z")

</div>

It would help to have sample data.

I usually debug these sort of issues using the generator input.

```auto
input {
  generator {
    lines => [
      "stop,wait for it,go go go - man",
      "halt,hesitate,hurry along"
    ]
    count => 1
  }
}

filter {
  csv {
    separator => ","
    columns => ["red","amber","green"]
  }
  if [red] == "halt" {
    mutate {
      add_field => { "[intonation]" => "posh"}
    }
  } else {
    mutate {
      add_field => { "[intonation]" => "hipster"}
    }
  }
}

output {
  stdout {
    codec => rubydebug
  }
}

```

Gives

```auto
{
         "green" => "hurry along",
       "message" => "halt,hesitate,hurry along",
           "red" => "halt",
      "sequence" => 0,
      "@version" => "1",
          "host" => "Elastics-MacBook-Pro.local",
    "@timestamp" => 2019-04-11T13:10:21.913Z,
         "amber" => "hesitate",
    "intonation" => "posh"
}
{
         "green" => "go go go - man",
       "message" => "stop,wait for it,go go go - man",
           "red" => "stop",
      "sequence" => 0,
      "@version" => "1",
          "host" => "Elastics-MacBook-Pro.local",
    "@timestamp" => 2019-04-11T13:10:21.896Z,
         "amber" => "wait for it",
    "intonation" => "hipster"
}

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 11, 2019, 1:48pm UTC](https://discuss.elastic.co/t/issues-with-logstash-conditionals-inside-filter/176322/3 "2019-04-11T13:48:40Z")

</div>

> [@Dilruwan\_Madubashi10](#):
>
> if [Node] =~ /^"tm"\*/

It makes my head explode that that works the way it does in a logstash conditional. I know Onigurama, ed, awk, perl, and csh all have different regexp syntaxes, but I thought I understood logstash. Apparently not.

---

<div class="post-metadata">

**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\
**Post date:** [April 11, 2019, 2:03pm UTC](https://discuss.elastic.co/t/issues-with-logstash-conditionals-inside-filter/176322/4 "2019-04-11T14:03:37Z")

</div>

I'm not sure it works as expected. The `*` asserts zero or more `"` characters meaning it is effectively `/^"tm/`

> **[Online regex tester and debugger: PHP, PCRE, Python, Golang and JavaScript](https://regex101.com/r/pLMdMP/1)**
>
> Regex101 allows you to create, debug, test and have your expressions explained for PHP, PCRE, Python, Golang and JavaScript. The website also features a community where you can share useful expressions.

---

<div class="post-metadata">

**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\
**Post date:** [April 11, 2019, 2:06pm UTC](https://discuss.elastic.co/t/issues-with-logstash-conditionals-inside-filter/176322/5 "2019-04-11T14:06:45Z")

</div>

@Dilruwan_Madubashi10

The `==` equality operator will fail if there is leading or trailing whitespace.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 11, 2019, 3:04pm UTC](https://discuss.elastic.co/t/issues-with-logstash-conditionals-inside-filter/176322/6 "2019-04-11T15:04:05Z")

</div>

> [@guyboertje](#):
>
> The `*` asserts zero or more `"` characters meaning it is effectively `/^"tm/`

For some reason I read the \* as +. Head now un-exploded.

---

<div class="post-metadata">

**Author:** ![Dilruwan\_Madubashi10](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dilruwan_madubashi10/32/43903_2.png) [@Dilruwan\_Madubashi10](https://discuss.elastic.co/u/Dilruwan_Madubashi10)\
**Post date:** [April 12, 2019, 1:48pm UTC](https://discuss.elastic.co/t/issues-with-logstash-conditionals-inside-filter/176322/7 "2019-04-12T13:48:00Z")

</div>

| Severity | Alert Received | Node |
| --- | --- | --- |
| normal | 28/02/2019-23:59:38 | paXXXXXX |
| minor | 28/02/2019-23:59:20 | [sp-XXX-XXX.columbus.stockex.com](http://sp-XXX-XXX.columbus.stockex.com) |
| minor | 28/02/2019-23:59:16 | paXXXXXX |
| normal | 28/02/2019-23:59:16 | paXXXXXX |
| minor | 28/02/2019-23:59:13 | paXXXXXX |
| normal | 28/02/2019-23:59:11 | paXXXXXX |
| normal | 28/02/2019-23:59:11 | paXXXXXX |
| normal | 28/02/2019-23:59:02 | paXXXXXX |
| normal | 28/02/2019-23:56:18 | [op-XXXX-CCC01.XXX-ops.abc.com](http://op-XXXX-CCC01.XXX-ops.abc.com) |
| major | 28/02/2019-23:56:12 | [dv-XXXX-CCC01.XXX-ops.abc.com](http://dv-XXXX-CCC01.XXX-ops.abc.com) |
| major | 28/02/2019-23:55:03 | op--XXXX-CCC01.XXX-ops.abc.com |

Sample data would be like this in a csv. I have made some changes as these are from a production environment. 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 10, 2019, 1:48pm UTC](https://discuss.elastic.co/t/issues-with-logstash-conditionals-inside-filter/176322/8 "2019-05-10T13:48:00Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
