# Parse with different grok filter problem

**URL:** <https://discuss.elastic.co/t/parse-with-different-grok-filter-problem/147387>\
**Category:** Logstash\
**Created:** [September 5, 2018, 11:28am UTC](https://discuss.elastic.co/t/parse-with-different-grok-filter-problem/147387 "2018-09-05T11:28:08Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![dyl](https://avatars.discourse-cdn.com/v4/letter/d/5fc32e/32.png) [@dyl](https://discuss.elastic.co/u/dyl)\
**Post date:** [September 5, 2018, 11:28am UTC](https://discuss.elastic.co/t/parse-with-different-grok-filter-problem/147387/1 "2018-09-05T11:28:08Z")

</div>

Hello everybody, I want to parse these events :

```
INFO;0000;000003;data/EDT/batchs/files/logs/REL015MT-20180807-20180807-2004.log

INFO;0000;000003; /data/EDT/batchs/files/logs/WKF997-20180904-20180904-2029-42.log
INFO;0000;000003; /data/EDT/batchs/file/logs/WKF997-20180611-20180611-2035-42-9000-01-20180611-203557.log
INFO;0000;000003; /data/EDT/batchs/files/logs/WKF999-20180904-20180904-2003-40.log
INFO;0000;000003; /data/EDT/batchs/file/logs/WKF999-20180904-20180904-1958-20-42-2297-1168-0-1705969074-20180904-200117.log

INFO;0000;000003; /data/EDT/batchs/files/logs/E-0077MT-20180810-20180810-2317.log

```

And to parse these logs I use 3 filter grok different, in same order than above :

```
%{WORD:NOM_BATCH}-%{BASE16NUM:DATE_JOURNEE_BATCH}-%{BASE16NUM:DATE_EXECUTION_BATCH}-%{BASE16NUM:HEURE_EXECUTION_BATCH}.log

%{WORD:NOM_BATCH}-%{BASE16NUM:DATE_JOURNEE_BATCH}-%{BASE16NUM:DATE_EXECUTION_BATCH}-%{BASE16NUM:HEURE_EXECUTION_BATCH}-%{GREEDYDATA:SUITE}.log

(?<NOM_BATCH>E-[0-9A-Z]+)-%{BASE16NUM:DATE_JOURNEE_BATCH}-%{BASE16NUM:DATE_EXECUTION_BATCH}-%{BASE16NUM:HEURE_EXECUTION_BATCH}.log

```

So all these grok works fine but I want to group all these filter to filter all event.

For exemple if the first is grok parse failure, pass to the next, etc ...

I tried this :

```
filter
{
  grok {
    match => { "message" => ["%{WORD:TYPE};%{DATA:ID1};%{NUMBER:ID2};%{GREEDYDATA:DESCRIPTION}"] }
  }
  
  if ([DESCRIPTION] =~ "\/data\/EDT\/batchs\/files\/logs\/[A-Za-z0-9_.]+.+.log")
  {
    grok {
      match => { "DESCRIPTION" => ["%{WORD:NOM_BATCH}-%{BASE16NUM:DATE_JOURNEE_BATCH}-%{BASE16NUM:DATE_EXECUTION_BATCH}-%{BASE16NUM:HEURE_EXECUTION_BATCH}.log"] }
      add_field => { "PATH_BATCH" => "%{NOM_BATCH}-%{DATE_JOURNEE_BATCH}-%{DATE_EXECUTION_BATCH}-%{HEURE_EXECUTION_BATCH}.log" }
    }
    grok {
      match => { "DESCRIPTION" => ["%{WORD:NOM_BATCH}-%{BASE16NUM:DATE_JOURNEE_BATCH}-%{BASE16NUM:DATE_EXECUTION_BATCH}-%{BASE16NUM:HEURE_EXECUTION_BATCH}-%{GREEDYDATA:SUITE}.log"] }
      add_field => { "PATH_BATCH" => "%{NOM_BATCH}-%{DATE_JOURNEE_BATCH}-%{DATE_EXECUTION_BATCH}-%{HEURE_EXECUTION_BATCH}-%{SUITE}.log" }
    }
    grok {
      match => { "DESCRIPTION" => ["(?<NOM_BATCH>E-[0-9A-Z]+)-%{BASE16NUM:DATE_JOURNEE_BATCH}-%{BASE16NUM:DATE_EXECUTION_BATCH}-%{BASE16NUM:HEURE_EXECUTION_BATCH}.log" ] }
      add_field => { "PATH_BATCH" => "%{NOM_BATCH}-%{DATE_JOURNEE_BATCH}-%{DATE_EXECUTION_BATCH}-%{HEURE_EXECUTION_BATCH}.log" }
    }
  }
}

```

but it doesn't work as expected..

Can somebody help me ? ty

---

<div class="post-metadata">

**Author:** ![dyl](https://avatars.discourse-cdn.com/v4/letter/d/5fc32e/32.png) [@dyl](https://discuss.elastic.co/u/dyl)\
**Post date:** [September 5, 2018, 11:46am UTC](https://discuss.elastic.co/t/parse-with-different-grok-filter-problem/147387/3 "2018-09-05T11:46:47Z")

</div>

Thx u for answer but I need to add a field specific to a grok filter so I can't group all these filter into 1 pattern

---

<div class="post-metadata">

**Author:** ![abhiroyg](https://avatars.discourse-cdn.com/v4/letter/a/e9bcb4/32.png) [@abhiroyg](https://discuss.elastic.co/u/abhiroyg)\
**Post date:** [September 5, 2018, 1:25pm UTC](https://discuss.elastic.co/t/parse-with-different-grok-filter-problem/147387/4 "2018-09-05T13:25:28Z")

</div>

Parsing is not done properly I guess, please check out the following conf

```
input {
    file {
        path => ["~/data.log"]
        start_position => "beginning"
    }
}

filter {
    grok {
        match => { "message" => "%{WORD:TYPE};%{DATA:ID1};%{NUMBER:ID2};%{GREEDYDATA:DESCRIPTION}" }
    }

    if ([DESCRIPTION] =~ "data\/EDT\/batchs\/files\/logs\/[A-Za-z0-9_.]+.+.log")
    {
        grok {
            match => { "DESCRIPTION" => "^data\/EDT\/batchs\/files\/logs\/%{WORD:NOM_BATCH}-%{BASE16NUM:DATE_JOURNEE_BATCH}-%{BASE16NUM:DATE_EXECUTION_BATCH}-%{BASE16NUM:HEURE_EXECUTION_BATCH}.log$" }
            add_field => { "PATH_BATCH" => "1" }
        }
        if (![PATH_BATCH]) {
            grok {
                match => { "DESCRIPTION" => "^ \/data\/EDT\/batchs\/files\/logs\/%{WORD:NOM_BATCH}-%{BASE16NUM:DATE_JOURNEE_BATCH}-%{BASE16NUM:DATE_EXECUTION_BATCH}-%{BASE16NUM:HEURE_EXECUTION_BATCH}-%{GREEDYDATA:SUITE}.log$" }
                add_field => { "PATH_BATCH" => "2" }
            }
        }
        if (![PATH_BATCH]) {
            grok {
                match => { "DESCRIPTION" => "^ \/data\/EDT\/batchs\/files\/logs\/(?<NOM_BATCH>E-[0-9A-Z]+)-%{BASE16NUM:DATE_JOURNEE_BATCH}-%{BASE16NUM:DATE_EXECUTION_BATCH}-%{BASE16NUM:HEURE_EXECUTION_BATCH}.log$" }
                add_field => { "PATH_BATCH" => "3" }
            }
        }
    }
}

output {
    stdout { codec => rubydebug }
}

```

I was not able to find out why GREEDYDATA is not reading the super long stuff or if it's failing in `if condition` itself for that `WKF<super long>.log` lines.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 3, 2018, 1:25pm UTC](https://discuss.elastic.co/t/parse-with-different-grok-filter-problem/147387/5 "2018-10-03T13:25:43Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.