# Logstash grok filter partial working

**URL:** <https://discuss.elastic.co/t/logstash-grok-filter-partial-working/166758>\
**Category:** Logstash\
**Created:** [February 1, 2019, 4:03pm UTC](https://discuss.elastic.co/t/logstash-grok-filter-partial-working/166758 "2019-02-01T16:03:12Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Rocky\_RK](https://avatars.discourse-cdn.com/v4/letter/r/ed655f/32.png) [@Rocky\_RK](https://discuss.elastic.co/u/Rocky_RK)\
**Post date:** [February 1, 2019, 4:03pm UTC](https://discuss.elastic.co/t/logstash-grok-filter-partial-working/166758/1 "2019-02-01T16:03:13Z")

</div>

This is just related to the [post](https://stackoverflow.com/questions/54142085/how-to-write-custom-grok-for-logstash/54161210) where i'm using the below grok filter to dissect the data to be visualize into kibana, below is what i'm using into my logstash conf file and working for the data as desired but today I got into a situation where its not filtering the data as desired.

Correct visual at Kibana are like:

> **received\_at** :February 1st 2019, 21:00:04.105 float:0.5, 0.0 **type** :rmlog **Hostname** :dba- foxon93 Date:19/02/01 **User\_1** :dv\_vxehw **@version** :1 **Hour\_since** :06 **Command** :rm -rf /data/rg/log

The above Fields are not visible for the data i have shown in the post at the end.

**grok filter in logstash conf file:**

```
 match => { "message" => "%{HOSTNAME:Hostname},%{DATE:Date},%{HOUR:Hour_since}:%{MINUTE:Mins_since},%{NUMBER}-%{WORD},%{USER:User_1},%{USER:User_2} %{NUMBER:Pid} %{NUMBER:float} %{NUMBER:float} %{NUMBER:Num_1} %{NUMBER:Num_2} %{DATA} %{HOUR:hour2}:%{MINUTE:minute2} %{HOUR:hour3}:%{MINUTE:minute3} %{GREEDYDATA:Command}" }

```

## My logstash conf file:

```
input {
  file {
    path => ["/data/mylogs/*.txt"]
    start_position => beginning
    sincedb_path => "/dev/null"
    type => "tac"
  }
}

filter {
  if [type] == "tac" {
    grok {
      match => { "message" => "%{HOSTNAME:Hostname},%{DATE:Date},%{HOUR:Hour_since}:%{MINUTE:Mins_since},%{NUMBER}-%{WORD},%{USER:User_1},%{USER:User_2} %{NUMBER:Pid} %{NUMBER:float} %{NUMBER:float} %{NUMBER:Num_1} %{NUMBER:Num_2} %{DATA} %{HOUR:hour2}:%{MINUTE:minute2} %{HOUR:hour3}:%{MINUTE:minute3} %{GREEDYDATA:Command}" }
      add_field => ["received_at", "%{@timestamp}"]
      remove_field => ["@version", "host", "message", "_type", "_index", "_score"]
   }
 }
}

output {
        if [type] == "rmlog" {
        elasticsearch {
                hosts => ["localhost:9200"]
                manage_template => false
                index => "tac-%{+YYYY.MM.dd}"
  }
 }
}

```

Below is the new data which is getting processed but i'm not getting the **Hostname** , **Command** etc fields for this data.

```
dbproj01,19/02/01,00:04,23-hrs,cvial,cvial 120804 0.0 0.0 106096 1200 pts/90 S Jan30 0:00 /bin/sh -c /bin/rm -f ../../../../../../tools.lnx86/dfII/etc/context/64bit/hBrowser.cxt ../../../../../../
tools.lnx86/dfII/etc/context/64bit/hBrowser.toc ../../../../../../tools.lnx86/dfII/etc/context/64bit/hBrowser.aux ../../../../../../tools.lnx86/dfII/etc/context/64bit/hBrowser.ini ; (CUR_DIR=`pwd` ;
 cd ../../../../obj/linux-x86-64/optimize/bin/virtuoso ; ${CUR_DIR}/../../../../../../tools.lnx86/dfII/bin/virtuoso -ilLoadIL hBrowserBuildContext.il -log hBrowserBuildContext.log -nograph && [ `/bi
n/grep -c Error hBrowserBuildContext.log` = 0 ]) || (echo '*** Error: Failed to build hBrowser context.' ; /bin/rm -f ../../../../../../tools.lnx86/dfII/etc/context/64bit/hBrowser.cxt ../../../../..
/../tools.lnx86/dfII/etc/context/64bit/hBrowser.toc ../../../../../../tools.lnx86/dfII/etc/context/64bit/hBrowser.aux ../../../../../../tools.lnx86/dfII/etc/context/64bit/hBrowser.ini ; exit 1),/pro
j/cvial/WS/BUNGEE/REBASE_190120-138_2/tools.lnx86/dfII/group/bin/src

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [February 1, 2019, 7:19pm UTC](https://discuss.elastic.co/t/logstash-grok-filter-partial-working/166758/2 "2019-02-01T19:19:06Z")

</div>

Jan30 0:00 does not match %{HOUR:hour2}:%{MINUTE:minute2} %{HOUR:hour3}:%{MINUTE:minute3}

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 1, 2019, 7:19pm UTC](https://discuss.elastic.co/t/logstash-grok-filter-partial-working/166758/3 "2019-03-01T19:19:09Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.