# Grok filter

**URL:** <https://discuss.elastic.co/t/grok-filter/269194>\
**Category:** Logstash\
**Created:** [April 4, 2021, 5:10pm UTC](https://discuss.elastic.co/t/grok-filter/269194 "2021-04-04T17:10:17Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![Dinesh\_Sharma](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dinesh_sharma/32/86511_2.png) [@Dinesh\_Sharma](https://discuss.elastic.co/u/Dinesh_Sharma)\
**Post date:** [April 4, 2021, 5:10pm UTC](https://discuss.elastic.co/t/grok-filter/269194/1 "2021-04-04T17:10:17Z")

</div>

Hi,

I have a csv file which has currently three rows in it which are as follow:

name,age,ip  
A,12,10.11.1.12  
B,13,10.11.1

Now I want to check during the data ingestion via this csv that whether the IP is in proper format or not using grok.  
If it is in proper format then message field =IP  
else message field should contain improper IP and and text that "ip is improper"  
Can you help please help me with the code.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 4, 2021, 6:54pm UTC](https://discuss.elastic.co/t/grok-filter/269194/2 "2021-04-04T18:54:50Z")

</div>

I would use a csv filter to split [message] into three fields ([name], [age], [ip]).  
You can then do

```
grok { match => { "ip" => "^%{IPV4}$" } }

```

and make a decision on what to do based on

```
if "_grokparsefailure" in [tags] { ...
```

---

<div class="post-metadata">

**Author:** ![Dinesh\_Sharma](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dinesh_sharma/32/86511_2.png) [@Dinesh\_Sharma](https://discuss.elastic.co/u/Dinesh_Sharma)\
**Post date:** [April 5, 2021, 7:38am UTC](https://discuss.elastic.co/t/grok-filter/269194/3 "2021-04-05T07:38:34Z")

</div>

Hi,

Here is my code. Please assist.

````auto
file{
path=>"/home/myuser/test/new.csv"
}
}
filter {
csv
{
columns => ["name","Age","IP"]
}

grok {
match => {"%{IP:validIP}" }
}
if "_grokparsefailure" in [tags] { 

*********want to create a new field like "isvalue_correct" in each document with value as "false"**** 
}

output{
elasticsearch {
    hosts => ["http://x.x.x.:9200"]
    index => "mytest_index"
    user => "xxxxxx"
    password => "xxxxxx"
}
}```
````

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [April 5, 2021, 2:33pm UTC](https://discuss.elastic.co/t/grok-filter/269194/4 "2021-04-05T14:33:38Z")

</div>

Just use mutate and add\_field see [here](https://www.elastic.co/guide/en/logstash/current/plugins-filters-mutate.html#plugins-filters-mutate-add_field)

---

<div class="post-metadata">

**Author:** ![Dinesh\_Sharma](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dinesh_sharma/32/86511_2.png) [@Dinesh\_Sharma](https://discuss.elastic.co/u/Dinesh_Sharma)\
**Post date:** [April 6, 2021, 6:24am UTC](https://discuss.elastic.co/t/grok-filter/269194/5 "2021-04-06T06:24:53Z")

</div>

Hi,

I tried below conf:

````auto
file
{
path=>"/home/myuser/test/new.csv"
start_position => "beginning"
}
}
filter
{
csv
{
columns => ["name","Age","IP"]
}
grok
{
match => {"%{IP:validIP}" }
}
if "_grokparsefailure" in [tags] {

      mutate {
        add_field => { "is_value_correct" => "false" }
      }
}
else
{
      mutate
 {
        add_field => { "is_value_correct" => "true" }
      }
}

output{
elasticsearch {
    hosts => ["http://x.x.x.x:9200"]
    index => "mytest_index"
    user => "xxxx"
    password => "xxxxxx"
}
}```

Error : 

````

[ERROR] 2021-04-06 06:20:33.936 [Converge PipelineAction::Create] agent - Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, :exception=\>"LogStash::ConfigurationError", :message=\>"Expected one of [\t\r\n], "#", "=\>" at line 16, column 27 (byte 166) after filter\n{\ncsv\n{\ncolumns =\> ["name","Age","IP"]\n}\ngrok \n{\nmatch =\> {"%{IP:validIP}" ", :backtrace=\>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:32:in `compile_imperative'", "org/logstash/execution/AbstractPipelineExt.java:184:in `initialize'", "org/logstash/execution/JavaBasePipelineExt.java:69:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:47:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline\_action/create.rb:52:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:367:in `block in converge\_state'"]}  
[INFO] 2021-04-06 06:20:34.169 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=\>9600}  
[INFO] 2021-04-06 06:20:39.231 [LogStash::Runner] runner - Logstash shut down.

```auto

Any help on this.
```

---

<div class="post-metadata">

**Author:** ![Cad](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cad/32/86661_2.png) [@Cad](https://discuss.elastic.co/u/Cad)\
**Post date:** [April 6, 2021, 7:58am UTC](https://discuss.elastic.co/t/grok-filter/269194/6 "2021-04-06T07:58:35Z")

</div>

According to the [documentation](https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html#plugins-filters-grok-match) :  
Match in grok filtre is "A hash that defines the mapping of where to look, and with which patterns."

You don't have any "where to look" as Badger show you in the first response.

---

<div class="post-metadata">

**Author:** ![Dinesh\_Sharma](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dinesh_sharma/32/86511_2.png) [@Dinesh\_Sharma](https://discuss.elastic.co/u/Dinesh_Sharma)\
**Post date:** [April 6, 2021, 8:13am UTC](https://discuss.elastic.co/t/grok-filter/269194/7 "2021-04-06T08:13:06Z")

</div>

Hi,  
Split the message in three column using csv filter and even define the name of the column. Post that I used the grok filter. Can you please suggest an edit in my code shown above as I am not able to understand your answer.

---

<div class="post-metadata">

**Author:** ![Cad](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cad/32/86661_2.png) [@Cad](https://discuss.elastic.co/u/Cad)\
**Post date:** [April 6, 2021, 8:23am UTC](https://discuss.elastic.co/t/grok-filter/269194/8 "2021-04-06T08:23:27Z")

</div>

The grok filter need be like that :  
`grok { match => {"IP" => "%{IP:validIP}" } }`

Cad.

---

<div class="post-metadata">

**Author:** ![Dinesh\_Sharma](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dinesh_sharma/32/86511_2.png) [@Dinesh\_Sharma](https://discuss.elastic.co/u/Dinesh_Sharma)\
**Post date:** [April 6, 2021, 9:00am UTC](https://discuss.elastic.co/t/grok-filter/269194/9 "2021-04-06T09:00:34Z")

</div>

Hi,

After the changes you suggest the code is as follow:

````auto
input{
file
{
path=>"/home/myuser/test/new.csv"
start_position => "beginning"
}
}
filter
{
csv
{
columns => ["name","Age","IP"]
}
grok { match => {"IP" => "%{IP:validIP}" } }

if "_grokparsefailure" in [tags] {

      mutate {
        add_field => { "is_value_correct" => "false" }
      }
}
else
{
      mutate
 {
        add_field => { "is_value_correct" => "true" }
      }
}

output{
elasticsearch {
    hosts => ["http://x.x.x.x:9200"]
    index => "mytest_index"
    user => "xxxx"
    password => "xxxxxx"
}
}```

Error : 

Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[INFO] 2021-04-06 08:53:43.751 [main] runner - Starting Logstash 
[WARN] 2021-04-06 08:53:44.088 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified
[ERROR] 2021-04-06 08:53:45.347 [Converge PipelineAction::Create<main>] agent - Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [\\t\\r\\n], \"#\", \"=>\" at line 30, column 15 (byte 404) after filter\n{\ncsv\n{\ncolumns => [\"name\",\"Age\",\"IP\"]\n}\ngrok { match => {\"IP\" => \"%{IP:validIP}\" } }\nif \"_grokparsefailure\" in [tags] { \n\n mutate {\n add_field => { \"is_value_correct\" => \"false\" }\n }\n}\nelse\n{\n mutate\n {\n add_field => { \"is_value_correct\" => \"true\" }\n }\n}\n\noutput{\nelasticsearch ", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:32:in `compile_imperative'", "org/logstash/execution/AbstractPipelineExt.java:184:in `initialize'", "org/logstash/execution/JavaBasePipelineExt.java:69:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:47:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:52:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:367:in `block in converge_state'"]}
[INFO] 2021-04-06 08:53:45.630 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600}
[INFO] 2021-04-06 08:53:50.489 [LogStash::Runner] runner - Logstash shut down.
````

---

<div class="post-metadata">

**Author:** ![Cad](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cad/32/86661_2.png) [@Cad](https://discuss.elastic.co/u/Cad)\
**Post date:** [April 6, 2021, 10:02am UTC](https://discuss.elastic.co/t/grok-filter/269194/10 "2021-04-06T10:02:56Z")

</div>

Hi,

You do not close the filter.  
A `}` is missing before the output.

Cad.

---

<div class="post-metadata">

**Author:** ![Dinesh\_Sharma](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dinesh_sharma/32/86511_2.png) [@Dinesh\_Sharma](https://discuss.elastic.co/u/Dinesh_Sharma)\
**Post date:** [April 6, 2021, 10:54am UTC](https://discuss.elastic.co/t/grok-filter/269194/11 "2021-04-06T10:54:51Z")

</div>

Bro,  
You are great!

Thanks for the this help.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 4, 2021, 10:55am UTC](https://discuss.elastic.co/t/grok-filter/269194/12 "2021-05-04T10:55:22Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.