# 2 grok filter NOT WOERKED ! :|
**URL:** https://discuss.elastic.co/t/2-grok-filter-not-woerked/109125
**Category:** Logstash
**Created:** [November 26, 2017, 7:46am UTC](https://discuss.elastic.co/t/2-grok-filter-not-woerked/109125 "2017-11-26T07:46:46Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@omid](https://discuss.elastic.co/u/omid)
#### Post date: [November 26, 2017, 7:46am UTC](https://discuss.elastic.co/t/2-grok-filter-not-woerked/109125/1 "2017-11-26T07:46:46Z")
**im use 2 grok filter to one config file but not worked !!!**
```
input {
beats {
port => 5443
type => syslog
ssl => true
ssl_certificate => "/etc/logstash/logstash.crt"
ssl_key => "/etc/logstash/logstash.key"
}
}
filter {
if [type] == "log" {
grok {
patterns_dir => "/etc/logstash/patterns"
match => { "%{IP:client} %{HTTPDUSER:ident} %{USER:auth} \[%{HTTPDATE:timestamp_server_genaration}\] \"%{WORD:method} %{URIPATHPARAM:request} HTTP/%{NUMBER:httpversion}\" %{NUMBER:status_code} %{NUMBER:bytes} %{QS:refferer} %{QS:user_agent} length %{NUMBER:length} rtime %{NUMBER:request_time} uri %{URIPATHPARAM:uri} realip %{IP:realip}" }
remove_tag => ["nginx_access", "_grokparsefailure"] }
add_field => {
"type" => "nginx_access"
}
remove_field => ["log"]
}
}
date {
match => ["time_local", "dd/MMM/YYYY:HH:mm:ss Z"]
target => "@timestamp"
remove_field => "time_local"
}
if [type] == "log" {
grok {
patterns_dir => "/etc/logstash/patterns"
match => { (?%{YEAR}[./-]%{MONTHNUM}[./-]%{MONTHDAY}[-]%{TIME}) \[%{LOGLEVEL:severity}\] %{POSINT:pid}#%{NUMBER}:%{GREEDYDATA:errormessage} }
remove_tag => ["nginx_error", "_grokparsefailure"] }
add_field => {
"type" => "nginx_error"
}
remove_field => ["log"]
}
date {
match => ["time_local", "YYYY/MM/dd HH:mm:ss"]
target => "@timestamp"
remove_field => "time_local"
}
output {
elasticsearch { hosts => ["localhost:9200"]
hosts => "localhost:9200"
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
```
}
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [November 26, 2017, 7:48am UTC](https://discuss.elastic.co/t/2-grok-filter-not-woerked/109125/2 "2017-11-26T07:48:14Z")
Which one? What does you log file look like? What do you mean when you say it doesn't work?
---
### Author:  [@omid](https://discuss.elastic.co/u/omid)
#### Post date: [November 26, 2017, 8:05am UTC](https://discuss.elastic.co/t/2-grok-filter-not-woerked/109125/3 "2017-11-26T08:05:24Z")
When this alone runs inside the file, it works without problems ↓↓
```
filter {
if [type] == "log" {
grok {
match => [
"message" => "%{IP:client} %{HTTPDUSER:ident} %{USER:auth} \[%{HTTPDATE:timestamp_server_genaration}\] \"%{WORD:method} %{URIPATHPARAM:request} HTTP/%{NUMBER:httpversion}\" %{NUMBER:status_code} %{NUMBER:bytes} %{QS:refferer} %{QS:user_agent} length %{NUMBER:length} rtime %{NUMBER:request_time} uri %{URIPATHPARAM:uri} realip %{IP:realip}"
]
remove_tag => ["_grokparsefailure"]
add_tag => ["nginx_access"]
}
geoip {
source => "client"
}
mutate {
convert => ["[geoip][location]", "float" ]
}
}
}
```
But when I put it all that wayو kibana this message **_No results found_** ↓↓
```
filter {
if [type] == "log" {
grok {
match => [
"message" => "%{IP:client} %{HTTPDUSER:ident} %{USER:auth} \[%{HTTPDATE:timestamp_server_genaration}\] \"%{WORD:method} %{URIPATHPARAM:request} HTTP/%{NUMBER:httpversion}\" %{NUMBER:status_code} %{NUMBER:bytes} %{QS:refferer} %{QS:user_agent} length %{NUMBER:length} rtime %{NUMBER:request_time} uri %{URIPATHPARAM:uri} realip %{IP:realip}"
]
remove_tag => ["_grokparsefailure"]
add_tag => ["nginx_access"]
}
geoip {
source => "client"
}
mutate {
convert => ["[geoip][location]", "float" ]
}
}
}
filter {
if [type] == "log" {
grok {
match => [
(?%{YEAR}[./-]%{MONTHNUM}[./-]%{MONTHDAY}[-]%{TIME}) \[%{LOGLEVEL:severity}\] %{POSINT:pid}#%{NUMBER}:%{GREEDYDATA:errormessage}
]
remove_tag => ["_grokparsefailure"]
add_tag => ["nginx_error"]
}
geoip {
source => "client"
}
mutate {
convert => ["[geoip][location]", "float" ]
}
}
}
```
And even when I put it like the first one ...
this file name == syslog-filter.conf
and path = /etc/logstash/conf.d
Kibana result after this config and grok = NO RESULT !
---
### Author:  [@guyboertje](https://discuss.elastic.co/u/guyboertje)
#### Post date: [November 26, 2017, 9:49am UTC](https://discuss.elastic.co/t/2-grok-filter-not-woerked/109125/4 "2017-11-26T09:49:07Z")
As you want to add the `nginx_access` or `nginx_error` tag to the event where the pattern matched you should use the `_grokparsefailure` of the first grok to drive the second grok.
```auto
filter {
if [type] == "log" {
grok {
match => [
"message" => "%{IP:client} %{HTTPDUSER:ident} %{USER:auth} \[%{HTTPDATE:timestamp_server_genaration}\] \"%{WORD:method} %{URIPATHPARAM:request} HTTP/%{NUMBER:httpversion}\" %{NUMBER:status_code} %{NUMBER:bytes} %{QS:refferer} %{QS:user_agent} length %{NUMBER:length} rtime %{NUMBER:request_time} uri %{URIPATHPARAM:uri} realip %{IP:realip}"
]
add_tag => ["nginx_access"]
}
if "_grokparsefailure" in [tags] {
grok {
match => [
(?%{YEAR}[./-]%{MONTHNUM}[./-]%{MONTHDAY}[-]%{TIME}) \[%{LOGLEVEL:severity}\] %{POSINT:pid}#%{NUMBER}:%{GREEDYDATA:errormessage}
]
add_tag => ["nginx_error"]
}
}
if "_grokparsefailure" not in [tags] {
geoip {
source => "client"
}
mutate {
convert => ["[geoip][location]", "float" ]
}
}
}
}
```
Also; see [https://www.elastic.co/guide/en/logstash/6.0/lookup-enrichment.html](https://www.elastic.co/guide/en/logstash/6.0/lookup-enrichment.html)
`[geoip][location]` is not what you think it is - it is an array of longitude and latitude and the values are floats already, so you don't need the mutate convert function.
I know you were following the tutorial I linked to in another post but you should use the same field name for the `timestamp` field in both groks so you can use one `date` filter with two patterns to set the event `@timestamp` field to be the parsed value from both log formats.
---
### Author:  [@omid](https://discuss.elastic.co/u/omid)
#### Post date: [November 28, 2017, 9:51am UTC](https://discuss.elastic.co/t/2-grok-filter-not-woerked/109125/5 "2017-11-28T09:51:21Z")
Hi
Im copied and pasted your filter
to syslog-filter.conf file
and Errore :
> [2017-11-26T00:00:00,519][ERROR][logstash.agent] Cannot create pipeline {:reason=\>"Expected one of #, {, ,, ] at line 15, column 13 (byte 240) after filter {\n if [type] == "log" {\n\tgrok {\n\t\tmatch =\> [ \n\t\t"message" "}
>
> [2017-11-26T00:00:10,785][ERROR][logstash.agent] Cannot create pipeline {:reason=\>"Expected one of #, {, ,, ] at line 15, column 13 (byte 240) after filter {\n if [type] == "log" {\n\tgrok {\n\t\tmatch =\> [ \n\t\t"message" "}
and after that ...
changed grok simple ... for test =
```
filter {
if [type] == "log" {
grok {
match => ["message" => "%{IPORHOST:clientip} - - \[%{HTTPDATE:timestamp}\]"]
add_tag => ["nginx_access"]
}
if "_grokparsefailure" in [tags] {
grok {
match => [
(?%{YEAR}[./-]%{MONTHNUM}[./-]%{MONTHDAY}[-]%{TIME}) \[%{LOGLEVEL:severity}\] %{POSINT:pid}#%{NUMBER}:%{GREEDYDATA:errormessage}
]
add_tag => ["nginx_error"]
}
}
if "_grokparsefailure" not in [tags] {
geoip {
source => "client"
}
mutate {
convert => ["[geoip][location]", "float" ]
}
}
}
}
```
And again again the same error
```
[2017-11-27T00:00:07,395][ERROR][logstash.agent] Cannot create pipeline {:reason=>"Expected one of #, {, ,, ] at line 26, column 19 (byte 473) after filter {\n if [type] == \"log\" {\n grok {\n match => [ \n \"message\" "}
[2017-11-27T00:00:17,591][ERROR][logstash.agent] Cannot create pipeline {:reason=>"Expected one of #, {, ,, ] at line 26, column 19 (byte 473) after filter {\n if [type] == \"log\" {\n grok {\n match => [ \n \"message\" "}
```
And once with this command :
`./logstash -t -f /etc/logstash/conf.d/syslog-filter.conf`
resulted ::
> ```
> ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console.
> WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
> Could not find log4j2 configuration at path //usr/share/logstash/config/log4j2.properties. Using default config which logs to console
> 13:20:07.335 [LogStash::Runner] FATAL logstash.runner - The given configuration is invalid. Reason: Expected one of #, {, ,, ] at line 15, column 33 (byte 256) after filter {
> if [type] == "log" {
> grok {
> match => ["message"
>
> ```
**What should I do ??**
---
### Author:  [@guyboertje](https://discuss.elastic.co/u/guyboertje)
#### Post date: [November 28, 2017, 10:26am UTC](https://discuss.elastic.co/t/2-grok-filter-not-woerked/109125/6 "2017-11-28T10:26:54Z")
Your original config was (I just copy pasted):
```auto
grok {
match => [
(?%{YEAR}[./-]%{MONTHNUM}[./-]%{MONTHDAY}[-]%{TIME}) \[%{LOGLEVEL:severity}\] %{POSINT:pid}#%{NUMBER}:%{GREEDYDATA:errormessage}
]
add_tag => ["nginx_error"]
}
```
It should be:
```auto
grok {
match => ["message" => "(?%{YEAR}[./-]%{MONTHNUM}[./-]%{MONTHDAY}[-]%{TIME}) \[%{LOGLEVEL:severity}\] %{POSINT:pid}#%{NUMBER}:%{GREEDYDATA:errormessage}"]
add_tag => ["nginx_error"]
}
```
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 26, 2017, 10:27am UTC](https://discuss.elastic.co/t/2-grok-filter-not-woerked/109125/7 "2017-12-26T10:27:03Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.