# Grok filter on logstash
**URL:** https://discuss.elastic.co/t/grok-filter-on-logstash/221410
**Category:** Logstash
**Created:** [February 28, 2020, 11:03am UTC](https://discuss.elastic.co/t/grok-filter-on-logstash/221410 "2020-02-28T11:03:12Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@carmine.fabrizio](https://discuss.elastic.co/u/carmine.fabrizio)
#### Post date: [February 28, 2020, 11:03am UTC](https://discuss.elastic.co/t/grok-filter-on-logstash/221410/1 "2020-02-28T11:03:12Z")
Hi there, I am trying to create some filter on logstash based on fields name example
> filebeat.yml
> filebeat.inputs:
>
> - type: log
> paths:
> - /var/log/httpd/\*log
> fields:
> app\_id: apache\_httpd
> fields\_under\_root: true
> enabled: true
>
> - type: log
> paths:
> - /var/log/\*log
> - /var/log/messages\*
> fields:
> app\_id: syslog
> fields\_under\_root: true
> enabled: true
so I do have to different fields here
- apache\_httpd
- syslog
and I'm trying to execute this filter
```
10-filter.conf
filter {
if [fields][app_id] == "apache_httpd" {
grok {
match => { "message" => "%{IP:client}" }
add_field => ["read_es", "%{@timestamp}"]
}
}
else if [fields][app_id] == "syslog" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} \[%{DATA}\] %{DATA} %{WORD:type} \"userText\":\"%{DATA:userText}\",\"prduction\":%{NUMBER:prduction:int},\"version\":%{NUMBER:version:int},\"bool\":%{DATA:bool},\{\"geoPoint\":\{\"location\":\"%{DATA:location}\",\"ip\":\"%{IP:ip}\",\"latitude\":%{BASE10NUM:latitude},\"longitude\":%{BASE10NUM:longitude},(\"optionalField\":%{BASE10NUM:optionalField},)?\}\}" }
add_field => ["read_es", "%{@timestamp}"]
}
date {
match => ["syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"]
}
mutate
{
remove_field => ["message"]
remove_field => ["syslog_timestamp"]
convert => ["bool","boolean"]
rename => { "ip" => "[geoPoint][ip]" }
rename => { "latitude" => "[geoPoint][latitude]" }
rename => { "longitude" => "[geoPoint][longitude]" }
rename => { "location" => "[geoPoint][location]" }
}
}
}
```
the log format of /var/log/httpd/\*.log is this
`192.168.9.1 - - [28/Feb/2020:10:37:44 +0000] "GET / HTTP/1.1" 403 4006 "-" "curl/7.64.1"`
and according to the filter that I've implemented, I should get only the client IP as the message, but for some reason, I'm still getting the full output, maybe I'm missing something or I misunderstood how a filter should be used.
Any help will be much appreciated.
Thanks!
---
### Author:  [@Fabio-sama](https://discuss.elastic.co/u/Fabio-sama)
#### Post date: [February 28, 2020, 12:39pm UTC](https://discuss.elastic.co/t/grok-filter-on-logstash/221410/2 "2020-02-28T12:39:16Z")
Hi Carmine,
can you post here couple of outputs (`apache_http` and `syslog`) of a simple pipeline without any filter applied to the events? Just take as input what you're already taking, and spit it in stdout.
Thanks
---
### Author:  [@carmine.fabrizio](https://discuss.elastic.co/u/carmine.fabrizio)
#### Post date: [February 28, 2020, 1:24pm UTC](https://discuss.elastic.co/t/grok-filter-on-logstash/221410/3 "2020-02-28T13:24:24Z")
Hi @Fabio-sama, thanks for the reply, do you mean something like this?
> output {
> stdout { codec =\> rubydebug }
> }
and this is the output that I get with `journalctl -u logstash`
apache\_httpd
> Feb 28 12:02:01 logstash01 logstash[6150]: "@timestamp" =\> 2020-02-28T12:02:27.812Z,
> Feb 28 12:02:01 logstash01 logstash[6150]: "input" =\> {
> Feb 28 12:02:01 logstash01 logstash[6150]: "type" =\> "log"
> Feb 28 12:02:01 logstash01 logstash[6150]: }
> Feb 28 12:02:01 logstash01 logstash[6150]: }
> Feb 28 12:02:01 logstash01 logstash[6150]: {
> Feb 28 12:02:01 logstash01 logstash[6150]: "host" =\> {
> Feb 28 12:02:01 logstash01 logstash[6150]: "hostname" =\> "server01",
> Feb 28 12:02:01 logstash01 logstash[6150]: "architecture" =\> "x86\_64",
> Feb 28 12:02:01 logstash01 logstash[6150]: "name" =\> "server01",
> Feb 28 12:02:01 logstash01 logstash[6150]: "containerized" =\> false,
> Feb 28 12:02:01 logstash01 logstash[6150]: "os" =\> {
> Feb 28 12:02:01 logstash01 logstash[6150]: "version" =\> "8 (Core)",
> Feb 28 12:02:01 logstash01 logstash[6150]: "codename" =\> "Core",
> Feb 28 12:02:01 logstash01 logstash[6150]: "name" =\> "CentOS Linux",
> Feb 28 12:02:01 logstash01 logstash[6150]: "kernel" =\> "4.18.0-80.el8.x86\_64",
> Feb 28 12:02:01 logstash01 logstash[6150]: "platform" =\> "centos",
> Feb 28 12:02:01 logstash01 logstash[6150]: "family" =\> "redhat"
> Feb 28 12:02:01 logstash01 logstash[6150]: },
> Feb 28 12:02:01 logstash01 logstash[6150]: "id" =\> "ee624addf73c4f94b3a5ed2d3b67c6bb"
> Feb 28 12:02:01 logstash01 logstash[6150]: },
> Feb 28 12:02:01 logstash01 logstash[6150]: "tags" =\> [
> Feb 28 12:02:01 logstash01 logstash[6150]: [0] "beats\_input\_codec\_plain\_applied"
> Feb 28 12:02:01 logstash01 logstash[6150]: ],
> Feb 28 12:02:01 logstash01 logstash[6150]: "app\_id" =\> "apache\_httpd",
> Feb 28 12:02:01 logstash01 logstash[6150]: "@version" =\> "1",
> Feb 28 12:02:01 logstash01 logstash[6150]: "agent" =\> {
> Feb 28 12:02:01 logstash01 logstash[6150]: "ephemeral\_id" =\> "25fbc19f-62b2-4d88-9ba0-640887f55d7e",
> Feb 28 12:02:01 logstash01 logstash[6150]: "version" =\> "7.6.0",
> Feb 28 12:02:01 logstash01 logstash[6150]: "hostname" =\> "server01",
> Feb 28 12:02:01 logstash01 logstash[6150]: "id" =\> "db0a0fb8-ab49-4030-bd57-22646c5d123c",
> Feb 28 12:02:01 logstash01 logstash[6150]: "type" =\> "filebeat"
> Feb 28 12:02:01 logstash01 logstash[6150]: },
> Feb 28 12:02:01 logstash01 logstash[6150]: "log" =\> {
> Feb 28 12:02:01 logstash01 logstash[6150]: "file" =\> {
> Feb 28 12:02:01 logstash01 logstash[6150]: "path" =\> "/var/log/httpd/access\_log"
> Feb 28 12:02:01 logstash01 logstash[6150]: },
> Feb 28 12:02:01 logstash01 logstash[6150]: "offset" =\> 15498
> Feb 28 12:02:01 logstash01 logstash[6150]: },
> Feb 28 12:02:01 logstash01 logstash[6150]: "message" =\> "192.168.9.1 - - [28/Feb/2020:12:02:21 +0000] "GET / HTTP/1.1" 403 4006 "-" "curl/7.64.1"",
> Feb 28 12:02:01 logstash01 logstash[6150]: "ecs" =\> {
> Feb 28 12:02:01 logstash01 logstash[6150]: "version" =\> "1.4.0"
> Feb 28 12:02:01 logstash01 logstash[6150]: },
> Feb 28 12:02:01 logstash01 logstash[6150]: "@timestamp" =\> 2020-02-28T12:02:27.812Z,
> Feb 28 12:02:01 logstash01 logstash[6150]: "input" =\> {
> Feb 28 12:02:01 logstash01 logstash[6150]: "type" =\> "log"
> Feb 28 12:02:01 logstash01 logstash[6150]: }
> Feb 28 12:02:01 logstash01 logstash[6150]: }
Syslog
> Feb 28 12:01:01 logstash01 logstash[6150]: "@timestamp" =\> 2020-02-28T12:01:27.814Z,
> Feb 28 12:01:01 logstash01 logstash[6150]: "input" =\> {
> Feb 28 12:01:01 logstash01 logstash[6150]: "type" =\> "log"
> Feb 28 12:01:01 logstash01 logstash[6150]: }
> Feb 28 12:01:01 logstash01 logstash[6150]: }
> Feb 28 12:01:01 logstash01 logstash[6150]: {
> Feb 28 12:01:01 logstash01 logstash[6150]: "host" =\> {
> Feb 28 12:01:01 logstash01 logstash[6150]: "hostname" =\> "server01",
> Feb 28 12:01:01 logstash01 logstash[6150]: "architecture" =\> "x86\_64",
> Feb 28 12:01:01 logstash01 logstash[6150]: "name" =\> "server01",
> Feb 28 12:01:01 logstash01 logstash[6150]: "containerized" =\> false,
> Feb 28 12:01:01 logstash01 logstash[6150]: "os" =\> {
> Feb 28 12:01:01 logstash01 logstash[6150]: "name" =\> "CentOS Linux",
> Feb 28 12:01:01 logstash01 logstash[6150]: "codename" =\> "Core",
> Feb 28 12:01:01 logstash01 logstash[6150]: "version" =\> "8 (Core)",
> Feb 28 12:01:01 logstash01 logstash[6150]: "kernel" =\> "4.18.0-80.el8.x86\_64",
> Feb 28 12:01:01 logstash01 logstash[6150]: "family" =\> "redhat",
> Feb 28 12:01:01 logstash01 logstash[6150]: "platform" =\> "centos"
> Feb 28 12:01:01 logstash01 logstash[6150]: },
> Feb 28 12:01:01 logstash01 logstash[6150]: "id" =\> "ee624addf73c4f94b3a5ed2d3b67c6bb"
> Feb 28 12:01:01 logstash01 logstash[6150]: },
> Feb 28 12:01:01 logstash01 logstash[6150]: "tags" =\> [
> Feb 28 12:01:01 logstash01 logstash[6150]: [0] "beats\_input\_codec\_plain\_applied"
> Feb 28 12:01:01 logstash01 logstash[6150]: ],
> Feb 28 12:01:01 logstash01 logstash[6150]: "app\_id" =\> "syslog",
> Feb 28 12:01:01 logstash01 logstash[6150]: "@version" =\> "1",
> Feb 28 12:01:01 logstash01 logstash[6150]: "agent" =\> {
> Feb 28 12:01:01 logstash01 logstash[6150]: "version" =\> "7.6.0",
> Feb 28 12:01:01 logstash01 logstash[6150]: "ephemeral\_id" =\> "25fbc19f-62b2-4d88-9ba0-640887f55d7e",
> Feb 28 12:01:01 logstash01 logstash[6150]: "hostname" =\> "server01",
> Feb 28 12:01:01 logstash01 logstash[6150]: "id" =\> "db0a0fb8-ab49-4030-bd57-22646c5d123c",
> Feb 28 12:01:01 logstash01 logstash[6150]: "type" =\> "filebeat"
> Feb 28 12:01:01 logstash01 logstash[6150]: },
> Feb 28 12:01:01 logstash01 logstash[6150]: "log" =\> {
> Feb 28 12:01:01 logstash01 logstash[6150]: "file" =\> {
> Feb 28 12:01:01 logstash01 logstash[6150]: "path" =\> "/var/log/messages"
> Feb 28 12:01:01 logstash01 logstash[6150]: },
> Feb 28 12:01:01 logstash01 logstash[6150]: "offset" =\> 170810
> Feb 28 12:01:01 logstash01 logstash[6150]: },
> Feb 28 12:01:01 logstash01 logstash[6150]: "message" =\> "Feb 28 12:01:17 server01 systemd[1]: Started Filebeat sends log files to Logstash or directly to Elasticsearch..",
> Feb 28 12:01:01 logstash01 logstash[6150]: "ecs" =\> {
> Feb 28 12:01:01 logstash01 logstash[6150]: "version" =\> "1.4.0"
> Feb 28 12:01:01 logstash01 logstash[6150]: },
Let me know if its enough, thanks!
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [February 28, 2020, 5:25pm UTC](https://discuss.elastic.co/t/grok-filter-on-logstash/221410/4 "2020-02-28T17:25:13Z")
> [@carmine.fabrizio](#):
>
> [fields][app\_id]
You set fields\_under\_root true, so your events have [app\_id], and not [fields][app\_id].
---
### Author:  [@carmine.fabrizio](https://discuss.elastic.co/u/carmine.fabrizio)
#### Post date: [March 2, 2020, 1:05pm UTC](https://discuss.elastic.co/t/grok-filter-on-logstash/221410/5 "2020-03-02T13:05:39Z")
Thanks @Badger!
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 30, 2020, 1:05pm UTC](https://discuss.elastic.co/t/grok-filter-on-logstash/221410/6 "2020-03-30T13:05:44Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.