# How to Filter log base on log line
**URL:** https://discuss.elastic.co/t/how-to-filter-log-base-on-log-line/126918
**Category:** Logstash
**Created:** [April 5, 2018, 12:05pm UTC](https://discuss.elastic.co/t/how-to-filter-log-base-on-log-line/126918 "2018-04-05T12:05:06Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@Bhargav\_Patel](https://discuss.elastic.co/u/Bhargav_Patel)
#### Post date: [April 5, 2018, 12:05pm UTC](https://discuss.elastic.co/t/how-to-filter-log-base-on-log-line/126918/1 "2018-04-05T12:05:06Z")
Hi I am newbee in logstash.
can someone let me know how to filter below log. I want ot break this line base on timestamp java class and my custom text like "Updated By" and "Audit Log" and log level etc.
**_2018-04-05 17:22:02,047 DEBUG [http-nio-8082-exec-1] DomainResource: Audit Log : Updating user information. User Id is AAA11BE | Updated By : bhargav_**
I have tried below filter but not working.
```
input {
file {
path => ["/home/truecom/PE/portaleditor.log"]
type => "pelog"
start_position => end
ignore_older => 0
sincedb_path => "null"
}
```
}filter {
mutate {
gsub =\> ['message', "\n", " "]
}
grok {
match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:time} %{LOGLEVEL:loglevel} %{JAVACLASS:class} - %{GREEDYDATA:msg}" }
}
}output {
stdout {
codec =\> rubydebug
}
elasticsearch {
hosts =\> ["localhost:9200"]
action =\> "index"
index =\> "portalditor-%{+YYYY.MM.dd}"
}
}
Kibana output is
```
path:/home/truecom/PE/portaleditor.log @timestamp:April 5th 2018, 18:54:37.678 @version:1 host:truecom message:2018-04-05 18:54:36,690 DEBUG [http-nio-8082-exec-9] ClientResource: Audit Log : Getting allClients type:pelog tags:_grokparsefailure _id:AWKV-wDXbTkgN2yFEvrM _type:pelog _index:portalditor-2018.04.05 _score:
```
Logstash Output
```
"path" => "/home/truecom/PE/portaleditor.log",
"@timestamp" => 2018-04-05T13:32:21.854Z,
"@version" => "1",
"host" => "truecom",
"message" => "2018-04-05 19:02:21,499 DEBUG [http-nio-8082-exec-5] ClientResource: Audit Log : Getting allClients",
"type" => "pelog",
"tags" => [
[0] "_grokparsefailure"
]
```
I want to extract message field. Like log level and timestamp and msg.
---
### Author:  [@Bhargav\_Patel](https://discuss.elastic.co/u/Bhargav_Patel)
#### Post date: [April 5, 2018, 2:04pm UTC](https://discuss.elastic.co/t/how-to-filter-log-base-on-log-line/126918/2 "2018-04-05T14:04:28Z")
Some how i am able to get the data like log level but i am getting blank for GREEDYDATA. what is wrong in my config file? i tried to put GREEDYDATA at the end as well. belows my update .conf file .
```
input {
file {
path => ["/home/truecom/PE/portaleditor.log"]
type => "pelog"
start_position => end
ignore_older => 0
sincedb_path => "null"
}
```
}
filter {
grok {
match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:timestamp}%{GREEDYDATA:input}%{LOGLEVEL:loglevel}" }
}
}
output {
stdout {
codec =\> rubydebug
}
elasticsearch {
hosts =\> ["localhost:9200"]
action =\> "index"
index =\> "portalditor-%{+YYYY.MM.dd}"
}
}
Log stash output is
```
{
"path" => "/home/truecom/PE/portaleditor.log",
"input" => " ",
"@timestamp" => 2018-04-05T14:01:24.784Z,
"loglevel" => "DEBUG",
"@version" => "1",
"host" => "truecom",
"message" => "2018-04-05 19:31:24,293 DEBUG [http-nio-8082-exec-2] ClientResource: Audit Log : Getting allClients",
"type" => "pelog",
"timestamp" => "2018-04-05 19:31:24,293"
```
}
---
### Author:  [@Bhargav\_Patel](https://discuss.elastic.co/u/Bhargav_Patel)
#### Post date: [April 5, 2018, 2:25pm UTC](https://discuss.elastic.co/t/how-to-filter-log-base-on-log-line/126918/3 "2018-04-05T14:25:39Z")
Getting more closer now. able to extract timestep and log level. but facing issue while getting thead and java class
```
grok {
match => { "message" => "%{TIMESTAMP_ISO8601:timestamp}%{SPACE}%{LOGLEVEL:loglevel}%{SPACE}%{THREAD:thread}%{SPACE}%{GREEDYDATA:msg}" }
```
}
Getting \_grokparsefailure while adding THREAD and JAVACLASS
Logstash error is
```
error=>"pattern %{THREAD:thread} not defined"}
```
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [April 6, 2018, 7:38pm UTC](https://discuss.elastic.co/t/how-to-filter-log-base-on-log-line/126918/4 "2018-04-06T19:38:20Z")
I don't believe THREAD is a standard grok pattern. What gave you that idea that it was? You can e.g. use
```
\[(?[^\]]*)\]
```
to match the thread name (capture everything between the opening square bracket and the first closing one).
---
### Author:  [@Bhargav\_Patel](https://discuss.elastic.co/u/Bhargav_Patel)
#### Post date: [April 9, 2018, 10:47am UTC](https://discuss.elastic.co/t/how-to-filter-log-base-on-log-line/126918/5 "2018-04-09T10:47:30Z")
Thanks for quick reply.
Not sure about the solution you have mention but this %{NOTSPACE:javathread} work for me. will also try the same which you have mention.
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [April 9, 2018, 11:13am UTC](https://discuss.elastic.co/t/how-to-filter-log-base-on-log-line/126918/6 "2018-04-09T11:13:40Z")
NOTSPACE works too but will capture the square brackets that aren't really part of the thread name. You could of course remove the square brackets with a mutate filter.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 7, 2018, 11:13am UTC](https://discuss.elastic.co/t/how-to-filter-log-base-on-log-line/126918/7 "2018-05-07T11:13:54Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.