How to filter only error from log file

Elastic Stack Logstash
1

Hi community,
i have a log file contains INFO,WARN and ERROR like :

17:37:17,103 ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].....rest of text.

what is the convenient grok to filter only All errors.
Thanks for any help.

2

you would use whatever grok statement that would match your entire log, I tested with a basic one just for simplicity

grok{
	  match => { "message" => "%{TIME:timestamp} %{LOGLEVEL:LEVEL} %{GREEDYDATA:whateverElseYoureFiltering}" }
	}

so then if you only want errors you would then add an if statement that checks if "ERROR" is in LEVEL

if	"ERROR" in [LEVEL]
{
}

Then do whatever else you needed with it

So I tested it with only printing if the word ERROR was in the message like so.

using htis code to test:

input {
	stdin{}
}

filter {

	grok{
	  match => { "message" => "%{TIME:timestamp} %{LOGLEVEL:LEVEL} %{GREEDYDATA:whateverElseYoureFiltering}" }
	}
	
}

output {
if	"ERROR" in [LEVEL]
{
	 stdout { codec => rubydebug }
}
}
3

Thank you so much @Jaxon_Kochel,i tested this code but the filter doesn't work.take a look around it.

input {
    beats {
       port => "5043"
    }
}
filter{
grok{
	  match => { "message" => "%{TIME:timestamp} %{LOGLEVEL:LEVEL} %{GREEDYDATA:errormsg}" }
	}
}
output {
if	"ERROR" in [LEVEL]
{
	 stdout { codec => rubydebug }
}
}
4

@echo off
sed -n "/ERROR/p" "cds.log" > "cds1.txt"
sed -e "/HRWPC_RFC_EP_READ_PHOTO_URI threw an Exception of type AbapException/d" "cds1.txt" > "cds2.txt"
sed -e "/ bad SQL grammar [DROP/d" "cds2.txt" > "cds3.txt"

@echo off
set intIN=-1
set intOUT=0
set /a intIN+=1
set /a intOUT+=1
sed -n "/ERROR/p" "cds.log" > "cds%intOUT%.txt"
set /a intIN+=1
set /a intOUT+=1
sed -e "/HRWPC_RFC_EP_READ_PHOTO_URI threw an Exception of type AbapException/d" "cds%intIN%.txt" > "cds%intOUT%.txt"
set /a intIN+=1
set /a intOUT+=1
sed -e "/ bad SQL grammar [DROP/d" "cds%intIN%.txt" > "cds%intOUT%.txt"

Ethan Stark
Apps4Rent

5

Not sure, works fine for me. if oyu remove the if error part and just do the stdout part does it print anything at all? if not, you're beats is not working correctly.

6

Yeh you are right,when i tested with grokdebug it works fine.but when i navigate to http://localhost:9200/logstash-2017.06.08/_search.i don't find the result wanted by the filter.so,where i can see the result of my filter.

7

You'll need to use Kibana or another tool to see your results that are stored inside of elasticsearch. In kibana you can even specify what you want to see. so you could query your elasticsearch storage for LEVEL:ERROR in kibana and it would only show you logs with the value "ERROR" for LOGLEVEL.

8

Yeh,but i need rest Api to consume it inside java application.if you have an idea to do it.

9

Yeh,but i need rest Api to consume it inside java application.if you have an idea to do it.

Questions about how to make ES queries from Java is better asked in the Elasticsearch category. When you do that please be more specific.

10

Thank you Sir,for helping me out.

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.