Grokparse error

I am using logstash configuration file to read log file and
then extract pattern: [ERROR] 2000023:Making status off as an error

***Objective: ***:
to extract pattern whether pattern is there in file or not and
if pattern exist take value in boolean yes
otherwise no

I tried and test these 2 grok pattern giving same results:

1--
%{DATESTAMP:mytimestamp}%{SPACE}%{WORD:type}%{SPACE}[%{DATA:ERROR}]%{SPACE}%{INT:INT}:%{GREEDYDATA:syslog_message}

2--
message" =>"%{DATESTAMP:mytimestamp}%{SPACE}%{WORD:type}%{SPACE}[%{DATA:ERROR}]%{SPACE}%{INT:INT}:%{GREEDYDATA:message}::

I am getting grok parsing error

{"path":"C:/ELK-Stack/log files/SAPInterface.log","tags":["_grokparsefailure"],"@version":"1","@timestamp":"2019-10-17T11:55:24.408Z","message":"2019-08-18 14:02:00 DataXferImplementor [ERROR] [Ljava.lang.StackTraceElement;@27982\r","host":"6JDSTT2"}
{"path":"C:/ELK-Stack/log files/SAPInterface.log","tags":["_grokparsefailure"],"@version":"1","@timestamp":"2019-10-17T11:55:24.408Z","message":"2019-08-18 14:02:00 FileUtility [DEBUG] Change the File Processing Status to : OFF\r","host":"6JDSTT2"}
{"path":"C:/ELK-Stack/log files/SAPInterface.log","tags":["_grokparsefailure"],"@version":"1","@timestamp":"2019-10-17T11:55:24.408Z","message":"2019-08-18 14:02:00 FileUtility [DEBUG] File Processing Status updated : OFF\r","host":"6JDSTT2"}
{"path":"C:/ELK-Stack/log files/SAPInterface.log","tags":["_grokparsefailure"],"@version":"1","@timestamp":"2019-10-17T11:55:24.408Z","message":"2019-08-18 14:02:00 DataXferImplementor [ERROR] \r","host":"6JDSTT2"}
{"path":"C:/ELK-Stack/log files/SAPInterface.log","tags":["_grokparsefailure"],"@version":"1","@timestamp":"2019-10-17T11:55:24.408Z","message":"2019-08-18 14:02:00 DataXferImplementor [ERROR] [Ljava.lang.StackTraceElement;@778255\r","host":"6JDSTT2"}
{"path":"C:/ELK-Stack/log files/SAPInterface.log","tags":["_grokparsefailure"],"@version":"1","@timestamp":"2019-10-17T11:55:24.408Z","message":"2019-08-18 14:02:00 DataXferImplementor [INFO] sDestinationFolder : D:/ETPSuite/ETP Integrator V5.5/SAP INBOUND_B","host":"6JDSTT2"}
{"path":"C:/ELK-Stack/log files/SAPInterface.log","tags":["_grokparsefailure"],"@version":"1","@timestamp":"2019-10-17T11:55:24.408Z","message":" sDestinationFileName : SALESM_NS_V018_20190818_20190818_140038.xml\r","host":"6JDSTT2"}
{"path":"C:/ELK-Stack/log files/SAPInterface.log","tags":["_grokparsefailure"],"@version":"1","@timestamp":"2019-10-17T11:55:24.408Z","message":"2019-08-18 14:02:00 DataXferImplementor [INFO] status for FIle Transfer>>true\r","host":"6JDSTT2"}
{"path":"C:/ELK-Stack/log files/SAPInterface.log","tags":["_grokparsefailure"],"@version":"1","@timestamp":"2019-10-17T11:55:24.408Z","message":"2019-08-18 14:02:00 DataXferImplementor [INFO] Path of SFTP server to upload:/lbrandsprod/POS/Inbound/\r","host":"6JDSTT2"}
{"path":"C:/ELK-Stack/log files/SAPInterface.log","tags":["_grokparsefailure"],"@version":"1","@timestamp":"2019-10-17T11:55:24.408Z","message":"2019-08-18 14:02:00 DataXferImplementor [INFO] PUT METHOD OF UPLOAD\r","host":"6JDSTT2"}
{"logsimestamp":"2019-08-18 14:02:00","INT":"2000023","@timestamp":"2019-10-17T11:55:24.408Z","type3":"status","type2":"Making","message":"2019-08-18 14:02:00 DataXferImplementor [ERROR] 2000023:Making status off as error occured due to::invalid type=104\r","host":"6JDSTT2","path":"C:/ELK-Stack/log files/SAPInterface.log","type4":"off","@version":"1","ERROR":"ERROR","type":"DataXferImplementor"}
{"path":"C:/ELK-Stack/log files/SAPInterface.log","tags":["_grokparsefailure"],"@version":"1","@timestamp":"2019-10-17T11:55:24.408Z","message":"2019-08-18 14:02:00 DataXferImplementor [ERROR] [Ljava.lang.StackTraceElement;@da90c\r","host":"6JDSTT2"}
{"path":"C:/ELK-Stack/log files/SAPInterface.log","tags":["_grokparsefailure"],"@version":"1","@timestamp":"2019-10-17T11:55:24.408Z","message":"2019-08-18 14:02:00 FileUtility [DEBUG] Change the File Processing Status to : OFF\r","host":"6JDSTT2"}
{"path":"C:/ELK-Stack/log files/SAPInterface.log","tags":["_grokparsefailure"],"@version":"1","@timestamp":"2019-10-17T11:55:24.408Z","message":"2019-08-18 14:02:00 FileUtility [DEBUG] File Processing Status updated : OFF\r","host":"6JDSTT2"}
{"path":"C:/ELK-Stack/log files/SAPInterface.log","tags":["_grokparsefailure"],"@version":"1","@timestamp":"2019-10-17T11:55:24.408Z","message":"2019-08-18 14:02:00 DataXferImplementor [ERROR] invalid type=104\r","host":"6JDSTT2"}

here is the data in log file for reference:

2019-08-18 14:02:00 DataXferImplementor [ERROR] [Ljava.lang.StackTraceElement;@1684706
2019-08-18 14:02:00 FileUtility [DEBUG] Change the File Processing Status to : OFF
2019-08-18 14:02:00 DataXferImplementor [ERROR] 2000023:Making status off as error occured due to::Cannot write file:SALESM_NS_1766_20190818_20190818_140038.xml

Here is my configuration file:

  input
 {

      file 
	{
        path => "C:/ELK-Stack/log files/SAPInterface.log"
start_position => "beginning"
sincedb_path => "NUL"
      }
 }


    
filter 
{
    grok
{
  match => [
      "message",
      "%{TIMESTAMP_ISO8601:logsimestamp}%{SPACE}%{WORD:type}%{SPACE}\[%{DATA:ERROR}\]%{SPACE}%{INT:INT}:%{WORD:type2}%{SPACE}%{WORD:type3}%{SPACE}%{WORD:type4}"
    ]

    }
 }





output
 {

	 stdout
 {
codec => json_lines
}
  
elasticsearch 
{
    hosts => ["localhost:9200"]
    index => "cs_00000000000000"

   
  }
}

so what I am getting (i believe):

the line in which pattern is matched is not having an error.
but I only want boolean value whether it is there or not and it should update it every minute so don't want it to read the file, again and again, it should reading from where it finds last pattern.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.