i want to get started with ELK but i'm stuck every-time, after 2 weeks of complication i can finally ingest my sample log into elastic search, but fell into another problem, i can't see all my records in kibana and those who are sent, they are tagged as _grokparsefailure, it mean something wrong with my grok filter, in the following i will give details of my Lab
my sample log file is:
user1 email1 pass1
user2 email2 pass2
user3 email3 pass3
The third line is probably missing due to you not having a newline after the last line. Also note that the field containing the data is named message and not @message, which is probably why the grok filter fails.
you are right concerning the third line, after i hit enter after the last line it was sent, now the remaining problem is the tag _grokparsefailure, the @ after the message is not the problem i just add it
are you referring to the @ after message, if so, i can assure you that is not the cause cuz i just add @ after message and yes i reprocessed my file to index the data with the new config and still have the _grokparsefailure issue
I do not know why the grok filter is having issues not what could be causing it. The only thing I can think of is the charset. I would recommend using the dissect filter instead. Change the field names and remove the grok filter.
after removing the grok filter it work just fine, but i wonder why i can't get grok working ?? because i read that
Also dissect is preferably used in situations where number of fields are always the same otherwise grok is a better option.
unfortunately it is my case, because i have different lines in the same log
like this :
2018-10-23 12:27:47.93 spid54 Using 'xpstar.dll' version '2014.120.2000' to execute extended stored procedure 'xp_instance_regread'. This is an informational message only; no user action is required.
2018-10-23 12:29:32.49 spid54 Attempting to load library 'xplog70.dll' into memory. This is an informational message only. No user action is required.
2018-10-23 12:29:32.52 spid54 Using 'xplog70.dll' version '2014.120.2000' to execute extended stored procedure 'xp_msver'. This is an informational message only; no user action is required.
2018-10-23 13:45:21.71 Logon Error: 18456, Severity: 14, State: 7.
2018-10-23 13:45:21.71 Logon Login failed for user 'sa'. Reason: An error occurred while evaluating the password. [CLIENT: ]
2018-10-23 13:46:54.70 Logon Error: 18470, Severity: 14, State: 1.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.