Logstash timestamp issue

Elastic Stack Logstash
1

I have logs which have got time stamp which are in the format 17/01/30 07:55:03

i have got it parsed using Grok filters but when i want to assing the same value to @timestamp it does not turn up.
My filters look like
grok {
patterns_dir => ["/etc/logstash/pattern"]
match => {
"message" => "%{sparktime:logdate} %{LOGLEVEL:loglevel} %{GREEDYDATA:details}"
}
tag_on_failure => []
}
date {
match => ["logdate", "yy/MM/dd HH:mm:ss"]
}

                        grok {
                             match => {
                                    "details" => "%{WORD:class}:  %{WORD}:%{GREEDYDATA:mes} : %{NUMBER:numOfLines:int} logType:%{GREEDYDATA:stage}:%{GREEDYDATA:substage} zipFile:%{GREEDYDATA:zipfile} stgFile:%{GREEDYDATA:textfile}"
                                     }
                            tag_on_failure => []
                            }

Could anyone please advise where am i going wrong. If i use date filter other fields also get disturbed but if i remove date then all parsing work but timestamp does not get assigned to acual log timestamp

2

i have got it parsed using Grok filters but when i want to assing the same value to @timestamp it does not turn up.

What does this mean, exactly? Your configuration looks correct. Please show a message that you think is wrong. Use a stdout { codec => rubydebug } output. If the date filter fails to parse a timestamp the Logstash log will contain clues about what it didn't like.

3

Hi Magnus

This does produce any logs in my file.
What i meant is the grok filter seems to be working but when i add the date filter it creates problems.
I do not get the fields mentioned in the grok filter but if the remove the date filter part it does produce the fields present in grok filter.
Do i need to change anything to see the output in logs using rubydebug because currently i see error info and warnign messages and none of them has any usefull info.

4

What i meant is the grok filter seems to be working but when i add the date filter it creates problems.
I do not get the fields mentioned in the grok filter but if the remove the date filter part it does produce the fields present in grok filter.

Show us your configuration and an example input line.

Do i need to change anything to see the output in logs using rubydebug because currently i see error info and warnign messages and none of them has any usefull info.

If you're running Logstash in a terminal it should show up in your terminal. If you're running Logstash as a service I think it ends up in logstash.stdout in the log directory.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.