Get the right timestamp for old log files

Elastic Stack Logstash
62

Use the remove_field parameter of any filter, preferably the date filter that makes the mytimestamp field superfluous.

63

So I should put the the remove_field parameter right under the match parameter under the date filter?

64

Anywhere in the date filter will do, but putting it at the end makes sense.

65

I have used the same timestamp format to target the @timestamp with the timestamp of my log file. But I got the _dateparsefailure and _grokparsefailure.

This is my config file

input
{
file
{

path => "C:\Users\eagasur\logs1.log"
start_position => "beginning"

}

}
filter
{
grok
{
match => [ "message", "%{MONTHDAY:day}/%{MONTHNUM:month}/%{YEAR:year}%:%{HOUR:hour}:%{MINUTE:minute}:%{SECOND:second} %{ISO8601_TIMEZONE:timezone} %{LOGLEVEL:log_level} %{NUMBER:line:int}" ]
}

ruby
{
code=> "hr=event['hour'].to_i ;
min = event['minute'].to_i ;
sec = event['second'].to_i;
hr_to_sec = hr * 60 * 60;
min_to_sec = min * 60;
total= hr_to_sec + min_to_sec + sec ;
event['time']=total * 1000;
event['difference'] = event['time'].to_i - var1;
var1=event['time'].to_i"
}
mutate
{
add_field => { "timestamp" => "%{day}/%{month}/%{year}:%{hour}:%{minute}:%{second} %{timezone}" }
}
date
{
match => ["timestamp", "dd/MMM/YYYY:HH:mm:ss Z" ]
}
}

output
{
stdout
{
codec => rubydebug{}
}
}

66

@Surbhi_Agarwal, please start a new topic for your unrelated question.

67

@simonrisberg @magnusbaeck

I have used the same procedure for targetting my timestamp with the logstash timestamp. But got _dateparsefailure and _grokparsefailure.

This is my config file.Could you please find out the error :
input
{
file
{
path => "C:\Users\eagasur\logs1.log"
start_position => "beginning"

}

}

filter
{
grok
{
match => [ "message", "%{MONTHDAY:day}/%{MONTHNUM:month}/%{YEAR:year}:%{HOUR:hour}:%{MINUTE:minute}:%{SECOND:second} %{ISO8601_TIMEZONE:timezone}" ]
}
mutate
{
add_field =>
{
"timestamp" => "%{day}/%{month}/%{year}:%{hour}:%{minute}:%{second} %{timezone}"
}
}
date
{
match => ["timestamp", "dd/MMM/YYYY:HH:mm:ss Z" ]
}
}

output
{
stdout
{
codec => rubydebug{}
}
}

68

Would you please upload your configuration.