So I am trying to backlog some older logs which do not have a MM:dd:YYY. My question is what should I do then? The timestamp will just be over written with the one of which logstash parsed the data. Thanks.
Mike
There's no timestamp in the file name or path?
Nope none of that. But There are logs in there that do have the mm:dd:YYYY stamps, its just a mess of logs. And after doing some experimenting logstash will actually throw a time stamp of year 1970 on it automatically when I use the date filter. So for example I have a log with only time 20:20:10,123 and I use a date filter that will match its parameters, it will append to that time a date also of 01-01-1970 (making up the month and day but the year is a 1970 default logstash appends).
So a better question might be how would I change the default date that logstash appends the 01-01-1970 after a date filter. Thanks.
Mike
Change it to what though? This is the problem, you have no indicator as to what LS should be setting the date to, so it just does it's best.
I understand, my idea is that maybe I should at least change the date to something more recent then 01-01-1970 because its a pain to go looking for it in kibana. So say that I know it cannot be after 05-05-2015 because there are logs dating this after the file. But should at least be a few days maybe even months old. So for example here are some dates I found:
20:34:59,110 blah blah
20:35:00,073 blah blah
20:35:00,229 blah blah
(then suddenly dates magically appear)
26-05-2015 blah blah
So I would like the date to be say the day before maybe 24-05-2015. How would I go about that?
You could do a conditional that if the date is missing, just use a default one, but that's pretty messy.