im using ELK-Stack to search through Cisco logs received by filebeat from our loghost.
the problem is, i only can sort the discover by Timestamp from Elasticsearch and not by the timestamp the message was received from the loghost.
Usualy its not a big deal cause the timestamps doenst differ much, but if you have rebuild an index the timestamps are totaly out of order.
So can i use the original timestamp the log was received by the loghost instead of the timestamp elasticsearch received the log from filebeat?
I cant let elasticsearch receive the logs directly cause i would have to change the config of about 400-500 switches to send the logs to elasticsearch and the loghost.
How about create Data View ("Index pattern" for older versions) with "original timestamp" as the time field?
Well i would like to do that but there are some problems
cisco logfile timestamps are in the format Jul 6 2022 10:36:01.447 UTC
and i can only disaable the year, the miliseconds and the timezone so the timestamp would change to Jul 6 10:36:01. but i cant change the format itself.
when i create a Data View i can only select @timestamp or --- i dont want to use time filter --- but not the field i want to use as a timestamp
so this seems to be no option or im doing something terribly wrong.
Simple sorting is not enough?
im using Elasticsearch 8 and @timestamp is the only field with an option to sort a-z or z-a
at all other fields sort is greyed out and therefore not available and thats the problem.
I'd recommend using Date processor | Elasticsearch Guide [master] | Elastic to convert the cisco timestamps to date type. Then you should be able to use it as a sort in a data view. Or by default it will write to
@timestamp which might fit your needs.
If you still want to time the log was written to ES you can use a set processor for that as shown on Event Fields | Elastic Common Schema (ECS) Reference [8.3] | Elastic
Thank you @matschaffer , that solved the problem even cisco seems to be not shure what timeformat they should use, but i think it works now.
Anyway that seemd to raised some other problems like reading loglines multiple times but thats for another topic.
Thanks again for that hint.
You're very welcome and glad you got it sorted out!
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.