Sort by time received instead of timestamp

Elastic Stack Elasticsearch
1

Hello,

im using ELK-Stack to search through Cisco logs received by filebeat from our loghost.
the problem is, i only can sort the discover by Timestamp from Elasticsearch and not by the timestamp the message was received from the loghost.
Usualy its not a big deal cause the timestamps doenst differ much, but if you have rebuild an index the timestamps are totaly out of order.
So can i use the original timestamp the log was received by the loghost instead of the timestamp elasticsearch received the log from filebeat?

I cant let elasticsearch receive the logs directly cause i would have to change the config of about 400-500 switches to send the logs to elasticsearch and the loghost.

2

How about create Data View ("Index pattern" for older versions) with "original timestamp" as the time field?

3

Well i would like to do that but there are some problems

  1. cisco logfile timestamps are in the format Jul 6 2022 10:36:01.447 UTC
    and i can only disaable the year, the miliseconds and the timezone so the timestamp would change to Jul 6 10:36:01. but i cant change the format itself.

  2. when i create a Data View i can only select @timestamp or --- i dont want to use time filter --- but not the field i want to use as a timestamp

so this seems to be no option or im doing something terribly wrong.

4

Simple sorting is not enough?

image
image2596×888 187 KB

image
image2595×944 218 KB

5

im using Elasticsearch 8 and @timestamp is the only field with an option to sort a-z or z-a
at all other fields sort is greyed out and therefore not available and thats the problem.

6

I'd recommend using Date processor | Elasticsearch Guide [master] | Elastic to convert the cisco timestamps to date type. Then you should be able to use it as a sort in a data view. Or by default it will write to @timestamp which might fit your needs.

If you still want to time the log was written to ES you can use a set processor for that as shown on Event Fields | Elastic Common Schema (ECS) Reference [8.3] | Elastic

1 Like
7

Thank you @matschaffer , that solved the problem even cisco seems to be not shure what timeformat they should use, but i think it works now.
Anyway that seemd to raised some other problems like reading loglines multiple times but thats for another topic.

Thanks again for that hint.

1 Like
8

You're very welcome and glad you got it sorted out!

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.