Hi there. I was not able to find a solution for my problem that is why I am creating this post. My problem is that we are sending logs to elasticsearch from linux nodes using either UTC or EST as their timezones. For EST nodes logs in Kibana are being displayed according to the time, but for linux nodes using UTC timezone I need to set my time for FUTURE(Next 5 hours) in Kibana in order to see the latest logs. My logstash filer is the following, but I am not even sure if Logstash is the one that needs to be fixed for this.
Thank you for quick response. I think I like this approach, what CONDITION should I be checking on if my entire filter right now looks like this? Should I be checking in my CONDITION if logs are coming from UTC then ..... else timezone => "US/Eastern"? Thank you in advance.
Hard to say. You say some hosts are UTC and some are US/Eastern. You need some test to tell you what timezone a given event is in. It would be possible to test the hostname that sent it, but depending on the rest of your configuration there may be other ways. For example, if the events are being shipped by filebeat you may be able to have filebeat tag them at source. Again, if you are using beats you might be able to send all of the UTC events to one port and all of the US/Eastern events to another, then tag them with the input. There are any number of possibilities.
Thank you very much. I do use filebeat to send logs from our linux nodes. Now, that I know the source of my issue I can start fixing it. Thank you again @Badger.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.