I am currently working on log ingestion and visualization in Kibana and have successfully configured multiple outputs. At present, I have four separate indices corresponding to different log sources:
* Firewall logs
* Syslog-ng logs
* UTM logs
* Server logs
While analyzing the data in Kibana, I observed that only the firewall logs are displayed correctly in descending order (latest to oldest) based on the timestamp. However, the other three indices (syslog-ng, UTM, and server logs) are not appearing in the proper chronological order.
This inconsistency is affecting log analysis and troubleshooting, as it becomes difficult to correlate events across different sources.
Could you please help me understand:
* Why only one index (firewall) is correctly sorted by time?
* What could be causing improper ordering in the other indices?
* Are there any recommended configurations or best practices to ensure consistent timestamp-based sorting across all indices in Kibana?
Please let me know if any additional details, configurations, or sample logs are required for further investigation.
Please find the attached screenshot for your reference regarding the issue observed.
Currently, I am using a single data view for all logs. The issue is specifically seen in the Discover section of Kibana, where the logs are not appearing in proper chronological order.
For context, the log flow in our setup is as follows: syslog-ng → Logstash → Elasticsearch → Kibana
However, only firewall logs are displayed correctly in descending (latest-to-oldest) order, while logs from other sources (syslog-ng, UTM, and server logs) are not following the expected time sequence.
Kindly let me know if any additional information is required from my end.
Looking at the screenshot it seems the dataview is not saved and you are reviewing the index individually from index management & than clicking on discover index.
Request you to go to discover , create a data view
Give the index patterns & timestamp field (note the field should be available in all the indices else it will only show those records in the dataview where this field is present ) :
I have created the data view as suggested, and within the Discover section, I am able to see the logs correctly in descending order (recent to old).
However, my concern is slightly different. When I open individual indices (such as the firewall index or server index) from Index Management and then view them in Discover, the logs are not displayed in descending order (latest to oldest) as expected.
Ideally, I would like all indices (firewall, server, syslog-ng, etc.) to consistently show logs from most recent to oldest by default.
Additionally, I would like to mention that I was unable to create the Kibana sample data (eCommerce). However, I have successfully ingested my own logs, and the data view appears to be working correctly.
Could you please guide me on:
How to ensure all indices display logs in descending order by default
Whether this behavior is related to index settings, timestamp field mapping, or Discover configuration
Try creating a data view on Discover page for each index pattern separately (for example “syslog-ng*”) and make sure to select “@timestamp“ in the “Create data view“ flyout too.
Links from Index Management page don’t include a time field. That could be improved in the future versions.
The e-commerce dataview was just an example and you can ignore that part. The correct way to view the documents is via dataview where you need to select the timestamp field by which it will sort your data. When you directly open the index it will only show you the data no sorting as per the timestamp field. Now you have 2 ways if all the indices has common timestamp field than you can give all the index pattern for 1 dataview with the timestamp field and it will show all the data under 1 dataview. If you do not have a common timestamp name field or you want to review the indices 1 at a time, create different dataviews for each index pattern.
I would like to inform you that I have configured separate data views for each index pattern as per your suggestion. I am now able to view the logs in the correct order, from most recent to older entries, within the Discover page.
Thank you for your valuable support and guidance in resolving this matter. Your assistance is greatly appreciated.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
