now i want to read this time stamp from the input file name and create an index along with file content, which i will use later to plot graph against time stamp vs file-data-values.
grok { match => { "someField" => "free_data_(?<ts>[0-9_]+).txt" } }
date { match => [ "ts", "dd_MM_HH_mm" ] }
You do not have a year in your timestamp, so the date filter will guess, and sometimes you will not like its guess. There are issues for that here and here.
facing this problem..
2019-04-03T20:42:54.626+0530 INFO log/harvester.go:216 Harvester started for file: /home/vankata/190_APS_QUALIFICATION/kalyan_elk_logs/free_data_11_09_02_30.txt
2019-04-03T20:42:54.626+0530 INFO prospector/prospector.go:121 Prospector ticker stopped
2019-04-03T20:42:54.626+0530 INFO log/prospector.go:411 Scan aborted because prospector stopped.
2019-04-03T20:42:54.626+0530 INFO prospector/prospector.go:121 Prospector ticker stopped
2019-04-03T20:42:54.626+0530 INFO prospector/prospector.go:138 Stopping Prospector: 8728499415371259904
2019-04-03T20:42:54.626+0530 INFO prospector/prospector.go:121 Prospector ticker stopped
2019-04-03T20:42:54.626+0530 INFO log/harvester.go:216 Harvester started for file: /home/vankata/190_APS_QUALIFICATION/kalyan_elk_logs/free_data_11_09_10_40.txt
filebeat version: filebeat-6.2.4
Note: tried to delete registry files by using clean_* command couldn't succeed though
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.