I have been trying to ingest some, time series data into elasticsearch using logstash http poller plugin.
my data is from a monitoring tool which capture the parameters like hostname, memory usage, cpu load, system status and time when it captured the data
hostname : server1 memory load : 95% cpu load : 80% system status : Up time: 01-08-2021 12:35:44
When i run logstash using the provided rest api, for every poll it ingests the complete data into elasticsearch, but i want logstash to read the files based on the latest time recorded, so that i can get rid of duplicate data.
Any suggestions please.