How to Read Real time data from url (.php real time file) and send data to elasticsearch? is there any method in logstash or filebeat to read real time data from url.?

Elastic Stack Logstash
1

TOPIC: Fetch data from url and send to Elasticsearch.

In php file (Ex: https://temeprature.co/temp.log) my real time data is written. i want to read this latest data log and send it to Elasticsearch how to do this task?

Read Url (for real time data) -->[what i can use here logstash/filebeat] --> Elasticsearch.

sample log data is:
07Sep2023 10:24:01 111001111011101011111011111011111011111011101011111011111011111011111001111011111011111011111011111011111011111011111011111011111011111001111011111011111011111011111011111011111011 END
07Sep2023 10:25:01 111001111011101011111011111011111011111011101011111011111011111011111001111011111011111011111011111011111011111011111011111011111011111001111011111011111011111011111011111011111011 END

2

You could try an http_poller input.

3

I have tried http poller plugin and configured logstash conf file to read data from http url but whenever file read the data after set interval it copy/send whole file to the elasticsearch index. so in Elasticsearch duplicate entry found of the data . Any solution's to this problem?

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.