# How to Read Real time data from url (.php real time file) and send data to elasticsearch? is there any method in logstash or filebeat to read real time data from url.?

**URL:** <https://discuss.elastic.co/t/how-to-read-real-time-data-from-url-php-real-time-file-and-send-data-to-elasticsearch-is-there-any-method-in-logstash-or-filebeat-to-read-real-time-data-from-url/342572>\
**Category:** Logstash\
**Created:** [September 8, 2023, 5:52am UTC](https://discuss.elastic.co/t/how-to-read-real-time-data-from-url-php-real-time-file-and-send-data-to-elasticsearch-is-there-any-method-in-logstash-or-filebeat-to-read-real-time-data-from-url/342572 "2023-09-08T05:52:09Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![bbkunbi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bbkunbi/32/107767_2.png) [@bbkunbi](https://discuss.elastic.co/u/bbkunbi)\
**Post date:** [September 8, 2023, 5:52am UTC](https://discuss.elastic.co/t/how-to-read-real-time-data-from-url-php-real-time-file-and-send-data-to-elasticsearch-is-there-any-method-in-logstash-or-filebeat-to-read-real-time-data-from-url/342572/1 "2023-09-08T05:52:10Z")

</div>

TOPIC: Fetch data from url and send to Elasticsearch.

In php file (Ex: [https://temeprature.co/temp.log](https://temeprature.co/temp.log)) my real time data is written. i want to read this latest data log and send it to Elasticsearch how to do this task?

Read Url (for real time data) --\>[what i can use here logstash/filebeat] --\> Elasticsearch.

sample log data is:  
07Sep2023 10:24:01 111001111011101011111011111011111011111011101011111011111011111011111001111011111011111011111011111011111011111011111011111011111011111001111011111011111011111011111011111011111011 END  
07Sep2023 10:25:01 111001111011101011111011111011111011111011101011111011111011111011111001111011111011111011111011111011111011111011111011111011111011111001111011111011111011111011111011111011111011 END

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [September 8, 2023, 1:27pm UTC](https://discuss.elastic.co/t/how-to-read-real-time-data-from-url-php-real-time-file-and-send-data-to-elasticsearch-is-there-any-method-in-logstash-or-filebeat-to-read-real-time-data-from-url/342572/2 "2023-09-08T13:27:53Z")

</div>

You could try an [http\_poller](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-http_poller.html) input.

---

<div class="post-metadata">

**Author:** ![bbkunbi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bbkunbi/32/107767_2.png) [@bbkunbi](https://discuss.elastic.co/u/bbkunbi)\
**Post date:** [September 11, 2023, 4:31am UTC](https://discuss.elastic.co/t/how-to-read-real-time-data-from-url-php-real-time-file-and-send-data-to-elasticsearch-is-there-any-method-in-logstash-or-filebeat-to-read-real-time-data-from-url/342572/3 "2023-09-11T04:31:31Z")

</div>

I have tried http poller plugin and configured logstash conf file to read data from http url but whenever file read the data after set interval it copy/send whole file to the elasticsearch index. so in Elasticsearch duplicate entry found of the data . Any solution's to this problem?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 9, 2023, 4:31am UTC](https://discuss.elastic.co/t/how-to-read-real-time-data-from-url-php-real-time-file-and-send-data-to-elasticsearch-is-there-any-method-in-logstash-or-filebeat-to-read-real-time-data-from-url/342572/4 "2023-10-09T04:31:55Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.