# Logstash doesn't load data until interrupted

**URL:** <https://discuss.elastic.co/t/logstash-doesnt-load-data-until-interrupted/142970>\
**Category:** Logstash\
**Created:** [August 3, 2018, 8:38pm UTC](https://discuss.elastic.co/t/logstash-doesnt-load-data-until-interrupted/142970 "2018-08-03T20:38:39Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![emanresu](https://avatars.discourse-cdn.com/v4/letter/e/9d8465/32.png) [@emanresu](https://discuss.elastic.co/u/emanresu)\
**Post date:** [August 3, 2018, 8:38pm UTC](https://discuss.elastic.co/t/logstash-doesnt-load-data-until-interrupted/142970/1 "2018-08-03T20:38:39Z")

</div>

My cURL command in the input below invokes an https endpoint that will stream data until interrupted. My config below works but no data is loaded to Elasticsearch until I interrupt Logstash. Logstash keeps the data in memory until I do Ctrl^C to kill the pipeline.

```
input {
   exec { 
      command => 'curl -X GET "https://feed.test.com/1.0/json/A98F049" -H "api_key:zBAHA"'
      interval => 600000000 
      condec => "json_lines"
   }
filter {
   json { source => message}
}
}
output {
  stdout {
    codec => rubydebug
  }
elasticsearch {
   hosts => "localhost:9200"
   index => "abc"
  }
}

```

My screen shows some stats like this which update in real time showing time since start and bytes received; bytes xfered stays zero:

> % Total % Received % Xferd Average Speed Time Time Time Current  
> 0 0 0 0 0 0 0 0--:--:-- 0:00:28 --:--:-- 887

but only after killing the pipeline, the data shows up on the screen and in Elasticsearch. How can I index the data as it arrives continuously? Changing the interval value didn't make a real change.

---

<div class="post-metadata">

**Author:** ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)\
**Post date:** [August 3, 2018, 8:44pm UTC](https://discuss.elastic.co/t/logstash-doesnt-load-data-until-interrupted/142970/2 "2018-08-03T20:44:28Z")

</div>

The Exec Input Plugin doesn't support streaming. It waits for the process to complete and then passes the output through to the codec to generate Events.

---

<div class="post-metadata">

**Author:** ![emanresu](https://avatars.discourse-cdn.com/v4/letter/e/9d8465/32.png) [@emanresu](https://discuss.elastic.co/u/emanresu)\
**Post date:** [August 3, 2018, 9:09pm UTC](https://discuss.elastic.co/t/logstash-doesnt-load-data-until-interrupted/142970/3 "2018-08-03T21:09:37Z")

</div>

Thank you for clearing the dust. It has been a long road to Exec Input. I had started with http and http\_poller as shown here: [How to set up input for https curl and header key?](https://discuss.elastic.co/t/how-to-set-up-input-for-https-curl-and-header-key/142231/7) without success. Which plugin is best-practice for streaming that supports continuous indexing?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 31, 2018, 9:09pm UTC](https://discuss.elastic.co/t/logstash-doesnt-load-data-until-interrupted/142970/4 "2018-08-31T21:09:40Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.