# Logstash not executing after pipeline main started

**URL:** <https://discuss.elastic.co/t/logstash-not-executing-after-pipeline-main-started/64531>\
**Category:** Logstash\
**Created:** [November 1, 2016, 8:55am UTC](https://discuss.elastic.co/t/logstash-not-executing-after-pipeline-main-started/64531 "2016-11-01T08:55:29Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![vann](https://avatars.discourse-cdn.com/v4/letter/v/eada6e/32.png) [@vann](https://discuss.elastic.co/u/vann)\
**Post date:** [November 1, 2016, 8:55am UTC](https://discuss.elastic.co/t/logstash-not-executing-after-pipeline-main-started/64531/1 "2016-11-01T08:55:29Z")

</div>

I get stuck in this situation for logstash not executing after pipeline main started.

> Settings: Default pipeline workers: 4  
> Pipeline main started

I have checked

1. I have "touch" the .csv so the file last modify date is less tahn one day
2. I have added sincedb\_path =\> "/dev/null" in the input {file{..}} session
3. I have deleted the .sincedb\_.. in logstash directory.
4. I found no extra line in my .csv file

When I try to run the configuration with --debug it keeps "Flushing buffer at interval", and but I fail to get the index in kibana.

Thank you so much.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [November 1, 2016, 9:30am UTC](https://discuss.elastic.co/t/logstash-not-executing-after-pipeline-main-started/64531/2 "2016-11-01T09:30:43Z")

</div>

Ignore the "Flushing buffer at interval" noise and look for anything related to the input file. If you post the debug log somewhere (e.g. in a gist) we can help.

---

<div class="post-metadata">

**Author:** ![vann](https://avatars.discourse-cdn.com/v4/letter/v/eada6e/32.png) [@vann](https://discuss.elastic.co/u/vann)\
**Post date:** [November 1, 2016, 9:58am UTC](https://discuss.elastic.co/t/logstash-not-executing-after-pipeline-main-started/64531/3 "2016-11-01T09:58:52Z")

</div>

> New Elasticsearch output {:class=\>"LogStash::Outputs::Elasticsearch", :hosts=\>["localhost:9200"], :level=\>:info, :file=\>"logstash/outputs/elasticsearch/common.rb", :line=\>"19", :method=\>"register"}  
> Pipeline started {:level=\>:info, :file=\>"logstash/pipeline.rb", :line=\>"109", :method=\>"run"}

> Logstash startup completed

> Flushing buffer at interval {:instance=\>"#\<LogStash::Outputs::Elasticsearch::Buffer:0x5d43ff72 @ operations\_mutex=#Mutex:0x53840aaa, @ max\_size=500, @ operations\_lock=#Java::JavaUtilConcurrentLocks::ReentrantLock:0x57bb9402, @submit\_proc=#Proc:0x5043c5db@/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.2.0-java/lib/logstash/outputs/elasticsearch/common.rb:55, @ logger=#\<Cabin::Channel:0x23615423 @ metrics=#\<Cabin::Metrics:0x2871cfc9 @metrics\_lock=#Mutex:0x7296ffde, @metrics={}, @channel=#\<Cabin::Channel:0x23615423 ...\>\>, @ subscriber\_lock=#Mutex:0x3c35814a, @level=:debug, @ subscribers={12590=\>#\<Cabin::Outputs::IO:0x33758056 @ io=#\<IO:fd 1\>, @lock=#Mutex:0x76d8e1e9\>}, @data={}\>, @ last\_flush=2016-11-01 14:58:40 +0800, @flush\_interval=1, @ stopping=#Concurrent::AtomicBoolean:0x66f711a9, @ buffer=, @flush\_thread=#\<Thread:0x138b06d6 run\>\>", :interval=\>1, :level=\>:info, :file=\>"logstash/outputs/elasticsearch/buffer.rb", :line=\>"90", :method=\>"interval\_flush"}

Then it keeps looping in the flushing buffer at interval.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [November 1, 2016, 10:02am UTC](https://discuss.elastic.co/t/logstash-not-executing-after-pipeline-main-started/64531/4 "2016-11-01T10:02:14Z")

</div>

That's not the whole debug log.

I strongly recommend that you don't enable any elasticsearch outputs until you've confirmed that data arrives to a `stdout { codec => rubydebug }` output.

---

<div class="post-metadata">

**Author:** ![vann](https://avatars.discourse-cdn.com/v4/letter/v/eada6e/32.png) [@vann](https://discuss.elastic.co/u/vann)\
**Post date:** [November 2, 2016, 3:43am UTC](https://discuss.elastic.co/t/logstash-not-executing-after-pipeline-main-started/64531/5 "2016-11-02T03:43:35Z")

</div>

> \_globbed\_files: /my/directory/\w+.csv: glob is: {:level=\>:debug, :file=\>"filewatch/watch.rb", :line=\>"346", :method=\>"\_globbed\_files"}  
> Pushing flush onto pipeline {:level=\>:debug, :file=\>"logstash/pipeline.rb", :line=\>"458", :method=\>"flush"}  
> Pushing flush onto pipeline {:level=\>:debug, :file=\>"logstash/pipeline.rb", :line=\>"458", :method=\>"flush"}  
> Pushing flush onto pipeline {:level=\>:debug, :file=\>"logstash/pipeline.rb", :line=\>"458", :method=\>"flush"}

is that the problem?  
since i found it keeps looping among these lines.

Additionally, if I `#sincedb_path => "/dev/null"` I found .sincedb is created.  
While it has nothing to pop after \>Settings: Default pipeline workers: 4 \>Pipeline main started

I have tried out stdout {codec =\> rubydebug }  
still couldn't figure out the problem. ☹

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [November 2, 2016, 4:59am UTC](https://discuss.elastic.co/t/logstash-not-executing-after-pipeline-main-started/64531/6 "2016-11-02T04:59:18Z")

</div>

So you have "/my/directory/\w+.csv" in your file input's `path` setting? Regexps aren't supported there, only glob patterns.

---

<div class="post-metadata">

**Author:** ![vann](https://avatars.discourse-cdn.com/v4/letter/v/eada6e/32.png) [@vann](https://discuss.elastic.co/u/vann)\
**Post date:** [November 2, 2016, 5:41am UTC](https://discuss.elastic.co/t/logstash-not-executing-after-pipeline-main-started/64531/7 "2016-11-02T05:41:25Z")

</div>

Thank you so much !

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:31am UTC](https://discuss.elastic.co/t/logstash-not-executing-after-pipeline-main-started/64531/8 "2017-07-06T04:31:36Z")

</div>