# Logstash Stuck at Pipeline Main Started

**URL:** <https://discuss.elastic.co/t/logstash-stuck-at-pipeline-main-started/56471>\
**Category:** Logstash\
**Created:** [July 27, 2016, 5:26am UTC](https://discuss.elastic.co/t/logstash-stuck-at-pipeline-main-started/56471 "2016-07-27T05:26:26Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Deepak\_Patnaik](https://avatars.discourse-cdn.com/v4/letter/d/73ab20/32.png) [@Deepak\_Patnaik](https://discuss.elastic.co/u/Deepak_Patnaik)\
**Post date:** [July 27, 2016, 5:26am UTC](https://discuss.elastic.co/t/logstash-stuck-at-pipeline-main-started/56471/1 "2016-07-27T05:26:26Z")

</div>

Hi,  
I ran logstash to index an apache log file. It worked. After that I used an application log file. It also worked. Then all of a sudden when I try to re-execute the apache log file, logstash is stuck at 'Pipeline main started'  
I tried to increase the LS\_heapsize to 2048m.

Please help asap

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [July 27, 2016, 5:36am UTC](https://discuss.elastic.co/t/logstash-stuck-at-pipeline-main-started/56471/2 "2016-07-27T05:36:47Z")

</div>

How are you starting it?  
Can you show us the output?  
What does your config look like?

---

<div class="post-metadata">

**Author:** ![Deepak\_Patnaik](https://avatars.discourse-cdn.com/v4/letter/d/73ab20/32.png) [@Deepak\_Patnaik](https://discuss.elastic.co/u/Deepak_Patnaik)\
**Post date:** [July 27, 2016, 5:44am UTC](https://discuss.elastic.co/t/logstash-stuck-at-pipeline-main-started/56471/3 "2016-07-27T05:44:03Z")

</div>

**Starting Process:**

/opt/logstash/bin$ sudo ./logstash agent -v -f /etc/logstash/indexer.conf

**Output:**

Adding pattern {"HTTPDATE"=\>"%{MONTHDAY}/%{MONTH}/%{YEAR}:%{TIME} %{INT}", :level=\>:info}  
Adding pattern {"QS"=\>"%{QUOTEDSTRING}", :level=\>:info}  
Adding pattern {"SYSLOGBASE"=\>"%{SYSLOGTIMESTAMP:timestamp} (?:%{SYSLOGFACILITY} )?%{SYSLOGHOST:logsource} %{SYSLOGPROG}:", :level=\>:info}  
Adding pattern {"COMMONAPACHELOG"=\>"%{IPORHOST:clientip} %{HTTPDUSER:ident} %{USER:auth} \[%{HTTPDATE:timestamp}\] "(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})" %{NUMBER:response} (?:%{NUMBER:bytes}|-)", :level=\>:info}  
Adding pattern {"COMBINEDAPACHELOG"=\>"%{COMMONAPACHELOG} %{QS:referrer} %{QS:agent}", :level=\>:info}  
Adding pattern {"HTTPD20\_ERRORLOG"=\>"\[%{HTTPDERROR\_DATE:timestamp}\] \[%{LOGLEVEL:loglevel}\] (?:\[client %{IPORHOST:clientip}\] ){0,1}%{GREEDYDATA:errormsg}", :level=\>:info}  
Adding pattern {"HTTPD24\_ERRORLOG"=\>"\[%{HTTPDERROR\_DATE:timestamp}\] \[%{WORD:module}:%{LOGLEVEL:loglevel}\] \pid %{POSINT:pid}:tid %{NUMBER:tid}\?( \[client %{IPORHOST:client}:%{POSINT:clientport}\])? %{DATA:errorcode}: %{GREEDYDATA:message}", :level=\>:info}  
Adding pattern {"HTTPD\_ERRORLOG"=\>"%{HTTPD20\_ERRORLOG}|%{HTTPD24\_ERRORLOG}", :level=\>:info}  
Adding pattern {"LOGLEVEL"=\>"([Aa]lert|ALERT|[Tt]race|TRACE|[Dd]ebug|DEBUG|[Nn]otice|NOTICE|[Ii]nfo|INFO|[Ww]arn?(?:ing)?|WARN?(?:ING)?|[Ee]rr?(?:or)?|ERR?(?:OR)?|[Cc]rit?(?:ical)?|CRIT?(?:ICAL)?|[Ff]atal|FATAL|[Ss]evere|SEVERE|EMERG(?:ENCY)?|[Ee]merg(?:ency)?)", :level=\>:info}  
Using geoip database {:path=\>"/var/log/logstash/ETLLogs/GeoLiteCity.dat", :level=\>:info}  
Starting pipeline {:id=\>"main", :pipeline\_workers=\>1, :batch\_size=\>125, :batch\_delay=\>5, :max\_inflight=\>125, :level=\>:info}  
Pipeline main started

**Config**

input  
{  
file {  
type =\> "apache"  
path =\> ['/var/log/logstash/ETLLogs/apache2.log']  
start\_position =\> "end"  
ignore\_older =\> 0  
sincedb\_path =\> "dev/null"

```
}

```

}  
filter  
{  
grok {  
match =\> { "message" =\> "%{COMBINEDAPACHELOG}" }  
}  
date {  
match =\> ["timestamp" , "dd/MMM/yyyy:HH:mm:ss Z"]  
}

```
geoip {
	source => "clientip"
	target => "geoip"
	database =>"/var/log/logstash/ETLLogs/GeoLiteCity.dat"  
	add_field => ["[geoip][coordinates]", "%{[geoip][longitude]}"]
	add_field => ["[geoip][coordinates]", "%{[geoip][latitude]}"]
}

mutate {
	convert => ["[geoip][coordinates]", "float"]
}

```

}

output  
{  
elasticsearch {  
index =\> "logstash-%{+YYYY.MM.dd}"  
hosts =\> ["localhost:9200"]  
}  
stdout {  
codec =\> rubydebug  
}  
}

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [July 27, 2016, 5:51am UTC](https://discuss.elastic.co/t/logstash-stuck-at-pipeline-main-started/56471/4 "2016-07-27T05:51:26Z")

</div>

> [@Deepak\_Patnaik](#):
>
> sincedb\_path =\> "dev/null"

Either that is a typo or it is not what you think it is.

I'd say that the sincedb is your problem.

---

<div class="post-metadata">

**Author:** ![Deepak\_Patnaik](https://avatars.discourse-cdn.com/v4/letter/d/73ab20/32.png) [@Deepak\_Patnaik](https://discuss.elastic.co/u/Deepak_Patnaik)\
**Post date:** [July 27, 2016, 6:03am UTC](https://discuss.elastic.co/t/logstash-stuck-at-pipeline-main-started/56471/5 "2016-07-27T06:03:13Z")

</div>

Okay, When I remove the sincedb\_path from the Config file, it works.

My question is what should be the sincedb\_path. Should it be just the directory or a file.

Reference:

> <https://github.com/elastic/logstash/issues/1535>

Make sincedb\_path accept a directory. Perhaps hinting at directory usage with a trailing slash, example: sincedb\_path =\> "/some/path/"

---

<div class="post-metadata">

**Author:** ![Deepak\_Patnaik](https://avatars.discourse-cdn.com/v4/letter/d/73ab20/32.png) [@Deepak\_Patnaik](https://discuss.elastic.co/u/Deepak_Patnaik)\
**Post date:** [July 27, 2016, 6:09am UTC](https://discuss.elastic.co/t/logstash-stuck-at-pipeline-main-started/56471/6 "2016-07-27T06:09:50Z")

</div>

Hi walkom,

One weird thing is, i deleted the sincedb\_path statement in the config file. Saved the file, then added it back, it worked. I really need to understand if it is some issue with the Logstash.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [July 27, 2016, 6:15am UTC](https://discuss.elastic.co/t/logstash-stuck-at-pipeline-main-started/56471/7 "2016-07-27T06:15:37Z")

</div>

I think you want `/dev/null` if you don't want to worry about it.

---

<div class="post-metadata">

**Author:** ![Deepak\_Patnaik](https://avatars.discourse-cdn.com/v4/letter/d/73ab20/32.png) [@Deepak\_Patnaik](https://discuss.elastic.co/u/Deepak_Patnaik)\
**Post date:** [July 27, 2016, 6:17am UTC](https://discuss.elastic.co/t/logstash-stuck-at-pipeline-main-started/56471/8 "2016-07-27T06:17:11Z")

</div>

Great!

How can I miss it... 😀

Thankyou Mark!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:46am UTC](https://discuss.elastic.co/t/logstash-stuck-at-pipeline-main-started/56471/9 "2017-07-06T04:46:22Z")

</div>


