# Logstash Shipping All Logs Except One

**URL:** <https://discuss.elastic.co/t/logstash-shipping-all-logs-except-one/46293>\
**Category:** Logstash\
**Created:** [April 4, 2016, 8:32pm UTC](https://discuss.elastic.co/t/logstash-shipping-all-logs-except-one/46293 "2016-04-04T20:32:47Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![JT1234567890](https://avatars.discourse-cdn.com/v4/letter/j/858c86/32.png) [@JT1234567890](https://discuss.elastic.co/u/JT1234567890)\
**Post date:** [April 4, 2016, 8:32pm UTC](https://discuss.elastic.co/t/logstash-shipping-all-logs-except-one/46293/1 "2016-04-04T20:32:47Z")

</div>

Hello,

I have a working Logstash configuration that ships syslog and Apache logs to a Graylog2 server - we love it, works great. Now we need to scrape an application's log file and ship it to the Graylog server too. I updated the Logstash configuration file and the syslog and Apache logs continue to flow, but the new logs only show up on the Graylog server under strange circumstances.

I had a problem with the grok filter for the new log. When entries were being tagged with \_grokparsefailure they would show up on the Graylog server _only_ if I ran Logstash in the foreground ( `/opt/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf -l /tmp/mylog.out` ). If I run it in daemon mode it ships syslog and Apache, but not myapp. Worked in foreground, did not work in the background.

Then, I fixed the grok filter problem. Now, when I run Logstash in debug mode I no longer see the \_grokparsefailure errors - but myapp logs aren't reaching the server. Syslog and Apache is still flowing to the Graylog server, but I only get silence from myapp.

At this point I'm stuck. myapp is generating the logs correctly on the local file system. Apache and syslog continue to get pushed to the Graylog server so it isn't an infrastructure issue. I believe it's a bug or, more likely, a misconfiguration by a newbie (me).

Any ideas on where to go next? Thanks in advance for your time!

**tl;dr** Added new log file to Logstash. It will push the log to the server _only_ when Logstash is running in the foreground and when the grok filter misses and spits out \_grokparsefailure. If I fix the grok filter, the logs are not sent to the server regardless of Logstash running the foreground or background. Existing logs continue to flow regardless of configuration.

---

<div class="post-metadata">

**Author:** ![JT1234567890](https://avatars.discourse-cdn.com/v4/letter/j/858c86/32.png) [@JT1234567890](https://discuss.elastic.co/u/JT1234567890)\
**Post date:** [April 4, 2016, 8:35pm UTC](https://discuss.elastic.co/t/logstash-shipping-all-logs-except-one/46293/2 "2016-04-04T20:35:20Z")

</div>

My config file. Please note I'm running Logstash 1.4.5 and upgrading might be quite an uphill battle.

input {  
syslog {  
type =\> "syslog"  
port =\> 5514  
}

file {  
type =\> "apache-error"  
path =\> "/var/log/apache2/apache\_error.log"  
add\_field =\> ["facility\_label", "apache/error"]  
}  
file {  
type =\> "mongos"  
path =\> "/var/log/mongos/mongos.log"  
add\_field =\> ["facility\_label", "mongos"]  
}  
file {  
type =\> "myapp"  
path =\> "/home/myapp/myapp/logs/current"  
add\_field =\> ["facility\_label", "myapp"]  
}  
}

filter {  
if [type] == "apache-access" {  
grok {  
pattern =\> "%{COMBINEDAPACHELOG}"  
}  
} else if [type] == "mongodb" {  
grok {  
match =\> { message =\> "nscanned:(?[0-9]+)._?nreturned:(?[0-9]+)._?(?[0-9]+)ms" }  
}  
} else if [type] == "myapp" {  
grok {  
match =\> { message =\> "%{TIMESTAMP\_ISO8601:time} [%{UUID:uuid}] %{GREEDYDATA:message}" }  
}  
}

mutate {  
replace =\> ["@source\_host", "myapp-server-01"]  
}  
}

output {  
gelf {  
host =\> "12.34.56.78"  
port =\> "12201"  
}  
}

---

<div class="post-metadata">

**Author:** ![JT1234567890](https://avatars.discourse-cdn.com/v4/letter/j/858c86/32.png) [@JT1234567890](https://discuss.elastic.co/u/JT1234567890)\
**Post date:** [April 11, 2016, 3:04pm UTC](https://discuss.elastic.co/t/logstash-shipping-all-logs-except-one/46293/3 "2016-04-11T15:04:49Z")

</div>

Any ideas on this one? To my newbie eyes it looks correct but the logs just aren't flowing.

---

<div class="post-metadata">

**Author:** ![JT1234567890](https://avatars.discourse-cdn.com/v4/letter/j/858c86/32.png) [@JT1234567890](https://discuss.elastic.co/u/JT1234567890)\
**Post date:** [April 12, 2016, 8:27pm UTC](https://discuss.elastic.co/t/logstash-shipping-all-logs-except-one/46293/4 "2016-04-12T20:27:31Z")

</div>

One final bump. The Mongo logs are flowing too - it's just the damn 'myapp' logs that aren't getting pushed to the server.

Thanks in advance.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:02am UTC](https://discuss.elastic.co/t/logstash-shipping-all-logs-except-one/46293/5 "2017-07-06T05:02:42Z")

</div>