# Logstash service behaving different than command line

**URL:** <https://discuss.elastic.co/t/logstash-service-behaving-different-than-command-line/63104>\
**Category:** Logstash\
**Created:** [October 15, 2016, 12:54pm UTC](https://discuss.elastic.co/t/logstash-service-behaving-different-than-command-line/63104 "2016-10-15T12:54:30Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![chrisan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrisan/32/12335_2.png) [@chrisan](https://discuss.elastic.co/u/chrisan)\
**Post date:** [October 15, 2016, 12:54pm UTC](https://discuss.elastic.co/t/logstash-service-behaving-different-than-command-line/63104/1 "2016-10-15T12:54:30Z")

</div>

Hello, I have 2 cloudfront distributions I am looking to analyze. The system is a fresh VM of Ubuntu 16.04 with latest Elasticsearch and Logstash installed via `apt-get`

I have the following 2 configs:

> <https://gist.github.com/chrisan/7312f5a894a49e0a3164699ccd4ed4a0>

In both cases I started with the `CLOUDFRONT_ACCESS_LOG` from [https://github.com/logstash-plugins/logstash-patterns-core/blob/master/patterns/aws](https://github.com/logstash-plugins/logstash-patterns-core/blob/master/patterns/aws) and added in some matches to the URL to perform some aggregates on.

When I run both of these via the command line

`$ /opt/logstash/bin/logstash -f /etc/logstash/conf.d/userlogs.conf`  
and  
`$ /opt/logstash/bin/logstash -f /etc/logstash/conf.d/companylogs.conf`

There are no parse errors and everything is coming out fine in elasticsearch queries.

I then reset everything with `curl -XDELETE localhost:9200/* && sudo rm -f /var/lib/logstash/.sincedb*`

I start the service `sudo service logstash start` and wait a few moments then try my search again, but this time I have _tons_ of `_grokparsefailure` and `_dateparsefailure` listed in tags.

> <https://gist.github.com/chrisan/91d6877876630c2c36dcb36218f8b767>

In the above you can see the search returned a type of `company_logs` but appears to be trying to grok with logs from the user cdn. There is 100% no path of `/v1/users` on the company CDN

Additionally while trying to debug the service you can see I added a `raw_data` field that just returns the original message `add_field => ["raw_data", "%{message}"]` What is odd is in the parse failures the raw\_data is spitting back out `"%{message}"` while when I run these configs from the command line that doesn't happen.

Right now I can only assume running these configs from the command line work because they are in isolation of each other being run one at a time, while the logstash service is loading both of the configs files and doing them in tandem.

What am I missing with setting up logstash as a service? I do not see any other config files in `/etc/logstash`

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [October 15, 2016, 2:20pm UTC](https://discuss.elastic.co/t/logstash-service-behaving-different-than-command-line/63104/2 "2016-10-15T14:20:24Z")

</div>

When you run Logstash as a service, it will concatenate both configuration files into one and all events will go through all filters as you are not using [conditionals](https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html#conditionals). Add a tag in each input so you can use this to apply the correct filters and output to each event using conditionals.

There are quite a few examples of this type of configuration in these forums, e.g. [this one](https://discuss.elastic.co/t/log-in-wrong-index/62740/6).

---

<div class="post-metadata">

**Author:** ![chrisan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrisan/32/12335_2.png) [@chrisan](https://discuss.elastic.co/u/chrisan)\
**Post date:** [October 15, 2016, 2:45pm UTC](https://discuss.elastic.co/t/logstash-service-behaving-different-than-command-line/63104/3 "2016-10-15T14:45:52Z")

</div>

Thank you Christian for the explanation and links. Adding a conditional to each block worked as desired

Much appreciated

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:34am UTC](https://discuss.elastic.co/t/logstash-service-behaving-different-than-command-line/63104/4 "2017-07-06T04:34:06Z")

</div>


