Logstash service behaving different than command line

Hello, I have 2 cloudfront distributions I am looking to analyze. The system is a fresh VM of Ubuntu 16.04 with latest Elasticsearch and Logstash installed via apt-get

I have the following 2 configs:

In both cases I started with the CLOUDFRONT_ACCESS_LOG from https://github.com/logstash-plugins/logstash-patterns-core/blob/master/patterns/aws and added in some matches to the URL to perform some aggregates on.

When I run both of these via the command line

$ /opt/logstash/bin/logstash -f /etc/logstash/conf.d/userlogs.conf
$ /opt/logstash/bin/logstash -f /etc/logstash/conf.d/companylogs.conf

There are no parse errors and everything is coming out fine in elasticsearch queries.

I then reset everything with curl -XDELETE localhost:9200/* && sudo rm -f /var/lib/logstash/.sincedb*

I start the service sudo service logstash start and wait a few moments then try my search again, but this time I have tons of _grokparsefailure and _dateparsefailure listed in tags.

In the above you can see the search returned a type of company_logs but appears to be trying to grok with logs from the user cdn. There is 100% no path of /v1/users on the company CDN

Additionally while trying to debug the service you can see I added a raw_data field that just returns the original message add_field => [ "raw_data", "%{message}" ] What is odd is in the parse failures the raw_data is spitting back out "%{message}" while when I run these configs from the command line that doesn't happen.

Right now I can only assume running these configs from the command line work because they are in isolation of each other being run one at a time, while the logstash service is loading both of the configs files and doing them in tandem.

What am I missing with setting up logstash as a service? I do not see any other config files in /etc/logstash

When you run Logstash as a service, it will concatenate both configuration files into one and all events will go through all filters as you are not using conditionals. Add a tag in each input so you can use this to apply the correct filters and output to each event using conditionals.

There are quite a few examples of this type of configuration in these forums, e.g. this one.

Thank you Christian for the explanation and links. Adding a conditional to each block worked as desired

Much appreciated