# Config file compatibility

**URL:** <https://discuss.elastic.co/t/config-file-compatibility/51743>\
**Category:** Logstash\
**Created:** [June 2, 2016, 8:39pm UTC](https://discuss.elastic.co/t/config-file-compatibility/51743 "2016-06-02T20:39:41Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![zozo6015](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zozo6015/32/12117_2.png) [@zozo6015](https://discuss.elastic.co/u/zozo6015)\
**Post date:** [June 2, 2016, 8:39pm UTC](https://discuss.elastic.co/t/config-file-compatibility/51743/1 "2016-06-02T20:39:41Z")

</div>

Hello,

I would like to ask if Logstash 5.0.0 alpha3 configuration is compatible with the version 2 config file?

I am asking cause I have copied my old version 2.x config file into /etc/logstash/conf.d/ but in elasticsearch does not seam that the information it is parsed by the config file.

If it's compatible how can I make it work?

Regards,  
Peter

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [June 2, 2016, 9:42pm UTC](https://discuss.elastic.co/t/config-file-compatibility/51743/2 "2016-06-02T21:42:33Z")

</div>

The pipeline file format is the same, but there are many plugins with changed and/or deprecated options.

What error did you get? How are you starting Logstash?

---

<div class="post-metadata">

**Author:** ![zozo6015](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zozo6015/32/12117_2.png) [@zozo6015](https://discuss.elastic.co/u/zozo6015)\
**Post date:** [June 2, 2016, 9:49pm UTC](https://discuss.elastic.co/t/config-file-compatibility/51743/3 "2016-06-02T21:49:13Z")

</div>

I don't get any error messages in the log files. I am parsing some nginx access.log with filebeat and forward it to logstash to parse the logs with some grok and also apply a geoip on the IP address and I observed that in the elasticsearch the information which is coming to logstash is stored as it is without being parsed by the grok and geoip filters.

I am starting the logstash using systemctl.

Regards,  
Peter

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [June 2, 2016, 9:51pm UTC](https://discuss.elastic.co/t/config-file-compatibility/51743/4 "2016-06-02T21:51:33Z")

</div>

I'd start by sending to stdout output first, and see if you can add verbose logging.

---

<div class="post-metadata">

**Author:** ![zozo6015](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zozo6015/32/12117_2.png) [@zozo6015](https://discuss.elastic.co/u/zozo6015)\
**Post date:** [June 2, 2016, 10:07pm UTC](https://discuss.elastic.co/t/config-file-compatibility/51743/5 "2016-06-02T22:07:15Z")

</div>

Putting logstash on verbose at startup I am getting the lots of line like this:

`{"timestamp":"2016-06-02T22:00:43.556000+0000","message":"Adding pattern","RAILS3FOOT":"Completed %{NUMBER:response}%{DATA} in %{NUMBER:totalms}ms %{RAILS3PROFILE}%{GREEDYDATA}","level":"info"} {"timestamp":"2016-06-02T22:00:43.558000+0000","message":"Adding pattern","RAILS3PROFILE":"(?:\\(Views: %{NUMBER:viewms}ms \\| ActiveRecord: %{NUMBER:activerecordms}ms|\\(ActiveRecord: %{NUMBER:activerecordms}ms)?","level":"info"} {"timestamp":"2016-06-02T22:00:43.558000+0000","message":"Adding pattern","RAILS3":"%{RAILS3HEAD}(?:%{RPROCESSING})?(?<context>(?:%{DATA}\\n)*)(?:%{RAILS3FOOT})?","level":"info"} {"timestamp":"2016-06-02T22:00:43.560000+0000","message":"Adding pattern","REDISTIMESTAMP":"%{MONTHDAY} %{MONTH} %{TIME}","level":"info"} {"timestamp":"2016-06-02T22:00:43.561000+0000","message":"Adding pattern","REDISLOG":"\\[%{POSINT:pid}\\] %{REDISTIMESTAMP:timestamp} \\* ","level":"info"} {"timestamp":"2016-06-02T22:00:43.563000+0000","message":"Adding pattern","RUBY_LOGLEVEL":"(?:DEBUG|FATAL|ERROR|WARN|INFO)","level":"info"} {"timestamp":"2016-06-02T22:00:43.564000+0000","message":"Adding pattern","RUBY_LOGGER":"[DFEWI], \\[%{TIMESTAMP_ISO8601:timestamp} #%{POSINT:pid}\\] *%{RUBY_LOGLEVEL:loglevel} -- +%{DATA:progname}: %{GREEDYDATA:message}","level":"info"} {"timestamp":"2016-06-02T22:00:43.596000+0000","message":"Using geoip database","path":"/etc/logstash/GeoLite2-City.mmdb","level":"info"} {"timestamp":"2016-06-02T22:00:43.645000+0000","message":"Starting pipeline","id":"main","pipeline.workers":4,"pipeline.batch.size":125,"pipeline.batch.delay":5,"pipeline.max_inflight":500,"level":"info"} {"timestamp":"2016-06-02T22:00:43.663000+0000","message":"Pipeline main started"}`

after the pipeline is started there is nothing more in the log.  
I have also put stdout { codec =\> json } but that does not change anything.

---

<div class="post-metadata">

**Author:** ![zozo6015](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zozo6015/32/12117_2.png) [@zozo6015](https://discuss.elastic.co/u/zozo6015)\
**Post date:** [June 2, 2016, 10:09pm UTC](https://discuss.elastic.co/t/config-file-compatibility/51743/6 "2016-06-02T22:09:01Z")

</div>

Also the logstash.conf looks like this:

`input { beats { host => "0.0.0.0" port => "5400" } } filter { if [type] == "nginx-access" { grok { match => { 'message' => '%{IPORHOST:clientip} %{USER:ident} %{USER:agent} \[%{HTTPDATE:timestamp}\] \"(?:%{WORD:verb} %{URIPATHPARAM:request}(?: HTTP/%{NUMBER:httpversion})?|)\" %{NUMBER:answer} (?:%{NUMBER:byte}|-) (?:\"(?:%{URI:referrer}|-))\" (?:%{QS:referree}) %{QS:agent}' } } geoip { source => "clientip" database => "/etc/logstash/GeoLite2-City.mmdb" } } } output { stdout { codec => json } elasticsearch { hosts => ["localhost:9200"] user => someuser password => somepass } }`

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 3, 2016, 3:39am UTC](https://discuss.elastic.co/t/config-file-compatibility/51743/7 "2016-06-03T03:39:16Z")

</div>

How do you know you're still getting data sent to the beats input?

---

<div class="post-metadata">

**Author:** ![zozo6015](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zozo6015/32/12117_2.png) [@zozo6015](https://discuss.elastic.co/u/zozo6015)\
**Post date:** [June 3, 2016, 6:42am UTC](https://discuss.elastic.co/t/config-file-compatibility/51743/8 "2016-06-03T06:42:38Z")

</div>

I have multiple checks, logstash index is increasing in size, logstash data shows up in elasticsearch unprocessed, tcpdump on the logstash/beats port on the server where filebeat is running show traffic.

---

<div class="post-metadata">

**Author:** ![zozo6015](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zozo6015/32/12117_2.png) [@zozo6015](https://discuss.elastic.co/u/zozo6015)\
**Post date:** [June 3, 2016, 11:45pm UTC](https://discuss.elastic.co/t/config-file-compatibility/51743/9 "2016-06-03T23:45:14Z")

</div>

no one has any idea why I am getting this issue?

---

<div class="post-metadata">

**Author:** ![zozo6015](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zozo6015/32/12117_2.png) [@zozo6015](https://discuss.elastic.co/u/zozo6015)\
**Post date:** [June 4, 2016, 3:02pm UTC](https://discuss.elastic.co/t/config-file-compatibility/51743/10 "2016-06-04T15:02:08Z")

</div>

I have figured it out, The new filebeat has two config file one normal and one full the normal one does not have the document\_type configuration and since my logstash config does need the document type to be nginx-access it does not parse the logfiles which came with the default document\_type: log. Once I configured the full file and renamed it to filebeat.yml everything started to work correctly.

However I am still not able to get the logs on logstash.

Regards,  
Peter

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:54am UTC](https://discuss.elastic.co/t/config-file-compatibility/51743/11 "2017-07-06T04:54:32Z")

</div>


