# Logstash listening on port 5000 and is able to connect with Filebeat but not parsing logs

**URL:** <https://discuss.elastic.co/t/logstash-listening-on-port-5000-and-is-able-to-connect-with-filebeat-but-not-parsing-logs/40329>\
**Category:** Logstash\
**Created:** [January 28, 2016, 7:15am UTC](https://discuss.elastic.co/t/logstash-listening-on-port-5000-and-is-able-to-connect-with-filebeat-but-not-parsing-logs/40329 "2016-01-28T07:15:50Z")\
**Posts on this page:** 19\
**Page:** 1

<div class="post-metadata">

**Author:** ![abinay](https://avatars.discourse-cdn.com/v4/letter/a/838e76/32.png) [@abinay](https://discuss.elastic.co/u/abinay)\
**Post date:** [January 28, 2016, 7:15am UTC](https://discuss.elastic.co/t/logstash-listening-on-port-5000-and-is-able-to-connect-with-filebeat-but-not-parsing-logs/40329/1 "2016-01-28T07:15:50Z")

</div>

Hii I have this architecture with me -  
Filebeat ships logs to logstash(listening on port 5000) . Logstash parses logs (i can see on my console too) and pushes them to elasticsearch. Now when I am running my Filebeat , it is able to connect to Logstash as it says "sending logs to logstash" . Further I am doing "tcpdump port 5000" on my Logstash server to see whether I am receiving any traffic or not . And the thing is that I am indeed receiving traffic at port 5000. But I am not seeing Logstash parsing any log and pushing to elasticsearch . What can be the issue ??? First I thought of some communication problem between Logstash and Filebeat due to certificate and key but then I thought this cannot be the case as Filebeat is able to connect to Logstash .If there would have been a certificate problem Filebeat could not have connected to Logstash right ????

---

<div class="post-metadata">

**Author:** ![abinay](https://avatars.discourse-cdn.com/v4/letter/a/838e76/32.png) [@abinay](https://discuss.elastic.co/u/abinay)\
**Post date:** [January 28, 2016, 7:17am UTC](https://discuss.elastic.co/t/logstash-listening-on-port-5000-and-is-able-to-connect-with-filebeat-but-not-parsing-logs/40329/2 "2016-01-28T07:17:25Z")

</div>

this is my Logstash configuration file . I am posting input { } , filter {} , and output{} separately.

input {

```
    beats {
            codec => multiline {
                            pattern => "^\["
                            what => "previous"
                            negate => true
             }
            ssl => true
            port => 5000
            ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
            ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
    }

```

}

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [January 28, 2016, 7:17am UTC](https://discuss.elastic.co/t/logstash-listening-on-port-5000-and-is-able-to-connect-with-filebeat-but-not-parsing-logs/40329/3 "2016-01-28T07:17:58Z")

</div>

Have you looked in the Logstash logs to confirm that it's getting the events and seeing whether it has any problems sending the logs to ES? Upping the log level with `--verbose` or `--debug` may provide additional insights.

---

<div class="post-metadata">

**Author:** ![abinay](https://avatars.discourse-cdn.com/v4/letter/a/838e76/32.png) [@abinay](https://discuss.elastic.co/u/abinay)\
**Post date:** [January 28, 2016, 7:18am UTC](https://discuss.elastic.co/t/logstash-listening-on-port-5000-and-is-able-to-connect-with-filebeat-but-not-parsing-logs/40329/4 "2016-01-28T07:18:10Z")

</div>

filter {  
grok {  
match =\> { "message" =\> "%{CISCO\_REASON}:%{ISO8601\_SECOND},%{ISO8601\_SECOND}%{DATA}%{LOGLEVEL:loglevel}"}  
match =\> { "message" =\> "%{SYSLOG5424SD} %{JAVACLASS:loglevel}" }

```
    }
    geoip {
            source => "clientip"
    }

    date {
            match => ["timestamp" , "dd/MMM/yyyy:HH:mm:ss Z"]
    }

```

}

---

<div class="post-metadata">

**Author:** ![abinay](https://avatars.discourse-cdn.com/v4/letter/a/838e76/32.png) [@abinay](https://discuss.elastic.co/u/abinay)\
**Post date:** [January 28, 2016, 7:18am UTC](https://discuss.elastic.co/t/logstash-listening-on-port-5000-and-is-able-to-connect-with-filebeat-but-not-parsing-logs/40329/5 "2016-01-28T07:18:33Z")

</div>

where can i see the logstash logs ??? i am working on ubuntu .

---

<div class="post-metadata">

**Author:** ![abinay](https://avatars.discourse-cdn.com/v4/letter/a/838e76/32.png) [@abinay](https://discuss.elastic.co/u/abinay)\
**Post date:** [January 28, 2016, 7:21am UTC](https://discuss.elastic.co/t/logstash-listening-on-port-5000-and-is-able-to-connect-with-filebeat-but-not-parsing-logs/40329/6 "2016-01-28T07:21:57Z")

</div>

I don't see anything when I go to /var/log/logstash

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [January 28, 2016, 7:26am UTC](https://discuss.elastic.co/t/logstash-listening-on-port-5000-and-is-able-to-connect-with-filebeat-but-not-parsing-logs/40329/7 "2016-01-28T07:26:11Z")

</div>

How are you starting Logstash?

---

<div class="post-metadata">

**Author:** ![abinay](https://avatars.discourse-cdn.com/v4/letter/a/838e76/32.png) [@abinay](https://discuss.elastic.co/u/abinay)\
**Post date:** [January 28, 2016, 7:28am UTC](https://discuss.elastic.co/t/logstash-listening-on-port-5000-and-is-able-to-connect-with-filebeat-but-not-parsing-logs/40329/8 "2016-01-28T07:28:07Z")

</div>

i am starting logstash by using the command -  
/opt/logstash/bin/logstash -f /etc/logstash/logstash.conf

---

<div class="post-metadata">

**Author:** ![abinay](https://avatars.discourse-cdn.com/v4/letter/a/838e76/32.png) [@abinay](https://discuss.elastic.co/u/abinay)\
**Post date:** [January 28, 2016, 7:30am UTC](https://discuss.elastic.co/t/logstash-listening-on-port-5000-and-is-able-to-connect-with-filebeat-but-not-parsing-logs/40329/9 "2016-01-28T07:30:47Z")

</div>

@magnusbaek can you specify the path for logstash logs . Doing -  
tail /var/log/logstash/logstash.log  
is of no use as the directory /var/log/logstash does not contain anything

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [January 28, 2016, 7:31am UTC](https://discuss.elastic.co/t/logstash-listening-on-port-5000-and-is-able-to-connect-with-filebeat-but-not-parsing-logs/40329/10 "2016-01-28T07:31:04Z")

</div>

Then Logstash will log to your terminal. I suggest you save the output to a file (e.g. with the `-l` flag) since enabling verbose output will produce a lot of text.

---

<div class="post-metadata">

**Author:** ![abinay](https://avatars.discourse-cdn.com/v4/letter/a/838e76/32.png) [@abinay](https://discuss.elastic.co/u/abinay)\
**Post date:** [January 28, 2016, 7:33am UTC](https://discuss.elastic.co/t/logstash-listening-on-port-5000-and-is-able-to-connect-with-filebeat-but-not-parsing-logs/40329/11 "2016-01-28T07:33:01Z")

</div>

@manusbaeck sorry don't get you . Do you want me to put the output of --debug into a file ?

---

<div class="post-metadata">

**Author:** ![abinay](https://avatars.discourse-cdn.com/v4/letter/a/838e76/32.png) [@abinay](https://discuss.elastic.co/u/abinay)\
**Post date:** [January 28, 2016, 7:33am UTC](https://discuss.elastic.co/t/logstash-listening-on-port-5000-and-is-able-to-connect-with-filebeat-but-not-parsing-logs/40329/12 "2016-01-28T07:33:59Z")

</div>

These kind of probelms are faced when there is a problem in the configuration. Do you see any problem with the configuration ?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [January 28, 2016, 7:35am UTC](https://discuss.elastic.co/t/logstash-listening-on-port-5000-and-is-able-to-connect-with-filebeat-but-not-parsing-logs/40329/13 "2016-01-28T07:35:21Z")

</div>

> Do you want me to put the output of --debug into a file ?

Yes, if you want to be able to inspect the data you're going to want to have it in a file.

---

<div class="post-metadata">

**Author:** ![abinay](https://avatars.discourse-cdn.com/v4/letter/a/838e76/32.png) [@abinay](https://discuss.elastic.co/u/abinay)\
**Post date:** [January 28, 2016, 7:37am UTC](https://discuss.elastic.co/t/logstash-listening-on-port-5000-and-is-able-to-connect-with-filebeat-but-not-parsing-logs/40329/14 "2016-01-28T07:37:20Z")

</div>

so how do I need to do that ?? can you tell me the command ? will it be like -  
/opt/logstash/bin/logstash -f /etc/logstash/logstash.conf -l /path/to/file

---

<div class="post-metadata">

**Author:** ![abinay](https://avatars.discourse-cdn.com/v4/letter/a/838e76/32.png) [@abinay](https://discuss.elastic.co/u/abinay)\
**Post date:** [January 28, 2016, 7:42am UTC](https://discuss.elastic.co/t/logstash-listening-on-port-5000-and-is-able-to-connect-with-filebeat-but-not-parsing-logs/40329/15 "2016-01-28T07:42:09Z")

</div>

@magnusbaeck what do you think can be the problem???

---

<div class="post-metadata">

**Author:** ![abinay](https://avatars.discourse-cdn.com/v4/letter/a/838e76/32.png) [@abinay](https://discuss.elastic.co/u/abinay)\
**Post date:** [January 28, 2016, 7:59am UTC](https://discuss.elastic.co/t/logstash-listening-on-port-5000-and-is-able-to-connect-with-filebeat-but-not-parsing-logs/40329/17 "2016-01-28T07:59:01Z")

</div>

@magnusbaeck Hey I am seeing a peculiar behavior . When I removed the codec plugin from input {} things worked fine . I need that codec plugin in order to merge stack traces of laravel logs with the log itself so that logstash does not treat the stack trace as separate logs . Any help with this ???? This should not happen , don't you think so ??? So where should I put my codec multiline ?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [January 28, 2016, 8:13am UTC](https://discuss.elastic.co/t/logstash-listening-on-port-5000-and-is-able-to-connect-with-filebeat-but-not-parsing-logs/40329/18 "2016-01-28T08:13:37Z")

</div>

If you have multiple different types of logs coming in and the multiline codec is not able to handle some of them, e.g. because each line does not start with the pattern the codec is looking for, you may need to specify multiple inputs and separate them.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [January 28, 2016, 8:15am UTC](https://discuss.elastic.co/t/logstash-listening-on-port-5000-and-is-able-to-connect-with-filebeat-but-not-parsing-logs/40329/19 "2016-01-28T08:15:15Z")

</div>

I don't think you can use the multiline codec with the beats input. The event arrives from Filebeat in structured form but the multiline codec assumes line-based input. You may be able to use the multiline filter instead, but ideally you should join the lines on the Filebeat side. That will be possible in Filebeat 1.1 which will be released soon. There are pre-release binaries to download or you can build the source yourself.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:13am UTC](https://discuss.elastic.co/t/logstash-listening-on-port-5000-and-is-able-to-connect-with-filebeat-but-not-parsing-logs/40329/20 "2017-07-06T05:13:59Z")

</div>


