# Issue in Logstash after Upgrade

**URL:** https://discuss.elastic.co/t/issue-in-logstash-after-upgrade/123421
**Category:** Logstash
**Created:** [March 11, 2018, 10:36am UTC](https://discuss.elastic.co/t/issue-in-logstash-after-upgrade/123421 "2018-03-11T10:36:24Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![sreejiths](https://avatars.discourse-cdn.com/v4/letter/s/4491bb/32.png) [@sreejiths](https://discuss.elastic.co/u/sreejiths)
#### Post date: [March 11, 2018, 10:36am UTC](https://discuss.elastic.co/t/issue-in-logstash-after-upgrade/123421/1 "2018-03-11T10:36:24Z")

</div>

I tested upgrade of ELK stack to V5.6.8 from V5.2.2 running on Centos 7 VM . Elasticsearch upgrade & Kibana upgrade Went fine ..But logstash ran into issue details below

testing Logatsh config :  
input {  
udp {  
type =\> "arista"  
port =\> "514"  
host =\> "x.x.x.x"  
}

```
     tcp {
   ssl_verify => false
   type => "arista"
   port => "514"
   host => "x.x.x.x"
 }
}

```

output {  
if [type] == "arista" {  
elasticsearch {  
ssl =\> false  
ssl\_certificate\_verification =\> false  
hosts =\> ["x.X.X.X:9200"]  
index =\> "ipsla-%{+YYYY.MM.dd}"  
}  
}  
}

Error :  
[2018-03-11T18:32:44,041][ERROR][logstash.pipeline] A plugin had an unrecoverable error. Will restart this plugin.  
Plugin: \<LogStash::Inputs::Tcp ssl\_verify=\>false, type=\>"arista", port=\>514, host=\>"10.67.10.208", id=\>"6b81d8f1ee654ee08ae4c5ac55c7a190737fb508-2", enable\_metric=\>true, codec=\>\<LogStash::Codecs::Line id=\>"line\_923f272b-69ab-4156-a39d-5c12be7043aa", enable\_metric=\>true, charset=\>"UTF-8", delimiter=\>"\n"\>, data\_timeout=\>-1, mode=\>"server", proxy\_protocol=\>false, ssl\_enable=\>false, ssl\_key\_passphrase=\>\>  
Error: Permission denied

When i enable only UDP port on the logstash config it gives me another error

[2018-03-11T18:40:23,004][WARN][logstash.inputs.udp] UDP listener died {:exception=\>#\<SocketError: bind: name or service not known\>, :backtrace=\>["org/jruby/ext/socket/RubyUDPSocket.java:164:in `bind'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-udp-3.2.1/lib/logstash/inputs/udp.rb:95:in`udp\_listener'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-udp-3.2.1/lib/logstash/inputs/udp.rb:56:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:470:in`inputworker'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:463:in `start\_input'"]}

What will be the reason for the issue ? ...Please help

---

<div class="post-metadata">

### Author: ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)
#### Post date: [March 11, 2018, 9:42pm UTC](https://discuss.elastic.co/t/issue-in-logstash-after-upgrade/123421/2 "2018-03-11T21:42:38Z")

</div>

I found a relevant issue on github, with comments that should provide a path forward:

> Ports in range 1 to 1024 are privileged and only root user can listen on it.
> 
> Options:
> 
> - run logstash as root (not a good idea)
> - use setcap to grant java permission to use privileged ports
> - use iptables or a proxy to forward port 514 to an unprivileged port.
> 
> -- [logstash will not open a listening port. · Issue #1587 · elastic/logstash · GitHub](https://github.com/elastic/logstash/issues/1587#issuecomment-50939823)

This blog post is specifically about using `setcap` to set capabilities for Java (although it references Java 7, and Logstash currently requires Java 8, so you may need to adjust it slightly):

> **[Oracle Blogs](https://blogs.oracle.com/)**
>
> Oracle Blogs

---

<div class="post-metadata">

### Author: ![sreejiths](https://avatars.discourse-cdn.com/v4/letter/s/4491bb/32.png) [@sreejiths](https://discuss.elastic.co/u/sreejiths)
#### Post date: [March 12, 2018, 7:52am UTC](https://discuss.elastic.co/t/issue-in-logstash-after-upgrade/123421/3 "2018-03-12T07:52:29Z")

</div>

Hi yaauie ..Appreciate your help ..The logstash is now listening on specific ports on the Config

But for some reason its not accepting logs from the clients ..Below is the test logstash config ..I dont see any logs on ruby debug ..I can see logs coming on ruby debug when i add self hearbeat in the input section ..I believe some issue in Input plugin ..Not sure ..Please advise

####NAGIOS###  
input {  
udp {  
type =\> "nagios"  
port =\> "1514"  
host =\> "X.X.X.X"  
}

```
 tcp {
   type => "nagios"
   ssl_verify => false
   port => "1514"
   host => "X.X.X.X"
 }
}

```

filter {

```
if [type] == "nagios" {

```

grok {

```
     match => [
        # IOS
        "message", "%{IP:host_nagios}: Nagios-Log device_id=%{WORD:hostname} rtt=%{NUMBER:RTT} avgSD=%{NUMBER:SDLATENCY} avgDS=%

```

{NUMBER:DSLATENCY} syslog\_sev\_level=%{INT:syslog\_sev\_level} syslog\_severity=%{WORD:syslog\_severity} hostgroup=%{WORD:hostgroup} %{GR  
EEDYDATA:log\_message}"  
]  
add\_tag =\> ["Nagios"]  
}  
}

if "Nagios" in [tags]  
{

mutate { convert =\> { "syslog\_sev\_level" =\> "integer" } }  
mutate { convert =\> { "RTT" =\> "float" } }  
mutate { convert =\> { "SDLATENCY" =\> "float" } }  
mutate { convert =\> { "DSLATENCY" =\> "float" } }

```
   }

```

}

output {  
if [type] == "nagios" {  
elasticsearch {  
hosts =\> ["X.X.X.X:9200"]  
index =\> "nagios-%{+YYYY.MM.dd}"

# user =\> "XXXXX"

# password =\> "XXXXXX"

```
    }

```

}

# For troubleshoot debug use

file{  
path =\> "/tmp/nagios\_1514.txt"  
codec =\> rubydebug  
}  
}

---

<div class="post-metadata">

### Author: ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)
#### Post date: [March 12, 2018, 8:12am UTC](https://discuss.elastic.co/t/issue-in-logstash-after-upgrade/123421/4 "2018-03-12T08:12:33Z")

</div>

In future, adding markdown code blocks around blocks of code (a line with three tildes like `~~~` will begin or end a block) makes reading it a lot easier.

* * *

What is sending the logs? Is it configured to send to port 1514, and has it been restarted recently to pick up the change? (When you opened this ticket, things were pointed at port 514).

---

<div class="post-metadata">

### Author: ![sreejiths](https://avatars.discourse-cdn.com/v4/letter/s/4491bb/32.png) [@sreejiths](https://discuss.elastic.co/u/sreejiths)
#### Post date: [March 12, 2018, 9:17am UTC](https://discuss.elastic.co/t/issue-in-logstash-after-upgrade/123421/5 "2018-03-12T09:17:54Z")

</div>

We understood the problem with using 514 ..Was testing one of the logstash config hearing on port 1514 which is higher port that 1024 ..When pointing logstash config to port 1514 the instance is up as below

[root@elkuat01 tmp]#  
[root@elkuat01 tmp]# netstat -an | grep 1514  
tcp6 0 0 10.67.10.208:1514 :::\* LISTEN  
udp6 0 0 10.67.10.208:1514 :::\*

But more some reason the logs is not been read to Elasticsearch & also not seen in rubydebug.But in tcpdump on the VM shows traffic from the client on port 1514 ..Not sure what is happening ..

---

<div class="post-metadata">

### Author: ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)
#### Post date: [March 13, 2018, 10:33pm UTC](https://discuss.elastic.co/t/issue-in-logstash-after-upgrade/123421/6 "2018-03-13T22:33:38Z")

</div>

Synopsis: Logstash is apparently _listening_ on the configured port (in this case 1514), but is apparently not receiving any connections.

We can manually send a message to Logstash with [`nc`](https://linux.die.net/man/1/nc) to validate that it is really listening (replace the `127.0.0.1` with Logstash's external interface IP when testing from another machine):

```auto
echo "test-tcp" | nc -w0 127.0.0.1 1514

```

```auto
echo "test-udp" | nc -w0 -u 127.0.0.1 1514

```

If you see these messages, then Logstash is in fact bound to the ports for both tcp and udp, and the problem lies somewhere between your sender and Logstash (e.g., Logstash inputs being bound to local-only IP address, no route from your other host, firewall blocking port ranges)

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [April 10, 2018, 10:34pm UTC](https://discuss.elastic.co/t/issue-in-logstash-after-upgrade/123421/7 "2018-04-10T22:34:07Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
