# Filebeat not receiving logs and error in logstash
**URL:** https://discuss.elastic.co/t/filebeat-not-receiving-logs-and-error-in-logstash/240653
**Category:** Logstash
**Created:** [July 10, 2020, 7:53am UTC](https://discuss.elastic.co/t/filebeat-not-receiving-logs-and-error-in-logstash/240653 "2020-07-10T07:53:48Z")
**Posts on this page:** 14
**Page:** 1
### Author:  [@Sai\_Avinash\_Duddupud](https://discuss.elastic.co/u/Sai_Avinash_Duddupud)
#### Post date: [July 10, 2020, 7:53am UTC](https://discuss.elastic.co/t/filebeat-not-receiving-logs-and-error-in-logstash/240653/1 "2020-07-10T07:53:48Z")
I am trying to setup filebeat and logstash on my **server1** and send data to elasticsearch located on **server2** and visualize it using kibana.
Following are filebeat logs and when i run _ **filebeat test output** _ it showed the result as show in image bleow.
[](https://i.stack.imgur.com/L8UJk.png)
As you can observer, filbeat is not receving logs at all....is this because of the error in logstash?
```
2020-07-10T07:40:14.852Z DEBUG [input] input/input.go:141 Run input
2020-07-10T07:40:14.852Z DEBUG [input] log/input.go:191 Start next scan
2020-07-10T07:40:14.852Z DEBUG [input] log/input.go:212 input states cleaned up. Before: 0, After: 0, Pending: 0
2020-07-10T07:40:24.853Z DEBUG [input] input/input.go:141 Run input
2020-07-10T07:40:24.853Z DEBUG [input] log/input.go:191 Start next scan
2020-07-10T07:40:24.853Z DEBUG [input] log/input.go:212 input states cleaned up. Before: 0, After: 0, Pending: 0
2020-07-10T07:40:34.853Z DEBUG [input] input/input.go:141 Run input
2020-07-10T07:40:34.853Z DEBUG [input] log/input.go:191 Start next scan
2020-07-10T07:40:34.853Z DEBUG [input] log/input.go:212 input states cleaned up. Before: 0, After: 0, Pending: 0
2020-07-10T07:40:44.828Z INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":20},"total":{"ticks":170,"time":{"ms":4},"value":170},"user":{"ticks":150,"time":{"ms":4}}},"handles":{"limit":{"hard":1048576,"soft":1024},"open":7},"info":{"ephemeral_id":"4f97f60d-b9f4-451c-b9f2-1935988798b1","uptime":{"ms":840027}},"memstats":{"gc_next":10220512,"memory_alloc":5959576,"memory_total":24826152},"runtime":{"goroutines":21}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":0}},"system":{"load":{"1":0.03,"15":0.08,"5":0.06,"norm":{"1":0.015,"15":0.04,"5":0.03}}}}}}
2020-07-10T07:40:44.853Z DEBUG [input] input/input.go:141 Run input
2020-07-10T07:40:44.856Z DEBUG [input] log/input.go:191 Start next scan
2020-07-10T07:40:44.856Z DEBUG [input] log/input.go:212 input states cleaned up. Before: 0, After: 0, Pending: 0
2020-07-10T07:40:54.856Z DEBUG [input] input/input.go:141 Run input
2020-07-10T07:40:54.856Z DEBUG [input] log/input.go:191 Start next scan
2020-07-10T07:40:54.856Z DEBUG [input] log/input.go:212 input states cleaned up. Before: 0, After: 0, Pending: 0
2020-07-10T07:41:04.856Z DEBUG [input] input/input.go:141 Run input
2020-07-10T07:41:04.856Z DEBUG [input] log/input.go:191 Start next scan
2020-07-10T07:41:04.856Z DEBUG [input] log/input.go:212 input states cleaned up. Before: 0, After: 0, Pending: 0
2020-07-10T07:41:14.828Z INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":20},"total":{"ticks":180,"time":{"ms":8},"value":180},"user":{"ticks":160,"time":{"ms":8}}},"handles":{"limit":{"hard":1048576,"soft":1024},"open":7},"info":{"ephemeral_id":"4f97f60d-b9f4-451c-b9f2-1935988798b1","uptime":{"ms":870027}},"memstats":{"gc_next":10216448,"memory_alloc":5117872,"memory_total":25101640},"runtime":{"goroutines":21}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":0}},"system":{"load":{"1":0.02,"15":0.08,"5":0.05,"norm":{"1":0.01,"15":0.04,"5":0.025}}}}}}
2020-07-10T07:41:14.856Z DEBUG [input] input/input.go:141 Run input
2020-07-10T07:41:14.856Z DEBUG [input] log/input.go:191 Start next scan
2020-07-10T07:41:14.856Z DEBUG [input] log/input.go:212 input states cleaned up. Before: 0, After: 0, Pending: 0
2020-07-10T07:41:24.856Z DEBUG [input] input/input.go:141 Run input
2020-07-10T07:41:24.856Z DEBUG [input] log/input.go:191 Start next scan
2020-07-10T07:41:24.856Z DEBUG [input] log/input.go:212 input states cleaned up. Before: 0, After: 0, Pending: 0
2020-07-10T07:41:34.857Z DEBUG [input] input/input.go:141 Run input
2020-07-10T07:41:34.857Z DEBUG [input] log/input.go:191 Start next scan
2020-07-10T07:41:34.857Z DEBUG [input] log/input.go:212 input states cleaned up. Before: 0, After: 0, Pending: 0
```
Following are logstash logs and **I am getting some exception errors**
```
[WARN] 2020-07-10 07:26:21.802 [nioEventLoopGroup-2-2] DefaultChannelPipeline - An
exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: -1
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:472) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:278) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:38) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:353) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
at io.netty.util.concurrent.DefaultEventExecutor.run(DefaultEventExecutor.java:66) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:897) [netty-all-4.1.30.Final.jar:4.1.30.Final]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-all-4.1.30.Final.jar:4.1.30.Final]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_252]
Caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: -1
at org.logstash.beats.Protocol.version(Protocol.java:22) ~[logstash-input-beats-6.0.9.jar:?]
at org.logstash.beats.BeatsParser.decode(BeatsParser.java:62) ~[logstash-input-beats-6.0.9.jar:?]
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:502) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:441) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
... 8 more
```
**filebeat.yml**
```
filebeat.inputs:
- type: log
enabled: true
paths:
- /home/mahesh/Documents/refactor/nomi/unity/media/*.log
output.logstash:
enabled: true
hosts: ["localhost:5044"]
```
**logstash.conf**
```
input {
beats {
port => 5044
ssl => false
}
}
filter {
grok {
match => { "message" => "%{TIMESTAMP_ISO8601:timestamp}] %{LOGLEVEL:loglevel}\|%{GREEDYDATA:module}\|%{GREEDYDATA:content}" }
}
date {
locale => "en"
match => ["timestamp", "YYYY-MM-dd HH:mm:ss"]
target => "@timestamp"
timezone => "America/New_York"
}
}
output {
elasticsearch {
hosts => "elk_server_ip:9200"
manage_template => false
index => "blend_test"
}
stdout { codec => rubydebug { metadata => true } }
}
```
---
### Author:  [@grumo35](https://discuss.elastic.co/u/grumo35)
#### Post date: [July 10, 2020, 9:02am UTC](https://discuss.elastic.co/t/filebeat-not-receiving-logs-and-error-in-logstash/240653/2 "2020-07-10T09:02:31Z")
First, can you show us logstash and filebeat version ?
Are they up to date ?
---
### Author:  [@Sai\_Avinash\_Duddupud](https://discuss.elastic.co/u/Sai_Avinash_Duddupud)
#### Post date: [July 10, 2020, 9:20am UTC](https://discuss.elastic.co/t/filebeat-not-receiving-logs-and-error-in-logstash/240653/3 "2020-07-10T09:20:25Z")
@grumo35
Filebeat version - 7.8.0
logstash version - 7.8.0
And to my surprise, there are no errors in logstash console, but filebeat logs are the same.... filbeat is not harvesting logs... and I have checked again and again that i have given the correct log file path.
---
### Author:  [@grumo35](https://discuss.elastic.co/u/grumo35)
#### Post date: [July 10, 2020, 9:23am UTC](https://discuss.elastic.co/t/filebeat-not-receiving-logs-and-error-in-logstash/240653/4 "2020-07-10T09:23:04Z")
What's the rights of the log files ?
---
### Author:  [@Sai\_Avinash\_Duddupud](https://discuss.elastic.co/u/Sai_Avinash_Duddupud)
#### Post date: [July 10, 2020, 9:36am UTC](https://discuss.elastic.co/t/filebeat-not-receiving-logs-and-error-in-logstash/240653/5 "2020-07-10T09:36:29Z")
@grumo35
`-rw-rw-r-- 1 ubuntu ubuntu 118366 Jul 10 09:27 django_blend.log`
Changed above one to using **chmod 777 command**
`-rwxrwxrwx 1 ubuntu ubuntu 121496 Jul 10 09:43 django_blend.log`
but still not receving logs even after restarting filebeat
---
### Author:  [@grumo35](https://discuss.elastic.co/u/grumo35)
#### Post date: [July 10, 2020, 10:02am UTC](https://discuss.elastic.co/t/filebeat-not-receiving-logs-and-error-in-logstash/240653/6 "2020-07-10T10:02:17Z")
when using tail -f /home/mahesh/Documents/refactor/nomi/unity/media/\*.log
Are they any newlines created ?
It's strage that filebeat isnt catchign anything, maybe the directory is not accessible ?
---
### Author:  [@Jiboss](https://discuss.elastic.co/u/Jiboss)
#### Post date: [July 10, 2020, 10:11am UTC](https://discuss.elastic.co/t/filebeat-not-receiving-logs-and-error-in-logstash/240653/7 "2020-07-10T10:11:11Z")
Hi,
You don't need to use filebeat if all the data came to the same host. You can add this directly in logstash :
```auto
input {
file {
path => "/path/to/file"
}
}
```
---
### Author:  [@Sai\_Avinash\_Duddupud](https://discuss.elastic.co/u/Sai_Avinash_Duddupud)
#### Post date: [July 10, 2020, 10:17am UTC](https://discuss.elastic.co/t/filebeat-not-receiving-logs-and-error-in-logstash/240653/8 "2020-07-10T10:17:49Z")
@grumo35 no new lines are present.
`drwxrwxr-x+` is the permission config for that folder where log file is present
---
### Author:  [@grumo35](https://discuss.elastic.co/u/grumo35)
#### Post date: [July 10, 2020, 10:21am UTC](https://discuss.elastic.co/t/filebeat-not-receiving-logs-and-error-in-logstash/240653/9 "2020-07-10T10:21:55Z")
If there is no newlines that's why filebeat isnt seeing new content, i guess your log files aren't populated ?
---
### Author:  [@Sai\_Avinash\_Duddupud](https://discuss.elastic.co/u/Sai_Avinash_Duddupud)
#### Post date: [July 10, 2020, 10:22am UTC](https://discuss.elastic.co/t/filebeat-not-receiving-logs-and-error-in-logstash/240653/10 "2020-07-10T10:22:03Z")
@Jiboss, u want me to relpace input node in logstash.conf with above input snippet you provided?
---
### Author:  [@Jiboss](https://discuss.elastic.co/u/Jiboss)
#### Post date: [July 10, 2020, 10:29am UTC](https://discuss.elastic.co/t/filebeat-not-receiving-logs-and-error-in-logstash/240653/12 "2020-07-10T10:29:03Z")
Yeah, if you don't need filebeat I would do that. I think you need to receive new data to see your logs. Try to send another file.
---
### Author:  [@Puneeth\_S\_B\_Gowda1](https://discuss.elastic.co/u/Puneeth_S_B_Gowda1)
#### Post date: [July 10, 2020, 11:34am UTC](https://discuss.elastic.co/t/filebeat-not-receiving-logs-and-error-in-logstash/240653/13 "2020-07-10T11:34:21Z")
Try with 7.6.2 version as this was the solution for my same issue it might work for you too
---
### Author:  [@Sai\_Avinash\_Duddupud](https://discuss.elastic.co/u/Sai_Avinash_Duddupud)
#### Post date: [July 12, 2020, 4:07pm UTC](https://discuss.elastic.co/t/filebeat-not-receiving-logs-and-error-in-logstash/240653/14 "2020-07-12T16:07:52Z")
Update :- it finally worked after I changed the filebeat.yml to the following config
```
filebeat.inputs:
- type: log
paths:
- /your/path/goes/here/*.log
output.logstash:
enabled: true
hosts: ["localhost:5044"]
logging.level: info
logging.to_files: true
logging.files:
path: /var/log/filebeat
name: filebeat
keepfiles: 7
permissions: 0644
```
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 11, 2020, 2:14am UTC](https://discuss.elastic.co/t/filebeat-not-receiving-logs-and-error-in-logstash/240653/16 "2020-08-11T02:14:24Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.