Could not load data from filebeat into logstash

Elastic Stack Beats
1

Greetings everyone!
I want to feed my log file from fealbeat into logstash. But i am not able to do it. I am receiving these errors.

[2018-06-05T11:30:44,112][INFO ][logstash.agent           ] Pipelines running {:count=>1, :pipelines=>["main"]}
[2018-06-05T11:32:05,045][INFO ][org.logstash.beats.BeatsHandler] [local: 0:0:0:0:0:0:0:1:5044, remote: 0:0:0:0:0:0:0:1:31903] Handling exception: org.logstash.beats.BeatsParser$InvalidFrameProtocolException: Invalid Frame Type, received: 69
org.logstash.beats.BeatsParser$InvalidFrameProtocolException: Invalid Frame Type, received: 84

My filebeat.yml file:

My config file:

# The # character at the beginning of a line indicates a comment. Use
# comments to describe your configuration.
input {
beats {
        port => "5044"
    }
}
output {
elasticsearch {
        hosts => [ "localhost:9200" ]
	}
}

My log data(logstash-tutorial-dataset):

83.149.9.216 - - [04/Jan/2015:05:13:42 +0000] "GET /presentations/logstash-monitorama-2013/images/kibana-search.png HTTP/1.1" 200 203023 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"
83.149.9.216 - - [04/Jan/2015:05:13:42 +0000] "GET /presentations/logstash-monitorama-2013/images/kibana-dashboard3.png HTTP/1.1" 200 171717 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"

I could not figure out where did i go wrong. I have tried everything . Data is going from logstash(stdin) to elasticsearch but not from filebeat->logstash->elasticsearch. I am a noob.

2

Hi,

I've checked your config file and found that you didn't enable prospector parameter. this is the reason your log files are not harvesting.

Your config:

  # Change to true to enable this prospector configuration.
  enabled: false

Please set this true like below and restart filebeat service:

# Change to true to enable this prospector configuration.
enabled: true

Regards,

3

Hey, I had also tried by changing it to true but it was not working beforehand so i toggled it back to false .
The main reason was there were no spaces between '[' and ' " ' , I added the spaces like
host : [ "localhost:5044" ]
and it started working.

And yes, Prosecutor enabled must be true.
Thank you so much anyway.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.