Hello,
New user, i'd like try the power of ELK on a debian 9.4 / openjdk version "1.8.0_151"
2 days I try to run ELK with no success so I decided to start again from the beginning and I encountered a issue fairly quickly.... Internet and Google is not my friend this time 
I have installed package DEB filebeat 6.3.2 and logstash 6.3.2-1 (lastet versions on the site).
I used the "getting start" guide https://www.elastic.co/guide/en/logstash/current/getting-started-with-logstash.html
The step 2 "Stashing your first event" in the guide) works fine
logstash -e 'input { stdin { } } output { stdout {} }'
It return as expeted but on this format :
bonjour
{
"@timestamp" => 2018-08-07T13:51:33.491Z,
"@version" => "1",
"message" => "bonjour",
"host" => "debian"
}
The step 3 "Parsing Logs with Logstash" doesn't work as expected and i think i respect all the guide. So, i put all my config files, command and output if necessary.
My filebeat config file is
root@debian:/etc/filebeat# cat filebeat.yml | egrep -v '^\ *#|^$'
filebeat.inputs:
- type: log
enabled: false
paths:
- /var/log/messages
filebeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: false
setup.template.settings:
index.number_of_shards: 3
setup.kibana:
output.logstash:
hosts: ["localhost:5044"]
My /var/log/message contains lines like
Aug 7 15:14:34 debian liblogging-stdlog: action 'action 0' resumed (module 'builtin:omfwd') [v8.24.0 try http://www.rsyslog.com/e/2359 ]
I also run some "logger" commands which are presents in /var/log/messages but nothing in logstash.
My logstash config file is
root@debian:/etc/filebeat# cat /etc/logstash/test.conf
input {
beats {
port => "5044"
}
}
# The filter part of this file is commented out to indicate that it is
# optional.
# filter {
#
# }
output {
stdout { codec => rubydebug }
}
I start filebeat like the user guide (foreground and debug mode)
filebeat -e -c filebeat.yml -d "publish"
...
2018-08-07T16:49:16.004+0200 INFO crawler/crawler.go:48 Loading Inputs: 1
2018-08-07T16:49:16.006+0200 INFO crawler/crawler.go:82 Loading and starting Inputs completed. Enabled inputs: 0
2018-08-07T16:49:16.004+0200 INFO [monitoring] log/log.go:97 Starting metrics logging every 30s
2018-08-07T16:49:16.008+0200 INFO cfgfile/reload.go:122 Config reloader started
2018-08-07T16:49:16.009+0200 INFO cfgfile/reload.go:214 Loading of config files completed.
And logstash...
First, i try the config.test_and_exit and the return is ok.
When I start the process now
/usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/test.conf --config.reload.automatic
I have the output below but nothing looks like a "normal" event as thrid part of the user guide.
root@debian:/etc/logstash# /usr/share/logstash/bin/logstash -f /etc/logstash/test.conf --config.reload.automatic
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continu ing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[WARN ] 2018-08-07 16:33:54.428 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified
[INFO ] 2018-08-07 16:33:57.166 [LogStash::Runner] runner - Starting Logstash {"logstash.version"=>"6.3.2"}
[INFO ] 2018-08-07 16:34:09.457 [Converge PipelineAction::Create<main>] pipeline - Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>1, "pipeline .batch.size"=>125, "pipeline.batch.delay"=>50}
[INFO ] 2018-08-07 16:34:11.222 [[main]-pipeline-manager] beats - Beats inputs: Starting input listener {:address=>"0.0.0.0:5044"}
[INFO ] 2018-08-07 16:34:11.270 [Converge PipelineAction::Create<main>] pipeline - Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0 x24fdd77a run>"}
[INFO ] 2018-08-07 16:34:11.674 [Ruby-0-Thread-1: /usr/share/logstash/lib/bootstrap/environment.rb:6] agent - Pipelines running {:count=>1, :running_pipelin es=>[:main], :non_running_pipelines=>[]}
[INFO ] 2018-08-07 16:34:11.714 [[main]<beats] Server - Starting server on port: 5044
[INFO ] 2018-08-07 16:34:12.895 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600}
[INFO ] 2018-08-07 16:34:34.313 [defaultEventExecutorGroup-5-1] BeatsHandler - [local: 127.0.0.1:5044, remote: 127.0.0.1:60474] Handling exception: org.logs tash.beats.BeatsParser$InvalidFrameProtocolException: Invalid Frame Type, received: 52
[WARN ] 2018-08-07 16:34:34.325 [nioEventLoopGroup-3-2] DefaultChannelPipeline - An exceptionCaught() event was fired, and it reached at the tail of the pip eline. It usually means the last handler in the pipeline did not handle the exception.
io.netty.handler.codec.DecoderException: org.logstash.beats.BeatsParser$InvalidFrameProtocolException: Invalid Frame Type, received: 52
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:459) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
at
... 8 more
[INFO ] 2018-08-07 16:34:34.537 [defaultEventExecutorGroup-5-1] BeatsHandler - [local: 127.0.0.1:5044, remote: 127.0.0.1:60474] Handling exception: org.logs tash.beats.BeatsParser$InvalidFrameProtocolException: Invalid Frame Type, received: 62
[WARN ] 2018-08-07 16:34:34.547 [nioEventLoopGroup-3-2] DefaultChannelPipeline - An exceptionCaught() event was fired, and it reached at the tail of the pip eline. It usually means the last handler in the pipeline did not handle the exception.
io.netty.handler.codec.DecoderException: org.logstash.beats.BeatsParser$InvalidFrameProtocolException: Invalid Frame Type, received: 62
at io.netty.channel.AbstractChannelHandlerContext$4.run(AbstractChannelHandlerContext.java:236) ~[netty-all-4.1.18.Final.jar:4.1.18.Final]
... 10 more
I loop on the same articles on the net without success. Hope someone will have mercy on me or simply give some investigation leads i will gracefully...
.