Hi Guys.
Need input from you all to get my setting right.
I am running my ELK stack in docker with sebp/elk.
And I have another instance with my Laravel App. Everything goess well until I try to configure filebeat in my Laravel app. Below is my configurations:
Configuration for logstash's beats-input.
input {
beats {
port => 5044
}
}
Configuration for filebeat.yml
#----------------------------- Logstash output --------------------------------
output.logstash:
enabled: true
# The Logstash hosts
hosts: ["127.0.0.1:5044"]
# Optional SSL. By default is off.
Log file for filebeat:
2018-03-23T11:23:26.639+0800 INFO crawler/crawler.go:82 Loading and starting Prospectors completed. Enabled prospectors: 1
2018-03-23T11:23:26.645+0800 INFO cfgfile/reload.go:127 Config reloader started
2018-03-23T11:23:26.645+0800 INFO log/harvester.go:216 Harvester started for file: /var/log/alternatives.log
2018-03-23T11:23:26.646+0800 INFO cfgfile/reload.go:219 Loading of config files completed.
2018-03-23T11:23:26.648+0800 INFO log/harvester.go:216 Harvester started for file: /var/log/apt/history.log
2018-03-23T11:23:26.653+0800 INFO log/harvester.go:216 Harvester started for file: /var/log/nginx/error.log
2018-03-23T11:23:26.655+0800 INFO log/harvester.go:216 Harvester started for file: /var/www/storage/logs/laravel.log
2018-03-23T11:23:26.655+0800 INFO log/harvester.go:216 Harvester started for file: /var/log/supervisor/supervisord.log
2018-03-23T11:23:26.655+0800 INFO log/harvester.go:216 Harvester started for file: /var/log/dpkg.log
2018-03-23T11:23:26.655+0800 INFO log/harvester.go:216 Harvester started for file: /var/log/apt/term.log
2018-03-23T11:23:26.655+0800 INFO log/harvester.go:216 Harvester started for file: /var/log/nginx/access.log
2018-03-23T11:23:26.658+0800 INFO log/harvester.go:216 Harvester started for file: /var/log/fontconfig.log
2018-03-23T11:23:27.691+0800 ERROR pipeline/output.go:74 Failed to connect: dial tcp 127.0.0.1:5044: getsockopt: connection refused
2018-03-23T11:23:29.692+0800 ERROR pipeline/output.go:74 Failed to connect: dial tcp 127.0.0.1:5044: getsockopt: connection refused
2018-03-23T11:23:33.693+0800 ERROR pipeline/output.go:74 Failed to connect: dial tcp 127.0.0.1:5044: getsockopt: connection refused
2018-03-23T11:23:41.693+0800 ERROR pipeline/output.go:74 Failed to connect: dial tcp 127.0.0.1:5044: getsockopt: connection refused
2018-03-23T11:23:56.600+0800 INFO [monitoring] log/log.go:124 Non-zero metrics in the last 30s
2018-03-23T11:23:57.696+0800 ERROR pipeline/output.go:74 Failed to connect: dial tcp 127.0.0.1:5044: getsockopt: connection refused
2018-03-23T11:24:26.599+0800 INFO [monitoring] log/log.go:124 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":40,"time":40},"total":{"ticks":140,"time":140,"value":140},"user":{"ticks":100,"time":100}},"info":{"ephemeral_id":"e6083c7a-95ac-4a3c-9d6f-62cbf130f7c8","uptime":{"ms":60009}},"memstats":{"gc_next":9202928,"memory_alloc":5202832,"memory_total":16519784}},"filebeat":{"harvester":{"open_files":9,"running":9}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":4117,"retry":2048}}},"registrar":{"states":{"current":10}},"system":{"load":{"1":0.01,"15":0.14,"5":0.15,"norm":{"1":0.005,"15":0.07,"5":0.075}}}}}}
2018-03-23T11:24:29.696+0800 ERROR pipeline/output.go:74 Failed to connect: dial tcp 127.0.0.1:5044: getsockopt: connection refused
2018-03-23T11:24:41.137+0800 INFO beater/filebeat.go:323 Stopping filebeat
2018-03-23T11:24:41.138+0800 INFO crawler/crawler.go:109 Stopping Crawler
2018-03-23T11:24:41.139+0800 INFO crawler/crawler.go:119 Stopping 1 prospectors
2018-03-23T11:24:41.139+0800 INFO prospector/prospector.go:121 Prospector ticker stopped
2018-03-23T11:24:41.139+0800 INFO prospector/prospector.go:138 Stopping Prospector: 10104363474912593672
2018-03-23T11:24:41.140+0800 INFO log/harvester.go:237 Reader was closed: /var/log/nginx/error.log. Closing.
2018-03-23T11:24:41.140+0800 INFO log/harvester.go:237 Reader was closed: /var/log/nginx/access.log. Closing.
2018-03-23T11:24:41.139+0800 INFO cfgfile/reload.go:222 Dynamic config reloader stopped
2018-03-23T11:24:41.142+0800 INFO log/harvester.go:237 Reader was closed: /var/log/fontconfig.log. Closing.
2018-03-23T11:24:41.142+0800 INFO log/harvester.go:237 Reader was closed: /var/log/apt/term.log. Closing.
Partial log file from logstash-plain.log
check to see if an Elasticsearch connection is working {:healthcheck_url=>http://
/localhost:9200/, :path=>"/"}
[2018-03-23T03:18:44,497][WARN ][logstash.outputs.elasticsearch] Restored connecc
tion to ES instance {:url=>"http://localhost:9200/"}
[2018-03-23T03:18:44,553][INFO ][logstash.outputs.elasticsearch] ES Output versii
on determined {:es_version=>nil}
[2018-03-23T03:18:44,557][WARN ][logstash.outputs.elasticsearch] Detected a 6.x
and above cluster: the type event field won't be used to determine the documenn
t _type {:es_version=>6}
[2018-03-23T03:18:44,581][INFO ][logstash.outputs.elasticsearch] New Elasticsearr
ch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//localhost"]}
[2018-03-23T03:18:45,361][INFO ][logstash.inputs.beats ] Beats inputs: Startii
ng input listener {:address=>"0.0.0.0:5044"}
[2018-03-23T03:18:45,447][INFO ][logstash.pipeline ] Pipeline started succ
cesfully {:pipeline_id=>"main", :thread=>"#<Thread:0x6fe447e8 run>"}
[2018-03-23T03:18:45,555][INFO ][org.logstash.beats.Server] Starting server on pp
ort: 5044
[2018-03-23T03:18:45,617][INFO ][logstash.agent ] Pipelines running {::
count=>1, :pipelines=>["main"]}
[2018-03-23T03:22:19,269][INFO ][org.logstash.beats.BeatsHandler] [local: 172.177
.0.3:5044, remote: 172.17.0.1:48824] Handling exception: org.logstash.beats.Beatt
sParser$InvalidFrameProtocolException: Invalid Frame Type, received: 84
[2018-03-23T03:22:49,279][INFO ][org.logstash.beats.BeatsHandler] [local: 172.177
.0.3:5044, remote: 172.17.0.1:48826] Handling exception: org.logstash.beats.Beatt
sParser$InvalidFrameProtocolException: Invalid Frame Type, received: 69
Hope these informations are enough to solve the issue, if you require more information please feel free to comment and I will respond to you ASAP