Filebeat log sending failed

Elastic Stack Beats
1

EK + Filebeat

But filebeat log sending to Kibana failed with unknown reason

Here is the conf

==================================
filebeat.inputs:

- type: log
  enabled: true
  paths:
    - /var/log/*.log

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: true

setup.template.settings:
  index.number_of_shards: 1
  index.codec: best_compression
  _source.enabled: true

setup.dashboards.enabled: true
setup.kibana:
  host: "localhost:5601"
  space.id:

output.elasticsearch:
  hosts: ["localhost:9200"]
  protocol: "http"
  username: "elastic"
  password: "NhUYVPIsnPLsfI9e"

processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~
===========================

Here is the partial log:

=================================
2019-04-30T15:41:59.315+0800	INFO	instance/beat.go:280	Setup Beat: filebeat; Version: 7.0.0
2019-04-30T15:41:59.315+0800	INFO	[index-management]	idxmgmt/std.go:165	Set output.elasticsearch.index to 'filebeat-7.0.0' as ILM is enabled.
2019-04-30T15:41:59.317+0800	INFO	elasticsearch/client.go:165	Elasticsearch url: http://localhost:9200
2019-04-30T15:41:59.317+0800	INFO	[publisher]	pipeline/module.go:97	Beat name: Matrix
2019-04-30T15:41:59.318+0800	INFO	elasticsearch/client.go:165	Elasticsearch url: http://localhost:9200
2019-04-30T15:41:59.321+0800	INFO	elasticsearch/client.go:734	Attempting to connect to Elasticsearch version 7.0.0
2019-04-30T15:41:59.348+0800	INFO	add_cloud_metadata/add_cloud_metadata.go:345	add_cloud_metadata: hosting provider type detected as ecs, metadata={"availability_zone":"cn-shanghai-b","instance":{"id":"i-uf65w0dtnuug27oeq6ih"},"provider":"ecs","region":"cn-shanghai"}
2019-04-30T15:41:59.358+0800	INFO	[index-management]	idxmgmt/std.go:223	Auto ILM enable success.
2019-04-30T15:41:59.391+0800	INFO	[index-management]	idxmgmt/std.go:238	ILM policy successfully loaded.
2019-04-30T15:41:59.391+0800	INFO	[index-management]	idxmgmt/std.go:361	Set setup.template.name to '{filebeat-7.0.0 {now/d}-000001}' as ILM is enabled.
2019-04-30T15:41:59.391+0800	INFO	[index-management]	idxmgmt/std.go:366	Set setup.template.pattern to 'filebeat-7.0.0-*' as ILM is enabled.
2019-04-30T15:41:59.391+0800	INFO	[index-management]	idxmgmt/std.go:400	Set settings.index.lifecycle.rollover_alias in template to {filebeat-7.0.0 {now/d}-000001} as ILM is enabled.
2019-04-30T15:41:59.391+0800	INFO	[index-management]	idxmgmt/std.go:404	Set settings.index.lifecycle.name in template to {filebeat-7.0.0 map[policy:{"phases":{"hot":{"actions":{"rollover":{"max_age":"30d","max_size":"50gb"}}}}}]} as ILM is enabled.
2019-04-30T15:41:59.395+0800	INFO	template/load.go:82	Loading template for Elasticsearch version: 7.0.0
2019-04-30T15:41:59.395+0800	INFO	template/load.go:84	Existing template will be overwritten, as overwrite is enabled.
2019-04-30T15:41:59.559+0800	INFO	template/load.go:145	Elasticsearch template with name 'filebeat-7.0.0' loaded
2019-04-30T15:41:59.559+0800	INFO	[index-management]	idxmgmt/std.go:272	Loaded index template.
2019-04-30T15:41:59.560+0800	INFO	[index-management]	idxmgmt/std.go:283	Write alias successfully generated.
2019-04-30T15:41:59.560+0800	INFO	kibana/client.go:118	Kibana url: http://localhost:5601
2019-04-30T15:41:59.731+0800	INFO	kibana/client.go:118	Kibana url: http://localhost:5601
2019-04-30T15:42:35.671+0800	INFO	instance/beat.go:741	Kibana dashboards successfully loaded.
2019-04-30T15:42:35.672+0800	INFO	elasticsearch/client.go:165	Elasticsearch url: http://localhost:9200
2019-04-30T15:42:35.675+0800	INFO	elasticsearch/client.go:734	Attempting to connect to Elasticsearch version 7.0.0
2019-04-30T15:42:35.694+0800	INFO	kibana/client.go:118	Kibana url: http://localhost:5601

===================================
2

Hi @brunoju :slightly_smiling_face:

Please, use markdown formatting so that we can read you configs easily. :wink:

You are not describing which module of Filebeat you are using. Please, also attach the filebeat config. Can you also describe a bit more the problem? You cannot see logs in Kibana but, can you see them in Elasticsearch connecting with a simple HTTP request?

Ensure that a Elasticsearch instance is running in your localhost too. I would also recommend that you change the password of that cluster, because you pasted it here.

3

Hi Mario,:wink:

Thanks a lot for your answer. I have found out the problem.
Just follow the command and open up the debug model to find the root cause:

sudo filebeat -e -d "*"

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.